2024-04-20T01:54:52.308 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T01:54:52.308 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_yuriw@teuthology --description rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} --name yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191 -- /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/orig.config.yaml 2024-04-20T01:54:52.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T01:54:52.449 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191 branch: wip-yuri7-testing-2024-04-18-1351-reef description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} email: yweinste@redhat.com first_in_suite: false job_id: '7664191' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri7-testing-2024-04-18-1351-reef ceph: conf: client: debug monc: 20 debug ms: 1 global: debug ms: 1 debug objecter: 20 debug rados: 20 mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 5 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(TOO_FEW_PGS\) - \(POOL_APP_NOT_ENABLED\) - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_FULL\) - \(SLOW_OPS\) - \(MON_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - slow request sha1: 15eecde2bab9803889d929c7e423d07970d8cbed ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 15eecde2bab9803889d929c7e423d07970d8cbed selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2024-04-18-1351-reef sha1: 15eecde2bab9803889d929c7e423d07970d8cbed owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 6867 sha1: 15eecde2bab9803889d929c7e423d07970d8cbed sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri7-testing-2024-04-18-1351-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 15eecde2bab9803889d929c7e423d07970d8cbed targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8EhPJpNg17ofpe396Fg19EHzoBdX1wmf3XBCqPWWDyrELXEamWW2QFMe3lZkKrzZyhM43plttJP5i42k2gZ9g= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAU+o5zdjLSk5hRO93XP+kIsoKn9Mr6zn7rU+cIPoMs6ca/AQxNR4dTX8Fni+/Fi0gXMqndJPbAxG1yMJidQozQ= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 store_thrash: true thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - rados/test.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_01:10:46 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T01:54:52.449 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa; will attempt to use it 2024-04-20T01:54:52.451 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa/tasks 2024-04-20T01:54:52.451 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T01:54:52.452 INFO:teuthology.task.internal:Checking packages... 2024-04-20T01:54:52.477 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '15eecde2bab9803889d929c7e423d07970d8cbed' 2024-04-20T01:54:52.478 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T01:54:52.478 INFO:teuthology.packaging:ref: None 2024-04-20T01:54:52.478 INFO:teuthology.packaging:tag: None 2024-04-20T01:54:52.478 INFO:teuthology.packaging:branch: wip-yuri7-testing-2024-04-18-1351-reef 2024-04-20T01:54:52.478 INFO:teuthology.packaging:sha1: 15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:54:52.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2024-04-18-1351-reef 2024-04-20T01:54:52.703 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1252.g15eecde2 2024-04-20T01:54:52.705 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T01:54:52.714 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T01:54:52.714 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T01:54:52.724 INFO:teuthology.task.internal:Saving configuration 2024-04-20T01:54:52.738 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T01:54:52.747 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T01:54:52.773 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 01:49:52.284574', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8EhPJpNg17ofpe396Fg19EHzoBdX1wmf3XBCqPWWDyrELXEamWW2QFMe3lZkKrzZyhM43plttJP5i42k2gZ9g='} 2024-04-20T01:54:52.798 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 01:49:52.286190', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAU+o5zdjLSk5hRO93XP+kIsoKn9Mr6zn7rU+cIPoMs6ca/AQxNR4dTX8Fni+/Fi0gXMqndJPbAxG1yMJidQozQ='} 2024-04-20T01:54:52.798 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T01:54:52.805 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-04-20T01:54:52.806 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-04-20T01:54:52.806 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T01:54:52.882 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4b2a0a7f70>, signals=[15]) 2024-04-20T01:54:52.882 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T01:54:52.892 INFO:teuthology.task.internal:Opening connections... 2024-04-20T01:54:52.892 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T01:54:52.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:54:52.975 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T01:54:52.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:54:53.051 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T01:54:53.059 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-20T01:54:53.081 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-20T01:54:53.082 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-20T01:54:53.142 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Red Hat Enterprise Linux" 2024-04-20T01:54:53.142 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8.6 (Ootpa)" 2024-04-20T01:54:53.142 INFO:teuthology.orchestra.run.smithi047.stdout:ID="rhel" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="fedora" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8.6" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.redhat.com/" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-20T01:54:53.143 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-20T01:54:53.144 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-20T01:54:53.164 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-04-20T01:54:53.183 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-04-20T01:54:53.183 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Red Hat Enterprise Linux" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8.6 (Ootpa)" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:ID="rhel" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="fedora" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8.6" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-20T01:54:53.240 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.redhat.com/" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-20T01:54:53.241 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-20T01:54:53.241 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-04-20T01:54:53.265 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T01:54:53.276 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T01:54:53.283 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T01:54:53.284 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-04-20T01:54:53.286 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-04-20T01:54:53.304 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T01:54:53.311 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T01:54:53.311 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-04-20T01:54:53.347 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-04-20T01:54:53.378 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T01:54:53.430 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T01:54:53.446 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-20T01:54:53.447 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-20T01:54:53.447 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T01:54:53.447 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T01:54:53.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:54:53.464 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-04-20T01:54:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:54:53.467 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-20T01:54:53.521 INFO:teuthology.orchestra.run.smithi179.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:54:53.522 INFO:teuthology.task.kernel:Running kernel on smithi179: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:54:53.522 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum install -y kernel 2024-04-20T01:54:53.528 INFO:teuthology.orchestra.run.smithi047.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:54:53.528 INFO:teuthology.task.kernel:Running kernel on smithi047: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:54:53.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-04-20T01:54:55.062 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T01:54:55.062 INFO:teuthology.orchestra.run.smithi179.stdout:Unable to read consumer identity 2024-04-20T01:54:55.156 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T01:54:55.156 INFO:teuthology.orchestra.run.smithi047.stdout:Unable to read consumer identity 2024-04-20T01:54:55.373 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:54:55.373 INFO:teuthology.orchestra.run.smithi179.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:54:55.373 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:54:55.524 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:54:55.525 INFO:teuthology.orchestra.run.smithi047.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:54:55.525 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:54:57.277 INFO:teuthology.orchestra.run.smithi047.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-04-20T01:54:57.295 INFO:teuthology.orchestra.run.smithi179.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2024-04-20T01:54:59.021 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 16 MB 00:01 2024-04-20T01:55:02.612 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 496 kB/s | 24 kB 00:00 2024-04-20T01:55:03.348 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-20T01:55:03.348 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-20T01:55:03.428 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T01:55:03.429 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T01:55:03.429 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T01:55:03.566 DEBUG:teuthology.orchestra.run.smithi179:> echo no | sudo yum reinstall kernel || true 2024-04-20T01:55:03.900 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T01:55:03.900 INFO:teuthology.orchestra.run.smithi179.stdout:Unable to read consumer identity 2024-04-20T01:55:04.107 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:55:04.108 INFO:teuthology.orchestra.run.smithi179.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:55:04.108 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:55:04.420 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:55:02 AM UTC. 2024-04-20T01:55:04.492 INFO:teuthology.orchestra.run.smithi179.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:55:04.493 INFO:teuthology.orchestra.run.smithi179.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:55:04.540 INFO:teuthology.orchestra.run.smithi179.stderr:Error: No packages marked for reinstall. 2024-04-20T01:55:04.588 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum reinstall -y kernel || true 2024-04-20T01:55:04.922 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T01:55:04.922 INFO:teuthology.orchestra.run.smithi179.stdout:Unable to read consumer identity 2024-04-20T01:55:05.165 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:55:05.165 INFO:teuthology.orchestra.run.smithi179.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:55:05.165 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T01:55:05.470 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:03 ago on Sat 20 Apr 2024 01:55:02 AM UTC. 2024-04-20T01:55:05.533 INFO:teuthology.orchestra.run.smithi179.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:55:05.533 INFO:teuthology.orchestra.run.smithi179.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:55:05.566 INFO:teuthology.orchestra.run.smithi179.stderr:Error: No packages marked for reinstall. 2024-04-20T01:55:05.613 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T01:55:05.781 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:55:05.782 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:55:05.782 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T01:55:05.782 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-04-20T01:55:05.782 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T01:55:05.833 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-04-20T01:55:05.846 DEBUG:teuthology.parallel:result is None 2024-04-20T01:55:13.187 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2024-04-20T01:55:17.285 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 274 kB/s | 24 kB 00:00 2024-04-20T01:55:18.121 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-20T01:55:18.122 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-20T01:55:18.202 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T01:55:18.203 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T01:55:18.204 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T01:55:18.496 DEBUG:teuthology.orchestra.run.smithi047:> echo no | sudo yum reinstall kernel || true 2024-04-20T01:55:18.887 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T01:55:18.887 INFO:teuthology.orchestra.run.smithi047.stdout:Unable to read consumer identity 2024-04-20T01:55:19.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:55:19.122 INFO:teuthology.orchestra.run.smithi047.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:55:19.122 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:55:19.494 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:55:17 AM UTC. 2024-04-20T01:55:19.583 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:55:19.592 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:55:19.638 INFO:teuthology.orchestra.run.smithi047.stderr:Error: No packages marked for reinstall. 2024-04-20T01:55:19.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum reinstall -y kernel || true 2024-04-20T01:55:20.075 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T01:55:20.075 INFO:teuthology.orchestra.run.smithi047.stdout:Unable to read consumer identity 2024-04-20T01:55:20.326 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:55:20.326 INFO:teuthology.orchestra.run.smithi047.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:55:20.326 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T01:55:20.688 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:03 ago on Sat 20 Apr 2024 01:55:17 AM UTC. 2024-04-20T01:55:20.762 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:55:20.763 INFO:teuthology.orchestra.run.smithi047.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:55:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:Error: No packages marked for reinstall. 2024-04-20T01:55:20.858 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T01:55:21.042 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:55:21.042 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:55:21.042 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T01:55:21.042 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-20T01:55:21.042 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T01:55:21.090 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-04-20T01:55:21.103 DEBUG:teuthology.parallel:result is None 2024-04-20T01:55:21.103 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T01:55:21.111 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T01:55:21.112 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T01:55:21.114 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T01:55:21.137 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T01:55:21.146 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T01:55:21.162 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T01:55:21.162 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T01:55:21.174 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T01:55:21.209 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T01:55:21.221 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T01:55:21.221 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T01:55:21.247 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T01:55:21.281 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:55:21.287 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:55:21.292 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:55:21.300 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:55:21.302 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T01:55:21.311 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T01:55:21.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T01:55:21.344 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T01:55:21.377 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T01:55:21.389 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T01:55:21.389 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T01:55:21.420 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T01:55:21.440 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:55:21.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:55:21.584 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:55:21.673 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:55:21.723 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T01:55:21.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T01:55:21.798 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:55:21.856 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:55:21.904 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:55:21.987 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:55:22.036 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T01:55:22.037 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T01:55:22.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-04-20T01:55:22.111 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-04-20T01:55:22.167 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:55:22.210 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:55:22.634 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T01:55:22.643 INFO:teuthology.task.internal:Starting timer... 2024-04-20T01:55:22.644 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T01:55:22.662 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T01:55:22.674 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-20T01:55:22.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-04-20T01:55:22.733 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-04-20T01:55:22.736 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-04-20T01:55:22.800 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-04-20T01:55:22.801 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T01:55:22.802 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-04-20T01:55:22.840 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-04-20T01:55:22.841 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-04-20T01:55:22.868 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-04-20T01:55:22.869 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-04-20T01:55:22.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:55:22.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:55:22.912 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:55:22.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:55:22.940 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T01:55:22.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-04-20T01:55:22.983 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-04-20T01:55:23.014 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T01:55:23.027 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T01:55:23.044 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T01:55:23.045 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T02:06:06.001 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-04-20T02:06:06.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T02:06:06.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T02:06:06.086 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-20T02:06:06.160 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T02:06:06.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-20T02:06:06.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T02:06:06.237 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-04-20T02:06:06.305 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-20T02:06:06.305 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T02:06:06.316 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T02:06:06.317 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T02:06:06.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T02:06:06.320 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T02:06:06.320 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T02:06:06.357 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T02:06:06.374 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T02:06:06.378 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T02:06:06.393 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T02:06:06.456 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-04-20T02:06:06.470 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-04-20T02:06:06.474 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-04-20T02:06:06.485 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-04-20T02:06:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T02:06:06.500 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T02:06:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T02:06:06.514 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T02:06:06.594 INFO:teuthology.orchestra.run.smithi179.stderr:bash: ntpq: command not found 2024-04-20T02:06:06.598 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T02:06:06.598 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-04-20T02:06:06.599 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.599 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.599 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.599 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.600 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2024-04-20T02:06:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T02:06:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-20T02:06:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.604 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.605 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:06:06.605 INFO:teuthology.run_tasks:Running task install... 2024-04-20T02:06:06.614 DEBUG:teuthology.task.install:project ceph 2024-04-20T02:06:06.614 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}} 2024-04-20T02:06:06.614 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'} 2024-04-20T02:06:06.614 INFO:teuthology.task.install:Using flavor: default 2024-04-20T02:06:06.620 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T02:06:06.620 INFO:teuthology.task.install:extra packages: [] 2024-04-20T02:06:06.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'tag': None, 'wait_for_package': False} 2024-04-20T02:06:06.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T02:06:06.624 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'tag': None, 'wait_for_package': False} 2024-04-20T02:06:06.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T02:06:06.775 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/ 2024-04-20T02:06:06.776 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1252.g15eecde2 2024-04-20T02:06:06.848 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/ 2024-04-20T02:06:06.848 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1252.g15eecde2 2024-04-20T02:06:06.963 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T02:06:06.964 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:06:06.964 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T02:06:06.994 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T02:06:06.994 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/15eecde2bab9803889d929c7e423d07970d8cbed/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T02:06:07.055 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T02:06:07.055 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:06:07.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T02:06:07.063 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T02:06:07.090 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T02:06:07.091 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/15eecde2bab9803889d929c7e423d07970d8cbed/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T02:06:07.143 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T02:06:07.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T02:06:07.209 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-04-20T02:06:07.211 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-04-20T02:06:07.255 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T02:06:07.286 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-04-20T02:06:07.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-04-20T02:06:07.585 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:06:07.722 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:06:09.123 INFO:teuthology.orchestra.run.smithi047.stdout:63 files removed 2024-04-20T02:06:09.165 INFO:teuthology.orchestra.run.smithi179.stdout:63 files removed 2024-04-20T02:06:09.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw 2024-04-20T02:06:09.217 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw 2024-04-20T02:06:09.552 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:06:09.562 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:06:11.246 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 192 kB/s | 80 kB 00:00 2024-04-20T02:06:11.555 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 42 kB/s | 11 kB 00:00 2024-04-20T02:06:11.796 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 147 kB/s | 80 kB 00:00 2024-04-20T02:06:12.097 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 2.9 kB/s | 1.5 kB 00:00 2024-04-20T02:06:12.292 INFO:teuthology.orchestra.run.smithi179.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-04-20T02:06:12.463 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-04-20T02:06:12.755 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2024-04-20T02:06:12.923 INFO:teuthology.orchestra.run.smithi047.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-04-20T02:06:13.799 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-04-20T02:06:14.467 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-04-20T02:06:17.382 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 390 kB/s | 24 kB 00:00 2024-04-20T02:06:18.507 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 313 kB/s | 24 kB 00:00 2024-04-20T02:06:18.784 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 67 MB 00:01 2024-04-20T02:06:19.208 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-04-20T02:06:22.243 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 67 MB 00:01 2024-04-20T02:06:29.743 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 60 MB 00:01 2024-04-20T02:06:34.310 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 60 MB 00:01 2024-04-20T02:06:40.535 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 17 MB/s | 8.8 MB 00:00 2024-04-20T02:06:47.880 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:06:47.882 INFO:teuthology.orchestra.run.smithi179.stdout:==================================================================================================== 2024-04-20T02:06:47.882 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:06:47.882 INFO:teuthology.orchestra.run.smithi179.stdout:==================================================================================================== 2024-04-20T02:06:47.882 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 13 M 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.7 M 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.6 M 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 5.5 M 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 22 M 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 24 k 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 781 k 2024-04-20T02:06:47.883 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 474 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.5 M 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 45 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 129 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 197 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 372 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 364 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 107 k 2024-04-20T02:06:47.884 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout:==================================================================================================== 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout:Install 21 Packages 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 2 Packages 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 59 M 2024-04-20T02:06:47.885 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:06:49.561 INFO:teuthology.orchestra.run.smithi179.stdout:(1/23): ceph-base-18.2.2-1252.g15eecde2.el8.x86 4.3 MB/s | 5.5 MB 00:01 2024-04-20T02:06:49.644 INFO:teuthology.orchestra.run.smithi179.stdout:(2/23): ceph-selinux-18.2.2-1252.g15eecde2.el8. 293 kB/s | 24 kB 00:00 2024-04-20T02:06:49.811 INFO:teuthology.orchestra.run.smithi179.stdout:(3/23): ceph-radosgw-18.2.2-1252.g15eecde2.el8. 8.5 MB/s | 13 MB 00:01 2024-04-20T02:06:49.845 INFO:teuthology.orchestra.run.smithi179.stdout:(4/23): libcephfs2-18.2.2-1252.g15eecde2.el8.x8 3.8 MB/s | 781 kB 00:00 2024-04-20T02:06:49.879 INFO:teuthology.orchestra.run.smithi179.stdout:(5/23): libradosstriper1-18.2.2-1252.g15eecde2. 6.9 MB/s | 474 kB 00:00 2024-04-20T02:06:49.937 INFO:teuthology.orchestra.run.smithi179.stdout:(6/23): python3-ceph-argparse-18.2.2-1252.g15ee 765 kB/s | 45 kB 00:00 2024-04-20T02:06:50.005 INFO:teuthology.orchestra.run.smithi179.stdout:(7/23): python3-ceph-common-18.2.2-1252.g15eecd 1.9 MB/s | 129 kB 00:00 2024-04-20T02:06:50.080 INFO:teuthology.orchestra.run.smithi179.stdout:(8/23): python3-cephfs-18.2.2-1252.g15eecde2.el 2.6 MB/s | 197 kB 00:00 2024-04-20T02:06:50.164 INFO:teuthology.orchestra.run.smithi179.stdout:(9/23): python3-rados-18.2.2-1252.g15eecde2.el8 4.4 MB/s | 372 kB 00:00 2024-04-20T02:06:50.255 INFO:teuthology.orchestra.run.smithi179.stdout:(10/23): python3-rbd-18.2.2-1252.g15eecde2.el8. 3.9 MB/s | 364 kB 00:00 2024-04-20T02:06:50.347 INFO:teuthology.orchestra.run.smithi179.stdout:(11/23): python3-rgw-18.2.2-1252.g15eecde2.el8. 1.1 MB/s | 107 kB 00:00 2024-04-20T02:06:50.656 INFO:teuthology.orchestra.run.smithi179.stdout:(12/23): librgw2-18.2.2-1252.g15eecde2.el8.x86_ 8.1 MB/s | 6.5 MB 00:00 2024-04-20T02:06:50.806 INFO:teuthology.orchestra.run.smithi179.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 393 kB/s | 59 kB 00:00 2024-04-20T02:06:51.048 INFO:teuthology.orchestra.run.smithi179.stdout:(14/23): ceph-common-18.2.2-1252.g15eecde2.el8. 7.8 MB/s | 22 MB 00:02 2024-04-20T02:06:51.073 INFO:teuthology.orchestra.run.smithi179.stdout:(15/23): gperftools-libs-2.7-9.el8.x86_64.rpm 422 kB/s | 306 kB 00:00 2024-04-20T02:06:51.241 INFO:teuthology.orchestra.run.smithi179.stdout:(16/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 282 kB/s | 47 kB 00:00 2024-04-20T02:06:51.266 INFO:teuthology.orchestra.run.smithi179.stdout:(17/23): libunwind-1.3.1-3.el8.x86_64.rpm 163 kB/s | 75 kB 00:00 2024-04-20T02:06:51.349 INFO:teuthology.orchestra.run.smithi179.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 405 kB/s | 44 kB 00:00 2024-04-20T02:06:51.460 INFO:teuthology.orchestra.run.smithi179.stdout:(19/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.1 MB/s | 354 kB 00:00 2024-04-20T02:06:51.483 INFO:teuthology.orchestra.run.smithi179.stdout:(20/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-04-20T02:06:51.566 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:06:51.566 INFO:teuthology.orchestra.run.smithi047.stdout:==================================================================================================== 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout:==================================================================================================== 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 13 M 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.7 M 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.6 M 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 5.5 M 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 22 M 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 24 k 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 781 k 2024-04-20T02:06:51.567 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 474 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.5 M 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 45 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 129 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 197 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 372 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 364 k 2024-04-20T02:06:51.568 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 107 k 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout:==================================================================================================== 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout:Install 21 Packages 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 2 Packages 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 59 M 2024-04-20T02:06:51.569 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:06:52.350 INFO:teuthology.orchestra.run.smithi179.stdout:(21/23): librbd1-18.2.2-1252.g15eecde2.el8.x86_ 4.1 MB/s | 3.6 MB 00:00 2024-04-20T02:06:52.634 INFO:teuthology.orchestra.run.smithi179.stdout:(22/23): librados2-18.2.2-1252.g15eecde2.el8.x8 3.1 MB/s | 3.7 MB 00:01 2024-04-20T02:06:52.759 INFO:teuthology.orchestra.run.smithi179.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.0 MB/s | 1.7 MB 00:01 2024-04-20T02:06:52.761 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:06:52.762 INFO:teuthology.orchestra.run.smithi179.stdout:Total 12 MB/s | 59 MB 00:04 2024-04-20T02:06:52.809 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:06:52.854 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:06:52.854 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:06:53.110 INFO:teuthology.orchestra.run.smithi047.stdout:(1/23): ceph-base-18.2.2-1252.g15eecde2.el8.x86 4.0 MB/s | 5.5 MB 00:01 2024-04-20T02:06:53.161 INFO:teuthology.orchestra.run.smithi047.stdout:(2/23): ceph-selinux-18.2.2-1252.g15eecde2.el8. 489 kB/s | 24 kB 00:00 2024-04-20T02:06:53.185 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:06:53.194 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:06:53.286 INFO:teuthology.orchestra.run.smithi047.stdout:(3/23): libcephfs2-18.2.2-1252.g15eecde2.el8.x8 6.1 MB/s | 781 kB 00:00 2024-04-20T02:06:53.411 INFO:teuthology.orchestra.run.smithi047.stdout:(4/23): libradosstriper1-18.2.2-1252.g15eecde2. 3.7 MB/s | 474 kB 00:00 2024-04-20T02:06:54.245 INFO:teuthology.orchestra.run.smithi047.stdout:(5/23): ceph-radosgw-18.2.2-1252.g15eecde2.el8. 5.2 MB/s | 13 MB 00:02 2024-04-20T02:06:54.345 INFO:teuthology.orchestra.run.smithi047.stdout:(6/23): python3-ceph-argparse-18.2.2-1252.g15ee 448 kB/s | 45 kB 00:00 2024-04-20T02:06:54.470 INFO:teuthology.orchestra.run.smithi047.stdout:(7/23): librgw2-18.2.2-1252.g15eecde2.el8.x86_6 6.2 MB/s | 6.5 MB 00:01 2024-04-20T02:06:54.495 INFO:teuthology.orchestra.run.smithi047.stdout:(8/23): python3-ceph-common-18.2.2-1252.g15eecd 858 kB/s | 129 kB 00:00 2024-04-20T02:06:54.590 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:06:54.738 INFO:teuthology.orchestra.run.smithi047.stdout:(9/23): ceph-common-18.2.2-1252.g15eecde2.el8.x 7.2 MB/s | 22 MB 00:03 2024-04-20T02:06:54.763 INFO:teuthology.orchestra.run.smithi047.stdout:(10/23): python3-cephfs-18.2.2-1252.g15eecde2.e 674 kB/s | 197 kB 00:00 2024-04-20T02:06:54.767 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-20T02:06:54.788 INFO:teuthology.orchestra.run.smithi047.stdout:(11/23): python3-rados-18.2.2-1252.g15eecde2.el 1.2 MB/s | 372 kB 00:00 2024-04-20T02:06:54.822 INFO:teuthology.orchestra.run.smithi047.stdout:(12/23): python3-rbd-18.2.2-1252.g15eecde2.el8. 4.2 MB/s | 364 kB 00:00 2024-04-20T02:06:54.839 INFO:teuthology.orchestra.run.smithi047.stdout:(13/23): python3-rgw-18.2.2-1252.g15eecde2.el8. 1.4 MB/s | 107 kB 00:00 2024-04-20T02:06:54.942 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T02:06:55.072 INFO:teuthology.orchestra.run.smithi047.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 235 kB/s | 59 kB 00:00 2024-04-20T02:06:55.089 INFO:teuthology.orchestra.run.smithi047.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 300 kB/s | 75 kB 00:00 2024-04-20T02:06:55.198 INFO:teuthology.orchestra.run.smithi047.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 748 kB/s | 306 kB 00:00 2024-04-20T02:06:55.214 INFO:teuthology.orchestra.run.smithi047.stdout:(17/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 376 kB/s | 47 kB 00:00 2024-04-20T02:06:55.290 INFO:teuthology.orchestra.run.smithi047.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2024-04-20T02:06:55.337 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T02:06:55.377 INFO:teuthology.orchestra.run.smithi047.stdout:(19/23): python3-prettytable-0.7.2-14.el8.noarc 246 kB/s | 44 kB 00:00 2024-04-20T02:06:55.415 INFO:teuthology.orchestra.run.smithi047.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-04-20T02:06:55.610 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T02:06:55.674 INFO:teuthology.orchestra.run.smithi047.stdout:(21/23): librbd1-18.2.2-1252.g15eecde2.el8.x86_ 14 MB/s | 3.6 MB 00:00 2024-04-20T02:06:55.744 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T02:06:55.749 INFO:teuthology.orchestra.run.smithi047.stdout:(22/23): thrift-0.13.0-2.el8.x86_64.rpm 2.6 MB/s | 1.7 MB 00:00 2024-04-20T02:06:55.932 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T02:06:55.999 INFO:teuthology.orchestra.run.smithi047.stdout:(23/23): librados2-18.2.2-1252.g15eecde2.el8.x8 5.9 MB/s | 3.7 MB 00:00 2024-04-20T02:06:56.002 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:06:56.002 INFO:teuthology.orchestra.run.smithi047.stdout:Total 13 MB/s | 59 MB 00:04 2024-04-20T02:06:56.056 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:06:56.107 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:06:56.107 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:06:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2024-04-20T02:06:56.475 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:06:56.484 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T02:06:56.485 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:06:56.863 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T02:06:57.067 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T02:06:57.142 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T02:06:57.301 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-20T02:06:57.365 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-20T02:06:57.574 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 9/25 2024-04-20T02:06:57.770 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 10/25 2024-04-20T02:06:57.860 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:06:57.969 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T02:06:58.072 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-20T02:06:58.200 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T02:06:58.226 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T02:06:58.434 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T02:06:58.505 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T02:06:58.640 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T02:06:58.684 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2024-04-20T02:06:58.917 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2024-04-20T02:06:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T02:06:59.103 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T02:06:59.265 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T02:06:59.522 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2024-04-20T02:06:59.725 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T02:07:00.158 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T02:07:00.217 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2024-04-20T02:07:00.425 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T02:07:00.433 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T02:07:00.495 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T02:07:00.528 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T02:07:00.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-20T02:07:00.783 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-20T02:07:00.784 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 18/25 2024-04-20T02:07:00.958 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 9/25 2024-04-20T02:07:00.968 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 19/25 2024-04-20T02:07:01.130 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 10/25 2024-04-20T02:07:01.323 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T02:07:01.580 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T02:07:01.758 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T02:07:01.823 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T02:07:01.935 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2024-04-20T02:07:02.155 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2024-04-20T02:07:02.997 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2024-04-20T02:07:03.175 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T02:07:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T02:07:03.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 18/25 2024-04-20T02:07:03.817 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 19/25 2024-04-20T02:07:04.677 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T02:07:04.933 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T02:07:05.524 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T02:07:05.820 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T02:07:05.888 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T02:07:06.111 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T02:07:07.957 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T02:07:08.192 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T02:07:08.851 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T02:07:09.117 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T02:07:09.209 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T02:07:09.367 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T02:07:25.980 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T02:07:25.980 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-04-20T02:07:25.980 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-04-20T02:07:25.980 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /dev 2024-04-20T02:07:25.980 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /run 2024-04-20T02:07:25.980 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-04-20T02:07:25.981 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-04-20T02:07:25.981 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-04-20T02:07:25.981 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-04-20T02:07:25.981 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /dev 2024-04-20T02:07:25.981 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:07:27.326 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T02:07:27.346 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T02:07:27.347 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T02:07:27.624 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T02:07:27.624 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T02:07:28.385 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T02:07:28.385 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 1/25 2024-04-20T02:07:28.385 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 5/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 6/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 8/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 9/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 10/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 13/25 2024-04-20T02:07:28.386 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 18/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-20T02:07:28.387 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 24/25 2024-04-20T02:07:31.641 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /run 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /dev 2024-04-20T02:07:31.642 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:07:33.097 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T02:07:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T02:07:33.124 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T02:07:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T02:07:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 1/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 5/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 6/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T02:07:34.064 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 8/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 9/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 10/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 13/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 18/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-20T02:07:34.065 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 24/25 2024-04-20T02:07:39.819 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T02:07:39.819 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:07:40.374 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:07:40.374 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-04-20T02:07:40.374 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.374 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.374 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:07:40.374 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.375 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:07:40.376 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:07:40.583 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-test 2024-04-20T02:07:40.938 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:07:42.947 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 4.1 kB 00:00 2024-04-20T02:07:43.323 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T02:07:46.026 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 47 M 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 142 k 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-20T02:07:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout:Install 6 Packages 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 48 M 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 213 M 2024-04-20T02:07:46.028 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:07:46.071 INFO:teuthology.orchestra.run.smithi179.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-04-20T02:07:46.278 INFO:teuthology.orchestra.run.smithi179.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-04-20T02:07:46.413 INFO:teuthology.orchestra.run.smithi179.stdout:(3/6): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-04-20T02:07:46.505 INFO:teuthology.orchestra.run.smithi179.stdout:(4/6): libcephsqlite-18.2.2-1252.g15eecde2.el8. 298 kB/s | 142 kB 00:00 2024-04-20T02:07:46.580 INFO:teuthology.orchestra.run.smithi179.stdout:(5/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-20T02:07:46.767 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T02:07:46.767 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:07:47.389 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:07:47.389 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-04-20T02:07:47.389 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.389 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.389 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:07:47.389 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-20T02:07:47.390 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:07:47.391 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:07:47.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-test 2024-04-20T02:07:48.047 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:07:50.237 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-20T02:07:50.581 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 kB/s | 4.5 kB 00:00 2024-04-20T02:07:51.572 INFO:teuthology.orchestra.run.smithi179.stdout:(6/6): ceph-test-18.2.2-1252.g15eecde2.el8.x86_ 8.5 MB/s | 47 MB 00:05 2024-04-20T02:07:51.574 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:07:51.574 INFO:teuthology.orchestra.run.smithi179.stdout:Total 8.7 MB/s | 48 MB 00:05 2024-04-20T02:07:51.611 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:07:51.633 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:07:51.633 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:07:51.900 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:07:51.909 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:07:52.396 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:07:52.552 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T02:07:52.657 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T02:07:52.913 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-20T02:07:53.077 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-20T02:07:53.314 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-20T02:07:53.341 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 47 M 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:07:53.342 INFO:teuthology.orchestra.run.smithi047.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 142 k 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout:============================================================================================ 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout:Install 6 Packages 2024-04-20T02:07:53.343 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:07:53.344 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 48 M 2024-04-20T02:07:53.344 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 213 M 2024-04-20T02:07:53.344 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:07:53.493 INFO:teuthology.orchestra.run.smithi047.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 470 kB/s | 70 kB 00:00 2024-04-20T02:07:53.509 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T02:07:53.701 INFO:teuthology.orchestra.run.smithi047.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-04-20T02:07:53.759 INFO:teuthology.orchestra.run.smithi047.stdout:(3/6): libcephsqlite-18.2.2-1252.g15eecde2.el8. 342 kB/s | 142 kB 00:00 2024-04-20T02:07:53.860 INFO:teuthology.orchestra.run.smithi047.stdout:(4/6): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2024-04-20T02:07:53.885 INFO:teuthology.orchestra.run.smithi047.stdout:(5/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-20T02:07:56.043 INFO:teuthology.orchestra.run.smithi047.stdout:(6/6): ceph-test-18.2.2-1252.g15eecde2.el8.x86_ 17 MB/s | 47 MB 00:02 2024-04-20T02:07:56.045 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:07:56.046 INFO:teuthology.orchestra.run.smithi047.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-20T02:07:56.086 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:07:56.110 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:07:56.110 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:07:56.407 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:07:56.417 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:07:56.916 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:07:57.077 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T02:07:57.166 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T02:07:57.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-20T02:07:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-20T02:07:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-20T02:07:57.859 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T02:07:57.999 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T02:07:58.298 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T02:07:58.564 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T02:07:58.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 1/6 2024-04-20T02:07:58.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2/6 2024-04-20T02:07:58.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-20T02:07:58.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-20T02:07:58.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-20T02:08:02.803 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T02:08:03.066 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T02:08:03.366 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T02:08:03.366 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 1/6 2024-04-20T02:08:03.366 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2/6 2024-04-20T02:08:03.366 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-20T02:08:03.366 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-20T02:08:03.366 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-20T02:08:12.277 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-20T02:08:12.278 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:08:12.826 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:12.826 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:08:12.826 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:12.826 INFO:teuthology.orchestra.run.smithi179.stdout: jq-1.6-7.el8.x86_64 2024-04-20T02:08:12.826 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:12.827 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-20T02:08:12.827 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-20T02:08:12.827 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-20T02:08:12.827 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:12.827 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:08:13.023 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph 2024-04-20T02:08:13.362 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:08:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-20T02:08:15.250 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:08:15.407 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:08:15.800 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout: jq-1.6-7.el8.x86_64 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-20T02:08:15.874 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-20T02:08:15.875 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-20T02:08:15.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:08:15.875 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:08:16.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph 2024-04-20T02:08:16.462 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:08:18.425 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:08:18.428 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================= 2024-04-20T02:08:18.428 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:08:18.428 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================= 2024-04-20T02:08:18.428 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:08:18.428 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.4 k 2024-04-20T02:08:18.428 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 2.3 M 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 1.6 M 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 246 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 4.8 M 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 18 M 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-20T02:08:18.429 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-20T02:08:18.430 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:08:18.431 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================= 2024-04-20T02:08:18.432 INFO:teuthology.orchestra.run.smithi179.stdout:Install 32 Packages 2024-04-20T02:08:18.432 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:18.432 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 31 M 2024-04-20T02:08:18.432 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 124 M 2024-04-20T02:08:18.432 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:08:18.498 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-20T02:08:18.880 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-04-20T02:08:19.148 INFO:teuthology.orchestra.run.smithi179.stdout:(1/32): ceph-18.2.2-1252.g15eecde2.el8.x86_64.r 17 kB/s | 6.4 kB 00:00 2024-04-20T02:08:19.648 INFO:teuthology.orchestra.run.smithi179.stdout:(2/32): ceph-mgr-18.2.2-1252.g15eecde2.el8.x86_ 1.8 MB/s | 1.6 MB 00:00 2024-04-20T02:08:19.806 INFO:teuthology.orchestra.run.smithi179.stdout:(3/32): ceph-mds-18.2.2-1252.g15eecde2.el8.x86_ 2.2 MB/s | 2.3 MB 00:01 2024-04-20T02:08:19.873 INFO:teuthology.orchestra.run.smithi179.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1252.g15ee 3.6 MB/s | 246 kB 00:00 2024-04-20T02:08:20.196 INFO:teuthology.orchestra.run.smithi179.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 135 kB/s | 44 kB 00:00 2024-04-20T02:08:20.424 INFO:teuthology.orchestra.run.smithi179.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 812 kB/s | 185 kB 00:00 2024-04-20T02:08:20.524 INFO:teuthology.orchestra.run.smithi179.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.7 MB/s | 173 kB 00:00 2024-04-20T02:08:20.658 INFO:teuthology.orchestra.run.smithi179.stdout:(8/32): ceph-mon-18.2.2-1252.g15eecde2.el8.x86_ 3.2 MB/s | 4.8 MB 00:01 2024-04-20T02:08:20.725 INFO:teuthology.orchestra.run.smithi179.stdout:(9/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 1.9 MB/s | 384 kB 00:00 2024-04-20T02:08:20.825 INFO:teuthology.orchestra.run.smithi179.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 176 kB/s | 18 kB 00:00 2024-04-20T02:08:20.850 INFO:teuthology.orchestra.run.smithi179.stdout:(11/32): python3-jaraco-6.2-6.el8.noarch.rpm 58 kB/s | 11 kB 00:00 2024-04-20T02:08:20.917 INFO:teuthology.orchestra.run.smithi179.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 532 kB/s | 49 kB 00:00 2024-04-20T02:08:21.041 INFO:teuthology.orchestra.run.smithi179.stdout:(13/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2024-04-20T02:08:21.076 INFO:teuthology.orchestra.run.smithi179.stdout:(14/32): python3-more-itertools-7.2.0-3.el8.noa 260 kB/s | 59 kB 00:00 2024-04-20T02:08:21.135 INFO:teuthology.orchestra.run.smithi179.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 166 kB/s | 16 kB 00:00 2024-04-20T02:08:21.169 INFO:teuthology.orchestra.run.smithi179.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 203 kB/s | 19 kB 00:00 2024-04-20T02:08:21.227 INFO:teuthology.orchestra.run.smithi179.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 260 kB/s | 24 kB 00:00 2024-04-20T02:08:21.261 INFO:teuthology.orchestra.run.smithi179.stdout:(18/32): python3-tempora-1.14.1-5.el8.noarch.rp 318 kB/s | 29 kB 00:00 2024-04-20T02:08:21.353 INFO:teuthology.orchestra.run.smithi179.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 220 kB/s | 27 kB 00:00 2024-04-20T02:08:21.462 INFO:teuthology.orchestra.run.smithi179.stdout:(20/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.3 MB/s | 251 kB 00:00 2024-04-20T02:08:21.495 INFO:teuthology.orchestra.run.smithi179.stdout:(21/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.1 MB/s | 255 kB 00:00 2024-04-20T02:08:21.554 INFO:teuthology.orchestra.run.smithi179.stdout:(22/32): python3-webtest-2.0.33-1.el8.noarch.rp 931 kB/s | 85 kB 00:00 2024-04-20T02:08:21.629 INFO:teuthology.orchestra.run.smithi179.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 171 kB/s | 23 kB 00:00 2024-04-20T02:08:21.633 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================= 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================= 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.4 k 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 2.3 M 2024-04-20T02:08:21.637 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 1.6 M 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 246 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 4.8 M 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 18 M 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-20T02:08:21.638 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-20T02:08:21.639 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================= 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout:Install 32 Packages 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:08:21.640 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 31 M 2024-04-20T02:08:21.641 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 124 M 2024-04-20T02:08:21.641 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:08:21.846 INFO:teuthology.orchestra.run.smithi179.stdout:(24/32): ceph-osd-18.2.2-1252.g15eecde2.el8.x86 8.4 MB/s | 18 MB 00:02 2024-04-20T02:08:21.897 INFO:teuthology.orchestra.run.smithi179.stdout:(25/32): python3-pycparser-2.14-14.el8.noarch.r 319 kB/s | 109 kB 00:00 2024-04-20T02:08:21.930 INFO:teuthology.orchestra.run.smithi179.stdout:(26/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 806 kB/s | 238 kB 00:00 2024-04-20T02:08:21.962 INFO:teuthology.orchestra.run.smithi047.stdout:(1/32): ceph-18.2.2-1252.g15eecde2.el8.x86_64.r 46 kB/s | 6.4 kB 00:00 2024-04-20T02:08:22.030 INFO:teuthology.orchestra.run.smithi179.stdout:(27/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.5 MB/s | 457 kB 00:00 2024-04-20T02:08:22.056 INFO:teuthology.orchestra.run.smithi179.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 310 kB/s | 39 kB 00:00 2024-04-20T02:08:22.089 INFO:teuthology.orchestra.run.smithi179.stdout:(29/32): python3-cryptography-3.2.1-7.el8_9.x86 2.3 MB/s | 559 kB 00:00 2024-04-20T02:08:22.131 INFO:teuthology.orchestra.run.smithi179.stdout:(30/32): python3-mako-1.0.6-14.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2024-04-20T02:08:22.137 INFO:teuthology.orchestra.run.smithi047.stdout:(2/32): ceph-mgr-18.2.2-1252.g15eecde2.el8.x86_ 5.0 MB/s | 1.6 MB 00:00 2024-04-20T02:08:22.179 INFO:teuthology.orchestra.run.smithi047.stdout:(3/32): ceph-mds-18.2.2-1252.g15eecde2.el8.x86_ 6.4 MB/s | 2.3 MB 00:00 2024-04-20T02:08:22.190 INFO:teuthology.orchestra.run.smithi179.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 650 kB/s | 103 kB 00:00 2024-04-20T02:08:22.223 INFO:teuthology.orchestra.run.smithi179.stdout:(32/32): python3-pytz-2017.2-11.el8.noarch.rpm 405 kB/s | 54 kB 00:00 2024-04-20T02:08:22.225 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:08:22.226 INFO:teuthology.orchestra.run.smithi179.stdout:Total 8.1 MB/s | 31 MB 00:03 2024-04-20T02:08:22.271 INFO:teuthology.orchestra.run.smithi047.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1252.g15ee 2.6 MB/s | 246 kB 00:00 2024-04-20T02:08:22.319 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:08:22.356 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:08:22.357 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:08:22.413 INFO:teuthology.orchestra.run.smithi047.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 308 kB/s | 44 kB 00:00 2024-04-20T02:08:22.488 INFO:teuthology.orchestra.run.smithi047.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 2.4 MB/s | 185 kB 00:00 2024-04-20T02:08:22.539 INFO:teuthology.orchestra.run.smithi047.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 3.4 MB/s | 173 kB 00:00 2024-04-20T02:08:22.582 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:08:22.591 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:08:22.597 INFO:teuthology.orchestra.run.smithi047.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 6.5 MB/s | 384 kB 00:00 2024-04-20T02:08:22.639 INFO:teuthology.orchestra.run.smithi047.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2024-04-20T02:08:22.681 INFO:teuthology.orchestra.run.smithi047.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 422 kB/s | 18 kB 00:00 2024-04-20T02:08:22.740 INFO:teuthology.orchestra.run.smithi047.stdout:(11/32): python3-logutils-0.3.5-11.el8.noarch.r 838 kB/s | 49 kB 00:00 2024-04-20T02:08:22.782 INFO:teuthology.orchestra.run.smithi047.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 1.4 MB/s | 59 kB 00:00 2024-04-20T02:08:22.832 INFO:teuthology.orchestra.run.smithi047.stdout:(13/32): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2024-04-20T02:08:22.874 INFO:teuthology.orchestra.run.smithi047.stdout:(14/32): python3-portend-2.6-1.el8.noarch.rpm 376 kB/s | 16 kB 00:00 2024-04-20T02:08:22.916 INFO:teuthology.orchestra.run.smithi047.stdout:(15/32): python3-simplegeneric-0.8.1-17.el8.noa 452 kB/s | 19 kB 00:00 2024-04-20T02:08:23.133 INFO:teuthology.orchestra.run.smithi047.stdout:(16/32): ceph-osd-18.2.2-1252.g15eecde2.el8.x86 18 MB/s | 18 MB 00:00 2024-04-20T02:08:23.150 INFO:teuthology.orchestra.run.smithi047.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 102 kB/s | 24 kB 00:00 2024-04-20T02:08:23.225 INFO:teuthology.orchestra.run.smithi047.stdout:(18/32): ceph-mon-18.2.2-1252.g15eecde2.el8.x86 3.8 MB/s | 4.8 MB 00:01 2024-04-20T02:08:23.242 INFO:teuthology.orchestra.run.smithi047.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 299 kB/s | 27 kB 00:00 2024-04-20T02:08:23.276 INFO:teuthology.orchestra.run.smithi047.stdout:(20/32): python3-tempora-1.14.1-5.el8.noarch.rp 206 kB/s | 29 kB 00:00 2024-04-20T02:08:23.301 INFO:teuthology.orchestra.run.smithi047.stdout:(21/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.2 MB/s | 251 kB 00:00 2024-04-20T02:08:23.334 INFO:teuthology.orchestra.run.smithi047.stdout:(22/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.4 MB/s | 85 kB 00:00 2024-04-20T02:08:23.360 INFO:teuthology.orchestra.run.smithi047.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 394 kB/s | 23 kB 00:00 2024-04-20T02:08:23.427 INFO:teuthology.orchestra.run.smithi047.stdout:(24/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.2 MB/s | 255 kB 00:00 2024-04-20T02:08:23.527 INFO:teuthology.orchestra.run.smithi047.stdout:(25/32): python3-pycparser-2.14-14.el8.noarch.r 568 kB/s | 109 kB 00:00 2024-04-20T02:08:23.619 INFO:teuthology.orchestra.run.smithi047.stdout:(26/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 920 kB/s | 238 kB 00:00 2024-04-20T02:08:23.702 INFO:teuthology.orchestra.run.smithi047.stdout:(27/32): python3-cryptography-3.2.1-7.el8_9.x86 2.1 MB/s | 559 kB 00:00 2024-04-20T02:08:23.761 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:08:23.764 INFO:teuthology.orchestra.run.smithi047.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 269 kB/s | 39 kB 00:00 2024-04-20T02:08:23.820 INFO:teuthology.orchestra.run.smithi047.stdout:(29/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.5 MB/s | 457 kB 00:00 2024-04-20T02:08:23.870 INFO:teuthology.orchestra.run.smithi047.stdout:(30/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 618 kB/s | 103 kB 00:00 2024-04-20T02:08:23.945 INFO:teuthology.orchestra.run.smithi047.stdout:(31/32): python3-mako-1.0.6-14.el8.noarch.rpm 867 kB/s | 157 kB 00:00 2024-04-20T02:08:24.004 INFO:teuthology.orchestra.run.smithi047.stdout:(32/32): python3-pytz-2017.2-11.el8.noarch.rpm 296 kB/s | 54 kB 00:00 2024-04-20T02:08:24.008 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:08:24.008 INFO:teuthology.orchestra.run.smithi047.stdout:Total 13 MB/s | 31 MB 00:02 2024-04-20T02:08:24.019 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-20T02:08:24.109 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:08:24.151 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:08:24.151 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:08:24.261 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-20T02:08:24.406 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:08:24.415 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:08:24.417 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 3/32 2024-04-20T02:08:24.658 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2024-04-20T02:08:25.052 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 5/32 2024-04-20T02:08:25.284 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2024-04-20T02:08:25.524 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2024-04-20T02:08:25.576 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:08:25.810 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-20T02:08:25.938 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 8/32 2024-04-20T02:08:25.998 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-20T02:08:26.172 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 3/32 2024-04-20T02:08:26.258 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 9/32 2024-04-20T02:08:26.379 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2024-04-20T02:08:26.436 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2024-04-20T02:08:26.597 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2024-04-20T02:08:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 5/32 2024-04-20T02:08:26.872 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2024-04-20T02:08:26.950 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2024-04-20T02:08:27.200 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2024-04-20T02:08:27.658 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 8/32 2024-04-20T02:08:27.931 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-20T02:08:27.938 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 9/32 2024-04-20T02:08:28.091 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2024-04-20T02:08:28.292 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2024-04-20T02:08:28.533 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2024-04-20T02:08:28.770 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-20T02:08:28.972 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-20T02:08:29.169 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-20T02:08:29.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-20T02:08:29.451 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-20T02:08:29.559 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-20T02:08:29.643 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-20T02:08:29.679 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-20T02:08:29.833 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-20T02:08:29.900 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-20T02:08:30.041 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-20T02:08:30.086 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-20T02:08:30.202 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-20T02:08:30.295 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-20T02:08:30.393 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-20T02:08:30.576 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-20T02:08:30.633 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-20T02:08:30.791 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-20T02:08:30.877 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-20T02:08:31.027 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-20T02:08:31.165 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-20T02:08:31.370 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-20T02:08:31.418 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-20T02:08:31.697 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-20T02:08:31.747 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-20T02:08:32.018 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-20T02:08:32.384 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-20T02:08:32.722 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 27/32 2024-04-20T02:08:32.805 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 27/32 2024-04-20T02:08:33.041 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T02:08:34.240 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T02:08:34.683 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T02:08:34.941 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T02:08:35.420 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T02:08:35.621 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T02:08:35.836 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T02:08:36.038 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T02:08:36.056 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T02:08:36.060 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T02:08:36.211 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T02:08:36.298 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 1/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 3/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 4/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 5/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 6/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-20T02:08:36.600 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-20T02:08:36.601 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 25/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 26/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 27/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 30/32 2024-04-20T02:08:36.602 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 31/32 2024-04-20T02:08:36.832 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T02:08:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T02:08:37.218 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T02:08:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T02:08:37.403 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T02:08:37.547 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T02:08:37.993 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 1/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 3/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 4/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 5/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 6/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-20T02:08:37.994 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 25/32 2024-04-20T02:08:37.995 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 26/32 2024-04-20T02:08:37.996 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 27/32 2024-04-20T02:08:37.996 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2024-04-20T02:08:37.996 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2024-04-20T02:08:37.996 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 30/32 2024-04-20T02:08:37.996 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 31/32 2024-04-20T02:08:45.320 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 32/32 2024-04-20T02:08:45.320 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:08:45.890 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-20T02:08:45.891 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-2.6-1.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-20T02:08:45.892 INFO:teuthology.orchestra.run.smithi179.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:45.893 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:08:46.076 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-base 2024-04-20T02:08:46.429 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:08:48.515 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 kB/s | 4.1 kB 00:00 2024-04-20T02:08:48.866 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T02:08:51.303 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:08:51.487 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:08:51.488 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:08:51.488 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:08:51.644 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install cephadm 2024-04-20T02:08:51.992 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:08:53.031 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 32/32 2024-04-20T02:08:53.031 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-20T02:08:53.710 INFO:teuthology.orchestra.run.smithi047.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-2.6-1.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-20T02:08:53.711 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:08:53.712 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:08:53.854 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T02:08:53.913 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-base 2024-04-20T02:08:54.199 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-20T02:08:54.316 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:08:56.466 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-04-20T02:08:56.809 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 173 k 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:08:56.810 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:08:56.811 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 173 k 2024-04-20T02:08:56.811 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 692 k 2024-04-20T02:08:56.811 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:08:56.838 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T02:08:57.146 INFO:teuthology.orchestra.run.smithi179.stdout:cephadm-18.2.2-1252.g15eecde2.el8.noarch.rpm 516 kB/s | 173 kB 00:00 2024-04-20T02:08:57.146 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:08:57.147 INFO:teuthology.orchestra.run.smithi179.stdout:Total 514 kB/s | 173 kB 00:00 2024-04-20T02:08:57.147 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:08:57.155 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:08:57.155 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:08:57.157 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:08:57.166 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:08:57.208 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:08:58.973 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:08:59.167 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:08:59.370 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:08:59.379 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:08:59.598 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:08:59.599 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:08:59.599 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:08:59.770 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install cephadm 2024-04-20T02:09:00.171 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:02.151 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:09:02.517 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T02:09:05.242 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 173 k 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:09:05.243 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:05.244 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:09:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:05.244 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 173 k 2024-04-20T02:09:05.244 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 692 k 2024-04-20T02:09:05.244 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:09:05.561 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm-18.2.2-1252.g15eecde2.el8.noarch.rpm 545 kB/s | 173 kB 00:00 2024-04-20T02:09:05.562 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:09:05.562 INFO:teuthology.orchestra.run.smithi047.stdout:Total 543 kB/s | 173 kB 00:00 2024-04-20T02:09:05.562 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:09:05.573 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:09:05.573 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:09:05.576 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:09:05.586 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:09:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:09:07.676 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:09:07.874 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:09:08.111 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:09:11.743 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:09:11.743 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:09:12.295 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:12.295 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:09:12.295 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:09:12.295 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:12.295 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:09:12.444 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T02:09:12.808 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:14.668 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-04-20T02:09:15.006 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-04-20T02:09:17.594 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 166 k 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:09:17.595 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:09:17.596 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:09:17.596 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:17.596 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 166 k 2024-04-20T02:09:17.596 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 503 k 2024-04-20T02:09:17.596 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:09:17.890 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-immutable-object-cache-18.2.2-1252.g15eecd 564 kB/s | 166 kB 00:00 2024-04-20T02:09:17.890 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:09:17.890 INFO:teuthology.orchestra.run.smithi179.stdout:Total 562 kB/s | 166 kB 00:00 2024-04-20T02:09:17.891 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:09:17.902 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:09:17.902 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:09:17.957 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:09:17.965 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:09:18.093 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:09:18.248 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T02:09:18.677 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T02:09:23.867 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T02:09:23.867 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:09:24.483 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:24.483 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:09:24.483 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:09:24.483 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:24.483 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:09:24.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T02:09:25.095 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:27.068 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 kB/s | 4.1 kB 00:00 2024-04-20T02:09:27.444 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-20T02:09:29.513 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T02:09:29.513 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:09:30.071 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:30.071 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:09:30.071 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:09:30.072 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:30.072 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:09:30.207 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:09:30.207 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 166 k 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:09:30.208 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:30.209 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 166 k 2024-04-20T02:09:30.209 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 503 k 2024-04-20T02:09:30.209 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:09:30.232 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr 2024-04-20T02:09:30.500 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-immutable-object-cache-18.2.2-1252.g15eecd 570 kB/s | 166 kB 00:00 2024-04-20T02:09:30.500 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:09:30.500 INFO:teuthology.orchestra.run.smithi047.stdout:Total 567 kB/s | 166 kB 00:00 2024-04-20T02:09:30.501 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:09:30.515 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:09:30.515 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:09:30.581 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:30.582 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:09:30.592 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:09:30.723 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:09:30.857 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T02:09:31.307 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T02:09:32.892 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T02:09:33.176 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 80 kB/s | 4.5 kB 00:00 2024-04-20T02:09:35.636 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:09:35.826 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:09:35.827 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:09:35.827 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:09:35.983 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T02:09:36.333 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:38.396 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 kB/s | 4.1 kB 00:00 2024-04-20T02:09:38.742 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T02:09:41.350 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 3.5 M 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 24 k 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 15 k 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:41.351 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:09:41.352 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:09:41.352 INFO:teuthology.orchestra.run.smithi179.stdout:Install 5 Packages 2024-04-20T02:09:41.352 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:09:41.352 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 3.8 M 2024-04-20T02:09:41.352 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 87 M 2024-04-20T02:09:41.352 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:09:42.077 INFO:teuthology.orchestra.run.smithi179.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1252.g15e 98 kB/s | 24 kB 00:00 2024-04-20T02:09:42.353 INFO:teuthology.orchestra.run.smithi179.stdout:(2/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 122 kB/s | 34 kB 00:00 2024-04-20T02:09:42.803 INFO:teuthology.orchestra.run.smithi179.stdout:(3/5): python3-routes-2.4.1-12.el8.noarch.rpm 437 kB/s | 196 kB 00:00 2024-04-20T02:09:43.170 INFO:teuthology.orchestra.run.smithi179.stdout:(4/5): ceph-prometheus-alerts-18.2.2-1252.g15ee 11 kB/s | 15 kB 00:01 2024-04-20T02:09:43.478 INFO:teuthology.orchestra.run.smithi179.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1252.g15eecde2 2.1 MB/s | 3.5 MB 00:01 2024-04-20T02:09:43.481 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:09:43.481 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.8 MB/s | 3.8 MB 00:02 2024-04-20T02:09:43.482 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:09:43.490 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:09:43.491 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:09:43.565 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:09:43.574 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:09:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:09:43.976 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T02:09:43.977 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:09:44.092 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-20T02:09:44.339 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-20T02:09:44.504 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T02:09:44.595 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:44.595 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:09:44.595 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:09:44.595 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:44.595 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:09:44.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr 2024-04-20T02:09:45.183 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:45.666 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 4/5 2024-04-20T02:09:45.964 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T02:09:46.259 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T02:09:46.259 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 1/5 2024-04-20T02:09:46.259 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 2/5 2024-04-20T02:09:46.259 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T02:09:46.260 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-20T02:09:47.336 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-04-20T02:09:47.605 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 103 kB/s | 4.5 kB 00:00 2024-04-20T02:09:50.088 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:09:50.301 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:09:50.302 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:09:50.302 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:09:50.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T02:09:50.860 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:09:53.051 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T02:09:53.420 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T02:09:56.146 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:09:56.147 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 3.5 M 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 24 k 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 15 k 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout:Install 5 Packages 2024-04-20T02:09:56.148 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:09:56.149 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 3.8 M 2024-04-20T02:09:56.149 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 87 M 2024-04-20T02:09:56.149 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:09:56.595 INFO:teuthology.orchestra.run.smithi047.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1252.g15ee 53 kB/s | 15 kB 00:00 2024-04-20T02:09:56.753 INFO:teuthology.orchestra.run.smithi047.stdout:(2/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 213 kB/s | 34 kB 00:00 2024-04-20T02:09:56.912 INFO:teuthology.orchestra.run.smithi047.stdout:(3/5): python3-routes-2.4.1-12.el8.noarch.rpm 1.2 MB/s | 196 kB 00:00 2024-04-20T02:09:56.970 INFO:teuthology.orchestra.run.smithi047.stdout:(4/5): ceph-mgr-dashboard-18.2.2-1252.g15eecde2 5.3 MB/s | 3.5 MB 00:00 2024-04-20T02:09:57.689 INFO:teuthology.orchestra.run.smithi047.stdout:(5/5): ceph-grafana-dashboards-18.2.2-1252.g15e 18 kB/s | 24 kB 00:01 2024-04-20T02:09:57.691 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:09:57.692 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.5 MB/s | 3.8 MB 00:01 2024-04-20T02:09:57.692 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:09:57.702 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:09:57.702 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:09:57.784 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:09:57.793 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:09:58.029 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:09:58.231 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-20T02:09:58.423 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-20T02:09:58.548 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T02:09:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 4/5 2024-04-20T02:10:00.120 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T02:10:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T02:10:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 1/5 2024-04-20T02:10:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 2/5 2024-04-20T02:10:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T02:10:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-20T02:10:02.093 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-20T02:10:02.094 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:10:02.660 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:02.660 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:10:02.660 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:02.660 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:02.661 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:02.661 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-20T02:10:02.661 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-20T02:10:02.661 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:02.661 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:10:02.837 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T02:10:03.193 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:10:05.645 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T02:10:05.939 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 64 kB/s | 4.5 kB 00:00 2024-04-20T02:10:08.541 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================================= 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================================= 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 7.4 M 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:10:08.542 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================================= 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout:Install 8 Packages 2024-04-20T02:10:08.543 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:08.544 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 36 M 2024-04-20T02:10:08.544 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 207 M 2024-04-20T02:10:08.544 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:10:08.710 INFO:teuthology.orchestra.run.smithi179.stdout:(1/8): libquadmath-8.5.0-20.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2024-04-20T02:10:08.761 INFO:teuthology.orchestra.run.smithi179.stdout:(2/8): libgfortran-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2024-04-20T02:10:08.869 INFO:teuthology.orchestra.run.smithi179.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.0 MB/s | 225 kB 00:00 2024-04-20T02:10:08.953 INFO:teuthology.orchestra.run.smithi179.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 15 MB/s | 3.7 MB 00:00 2024-04-20T02:10:09.111 INFO:teuthology.orchestra.run.smithi179.stdout:(5/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 22 MB/s | 5.2 MB 00:00 2024-04-20T02:10:09.228 INFO:teuthology.orchestra.run.smithi179.stdout:(6/8): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2024-04-20T02:10:09.312 INFO:teuthology.orchestra.run.smithi179.stdout:(7/8): ceph-mgr-diskprediction-local-18.2.2-125 9.6 MB/s | 7.4 MB 00:00 2024-04-20T02:10:09.520 INFO:teuthology.orchestra.run.smithi179.stdout:(8/8): python3-scipy-1.0.0-21.module+el8.9.0+20 34 MB/s | 14 MB 00:00 2024-04-20T02:10:09.522 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:10:09.522 INFO:teuthology.orchestra.run.smithi179.stdout:Total 37 MB/s | 36 MB 00:00 2024-04-20T02:10:09.766 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:10:09.785 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:10:09.785 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:10:10.095 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:10:10.106 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:10:10.690 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:10:10.898 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T02:10:11.022 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T02:10:11.216 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T02:10:11.845 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T02:10:12.057 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T02:10:12.668 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T02:10:12.907 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T02:10:13.582 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T02:10:13.952 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T02:10:14.190 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T02:10:15.944 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T02:10:17.259 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-20T02:10:17.456 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T02:10:17.808 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 1/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 3/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 6/8 2024-04-20T02:10:17.809 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 7/8 2024-04-20T02:10:18.173 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-20T02:10:18.173 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:10:18.784 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:10:19.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T02:10:19.420 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:10:21.768 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:10:22.162 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T02:10:24.936 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:10:24.937 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================================= 2024-04-20T02:10:24.937 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:10:24.937 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================================= 2024-04-20T02:10:24.937 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:10:24.937 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 7.4 M 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================================= 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout:Install 8 Packages 2024-04-20T02:10:24.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:10:24.939 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 36 M 2024-04-20T02:10:24.939 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 207 M 2024-04-20T02:10:24.939 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:10:25.185 INFO:teuthology.orchestra.run.smithi047.stdout:(1/8): libgfortran-8.5.0-20.el8.x86_64.rpm 2.6 MB/s | 645 kB 00:00 2024-04-20T02:10:25.210 INFO:teuthology.orchestra.run.smithi047.stdout:(2/8): libquadmath-8.5.0-20.el8.x86_64.rpm 639 kB/s | 172 kB 00:00 2024-04-20T02:10:25.294 INFO:teuthology.orchestra.run.smithi047.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.6 MB/s | 225 kB 00:00 2024-04-20T02:10:25.486 INFO:teuthology.orchestra.run.smithi047.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 12 MB/s | 3.7 MB 00:00 2024-04-20T02:10:25.595 INFO:teuthology.orchestra.run.smithi047.stdout:(5/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.2 MB 00:00 2024-04-20T02:10:25.778 INFO:teuthology.orchestra.run.smithi047.stdout:(6/8): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-04-20T02:10:25.981 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 8/8 2024-04-20T02:10:25.982 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:10:26.045 INFO:teuthology.orchestra.run.smithi047.stdout:(7/8): python3-scipy-1.0.0-21.module+el8.9.0+20 30 MB/s | 14 MB 00:00 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:26.543 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:10:26.755 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-rook 2024-04-20T02:10:27.105 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:10:27.129 INFO:teuthology.orchestra.run.smithi047.stdout:(8/8): ceph-mgr-diskprediction-local-18.2.2-125 3.4 MB/s | 7.4 MB 00:02 2024-04-20T02:10:27.130 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:10:27.130 INFO:teuthology.orchestra.run.smithi047.stdout:Total 16 MB/s | 36 MB 00:02 2024-04-20T02:10:27.397 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:10:27.418 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:10:27.418 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:10:27.762 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:10:27.772 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:10:28.400 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:10:28.607 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T02:10:28.773 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T02:10:28.916 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T02:10:29.387 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 23 kB/s | 4.1 kB 00:00 2024-04-20T02:10:29.592 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T02:10:29.740 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T02:10:29.765 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T02:10:30.446 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T02:10:30.616 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T02:10:31.417 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T02:10:31.723 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T02:10:31.932 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T02:10:32.375 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:10:32.377 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================== 2024-04-20T02:10:32.377 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:10:32.377 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================== 2024-04-20T02:10:32.377 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:10:32.377 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 49 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-20T02:10:32.378 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================== 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout:Install 14 Packages 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:32.379 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 2.0 M 2024-04-20T02:10:32.380 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 25 M 2024-04-20T02:10:32.380 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:10:32.662 INFO:teuthology.orchestra.run.smithi179.stdout:(1/14): python3-certifi-2018.10.15-7.el8.noarch 112 kB/s | 16 kB 00:00 2024-04-20T02:10:32.687 INFO:teuthology.orchestra.run.smithi179.stdout:(2/14): python3-cachetools-3.1.1-4.el8.noarch.r 193 kB/s | 33 kB 00:00 2024-04-20T02:10:32.812 INFO:teuthology.orchestra.run.smithi179.stdout:(3/14): python3-google-auth-1.1.1-10.el8.noarch 541 kB/s | 81 kB 00:00 2024-04-20T02:10:32.838 INFO:teuthology.orchestra.run.smithi179.stdout:(4/14): ceph-mgr-rook-18.2.2-1252.g15eecde2.el8 153 kB/s | 49 kB 00:00 2024-04-20T02:10:32.905 INFO:teuthology.orchestra.run.smithi179.stdout:(5/14): python3-rsa-4.9-2.el8.noarch.rpm 697 kB/s | 64 kB 00:00 2024-04-20T02:10:33.055 INFO:teuthology.orchestra.run.smithi179.stdout:(6/14): python3-websocket-client-0.56.0-5.el8.n 282 kB/s | 61 kB 00:00 2024-04-20T02:10:33.089 INFO:teuthology.orchestra.run.smithi179.stdout:(7/14): python3-kubernetes-11.0.0-6.el8.noarch. 2.9 MB/s | 1.2 MB 00:00 2024-04-20T02:10:33.122 INFO:teuthology.orchestra.run.smithi179.stdout:(8/14): python3-jwt-1.6.1-2.el8.noarch.rpm 196 kB/s | 43 kB 00:00 2024-04-20T02:10:33.198 INFO:teuthology.orchestra.run.smithi179.stdout:(9/14): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.1 MB/s | 155 kB 00:00 2024-04-20T02:10:33.281 INFO:teuthology.orchestra.run.smithi179.stdout:(10/14): python3-jsonpointer-1.10-11.el8.noarch 130 kB/s | 20 kB 00:00 2024-04-20T02:10:33.340 INFO:teuthology.orchestra.run.smithi179.stdout:(11/14): python3-requests-oauthlib-1.0.0-1.el8. 174 kB/s | 43 kB 00:00 2024-04-20T02:10:33.390 INFO:teuthology.orchestra.run.smithi179.stdout:(12/14): python3-pyasn1-modules-0.3.7-6.el8.noa 573 kB/s | 110 kB 00:00 2024-04-20T02:10:33.474 INFO:teuthology.orchestra.run.smithi179.stdout:(13/14): python3-jsonpatch-1.21-2.el8.noarch.rp 138 kB/s | 27 kB 00:00 2024-04-20T02:10:33.541 INFO:teuthology.orchestra.run.smithi179.stdout:(14/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 629 kB/s | 126 kB 00:00 2024-04-20T02:10:33.541 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:10:33.541 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-04-20T02:10:33.615 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:10:33.630 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:10:33.630 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:10:33.712 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:10:33.721 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:10:33.907 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T02:10:34.295 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:10:34.579 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-20T02:10:34.856 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/14 2024-04-20T02:10:35.169 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/14 2024-04-20T02:10:35.369 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/14 2024-04-20T02:10:35.555 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/14 2024-04-20T02:10:35.832 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 6/14 2024-04-20T02:10:35.890 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-20T02:10:36.089 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T02:10:36.158 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 7/14 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 1/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 3/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 6/8 2024-04-20T02:10:36.438 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 7/8 2024-04-20T02:10:36.490 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 8/14 2024-04-20T02:10:36.694 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 9/14 2024-04-20T02:10:36.951 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-20T02:10:37.252 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 11/14 2024-04-20T02:10:38.355 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 12/14 2024-04-20T02:10:38.656 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 13/14 2024-04-20T02:10:38.862 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 1/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/14 2024-04-20T02:10:39.121 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 8/14 2024-04-20T02:10:39.122 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 9/14 2024-04-20T02:10:39.122 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 10/14 2024-04-20T02:10:39.122 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/14 2024-04-20T02:10:39.122 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/14 2024-04-20T02:10:39.122 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 13/14 2024-04-20T02:10:54.084 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 8/8 2024-04-20T02:10:54.084 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:10:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:10:54.699 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:10:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-20T02:10:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-20T02:10:54.699 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-20T02:10:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-20T02:10:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-20T02:10:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-20T02:10:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-20T02:10:54.700 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:10:54.700 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:10:54.927 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-rook 2024-04-20T02:10:55.331 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:10:57.328 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T02:10:57.721 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T02:10:57.747 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 14/14 2024-04-20T02:10:57.747 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:10:58.306 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:58.306 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:10:58.306 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:10:58.306 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-20T02:10:58.306 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:10:58.307 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:10:58.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T02:10:58.838 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:11:00.505 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:11:00.507 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================== 2024-04-20T02:11:00.507 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:11:00.507 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================== 2024-04-20T02:11:00.507 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 49 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T02:11:00.508 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================== 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout:Install 14 Packages 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 2.0 M 2024-04-20T02:11:00.509 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 25 M 2024-04-20T02:11:00.510 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:11:00.948 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T02:11:01.294 INFO:teuthology.orchestra.run.smithi047.stdout:(1/14): python3-certifi-2018.10.15-7.el8.noarch 37 kB/s | 16 kB 00:00 2024-04-20T02:11:01.319 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T02:11:01.447 INFO:teuthology.orchestra.run.smithi047.stdout:(2/14): python3-cachetools-3.1.1-4.el8.noarch.r 55 kB/s | 33 kB 00:00 2024-04-20T02:11:01.562 INFO:teuthology.orchestra.run.smithi047.stdout:(3/14): ceph-mgr-rook-18.2.2-1252.g15eecde2.el8 69 kB/s | 49 kB 00:00 2024-04-20T02:11:01.595 INFO:teuthology.orchestra.run.smithi047.stdout:(4/14): python3-google-auth-1.1.1-10.el8.noarch 270 kB/s | 81 kB 00:00 2024-04-20T02:11:01.688 INFO:teuthology.orchestra.run.smithi047.stdout:(5/14): python3-websocket-client-0.56.0-5.el8.n 668 kB/s | 61 kB 00:00 2024-04-20T02:11:01.713 INFO:teuthology.orchestra.run.smithi047.stdout:(6/14): python3-rsa-4.9-2.el8.noarch.rpm 426 kB/s | 64 kB 00:00 2024-04-20T02:11:01.755 INFO:teuthology.orchestra.run.smithi047.stdout:(7/14): python3-kubernetes-11.0.0-6.el8.noarch. 3.8 MB/s | 1.2 MB 00:00 2024-04-20T02:11:01.930 INFO:teuthology.orchestra.run.smithi047.stdout:(8/14): python3-jwt-1.6.1-2.el8.noarch.rpm 176 kB/s | 43 kB 00:00 2024-04-20T02:11:02.024 INFO:teuthology.orchestra.run.smithi047.stdout:(9/14): python3-oauthlib-2.1.0-1.el8.noarch.rpm 498 kB/s | 155 kB 00:00 2024-04-20T02:11:02.049 INFO:teuthology.orchestra.run.smithi047.stdout:(10/14): python3-requests-oauthlib-1.0.0-1.el8. 154 kB/s | 43 kB 00:00 2024-04-20T02:11:02.066 INFO:teuthology.orchestra.run.smithi047.stdout:(11/14): python3-jsonpointer-1.10-11.el8.noarch 147 kB/s | 20 kB 00:00 2024-04-20T02:11:02.133 INFO:teuthology.orchestra.run.smithi047.stdout:(12/14): python3-pyasn1-modules-0.3.7-6.el8.noa 1.0 MB/s | 110 kB 00:00 2024-04-20T02:11:02.175 INFO:teuthology.orchestra.run.smithi047.stdout:(13/14): python3-jsonpatch-1.21-2.el8.noarch.rp 211 kB/s | 27 kB 00:00 2024-04-20T02:11:02.200 INFO:teuthology.orchestra.run.smithi047.stdout:(14/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 946 kB/s | 126 kB 00:00 2024-04-20T02:11:02.201 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:11:02.201 INFO:teuthology.orchestra.run.smithi047.stdout:Total 1.2 MB/s | 2.0 MB 00:01 2024-04-20T02:11:02.285 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:11:02.303 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:11:02.304 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:11:02.397 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:11:02.407 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:11:02.966 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:11:03.218 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-20T02:11:03.472 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/14 2024-04-20T02:11:03.645 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/14 2024-04-20T02:11:03.829 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/14 2024-04-20T02:11:03.974 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================================== 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================================== 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 138 k 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-20T02:11:03.975 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================================== 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout:Install 5 Packages 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:03.976 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 6.0 M 2024-04-20T02:11:03.977 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 26 M 2024-04-20T02:11:03.977 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:11:03.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/14 2024-04-20T02:11:04.236 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 6/14 2024-04-20T02:11:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 7/14 2024-04-20T02:11:04.594 INFO:teuthology.orchestra.run.smithi179.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 140 kB/s | 60 kB 00:00 2024-04-20T02:11:04.653 INFO:teuthology.orchestra.run.smithi179.stdout:(2/5): ceph-mgr-cephadm-18.2.2-1252.g15eecde2.e 282 kB/s | 138 kB 00:00 2024-04-20T02:11:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 8/14 2024-04-20T02:11:04.862 INFO:teuthology.orchestra.run.smithi179.stdout:(3/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.6 MB/s | 538 kB 00:00 2024-04-20T02:11:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 9/14 2024-04-20T02:11:04.989 INFO:teuthology.orchestra.run.smithi179.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 578 kB/s | 476 kB 00:00 2024-04-20T02:11:05.054 INFO:teuthology.orchestra.run.smithi179.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 10 MB/s | 4.8 MB 00:00 2024-04-20T02:11:05.054 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:11:05.055 INFO:teuthology.orchestra.run.smithi179.stdout:Total 5.5 MB/s | 6.0 MB 00:01 2024-04-20T02:11:05.108 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:11:05.119 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:11:05.119 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:11:05.207 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:11:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-20T02:11:05.216 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:11:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 11/14 2024-04-20T02:11:06.188 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:11:06.579 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-20T02:11:06.710 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 12/14 2024-04-20T02:11:06.818 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-20T02:11:07.189 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T02:11:07.675 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-20T02:11:07.899 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T02:11:07.994 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 13/14 2024-04-20T02:11:08.161 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T02:11:08.166 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T02:11:08.166 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/5 2024-04-20T02:11:08.166 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-20T02:11:08.166 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T02:11:08.166 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 1/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 8/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 9/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 10/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/14 2024-04-20T02:11:08.479 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 13/14 2024-04-20T02:11:25.317 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 14/14 2024-04-20T02:11:25.317 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:11:25.399 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-20T02:11:25.399 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:11:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:25.953 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:11:25.953 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-20T02:11:25.954 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-20T02:11:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-20T02:11:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-20T02:11:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-20T02:11:25.955 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:25.955 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:11:25.964 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:25.965 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:11:26.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-fuse 2024-04-20T02:11:26.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T02:11:26.498 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:11:26.605 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:11:28.840 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T02:11:28.895 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-04-20T02:11:29.170 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 kB/s | 4.5 kB 00:00 2024-04-20T02:11:29.279 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T02:11:31.928 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:11:31.929 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================================== 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================================== 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 138 k 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================================== 2024-04-20T02:11:31.930 INFO:teuthology.orchestra.run.smithi047.stdout:Install 5 Packages 2024-04-20T02:11:31.931 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:31.931 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 6.0 M 2024-04-20T02:11:31.931 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 26 M 2024-04-20T02:11:31.931 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:11:32.011 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 859 k 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-20T02:11:32.012 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 942 k 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 3.0 M 2024-04-20T02:11:32.013 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:11:32.215 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 414 kB/s | 83 kB 00:00 2024-04-20T02:11:32.670 INFO:teuthology.orchestra.run.smithi047.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 174 kB/s | 60 kB 00:00 2024-04-20T02:11:32.693 INFO:teuthology.orchestra.run.smithi047.stdout:(2/5): ceph-mgr-cephadm-18.2.2-1252.g15eecde2.e 372 kB/s | 138 kB 00:00 2024-04-20T02:11:32.902 INFO:teuthology.orchestra.run.smithi047.stdout:(3/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.5 MB/s | 538 kB 00:00 2024-04-20T02:11:32.985 INFO:teuthology.orchestra.run.smithi047.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 717 kB/s | 476 kB 00:00 2024-04-20T02:11:33.086 INFO:teuthology.orchestra.run.smithi047.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-04-20T02:11:33.087 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:11:33.087 INFO:teuthology.orchestra.run.smithi047.stdout:Total 5.2 MB/s | 6.0 MB 00:01 2024-04-20T02:11:33.148 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:11:33.161 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:11:33.161 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:11:33.261 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:11:33.271 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:11:33.548 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): ceph-fuse-18.2.2-1252.g15eecde2.el8.x86_ 560 kB/s | 859 kB 00:01 2024-04-20T02:11:33.548 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:11:33.549 INFO:teuthology.orchestra.run.smithi179.stdout:Total 613 kB/s | 942 kB 00:01 2024-04-20T02:11:33.563 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:11:33.586 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:11:33.586 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:11:33.676 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:11:33.685 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:11:33.857 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:11:34.162 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-20T02:11:34.346 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:11:34.370 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T02:11:34.698 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-20T02:11:34.719 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T02:11:34.719 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/2 2024-04-20T02:11:34.897 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-20T02:11:35.233 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T02:11:35.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-20T02:11:35.732 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T02:11:36.038 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T02:11:36.038 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/5 2024-04-20T02:11:36.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-20T02:11:36.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T02:11:36.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-20T02:11:49.320 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-20T02:11:49.320 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:11:49.897 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:49.897 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:11:49.898 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-20T02:11:49.898 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:49.898 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:11:50.066 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-volume 2024-04-20T02:11:50.427 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:11:50.544 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-20T02:11:50.544 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:11:51.178 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:51.178 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:11:51.178 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:11:51.178 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-20T02:11:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-20T02:11:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-20T02:11:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-20T02:11:51.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:51.179 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:11:51.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-fuse 2024-04-20T02:11:51.777 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:11:52.467 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:11:52.795 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-04-20T02:11:54.229 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-20T02:11:54.602 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-20T02:11:55.510 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:11:55.511 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================== 2024-04-20T02:11:55.511 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-20T02:11:55.511 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================== 2024-04-20T02:11:55.511 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:11:55.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 263 k 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================================== 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:11:55.512 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 311 k 2024-04-20T02:11:55.513 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 1.4 M 2024-04-20T02:11:55.513 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:11:55.687 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): python3-packaging-16.8-9.el8.noarch.rpm 278 kB/s | 48 kB 00:00 2024-04-20T02:11:55.837 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): ceph-volume-18.2.2-1252.g15eecde2.el8.no 812 kB/s | 263 kB 00:00 2024-04-20T02:11:55.837 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:11:55.837 INFO:teuthology.orchestra.run.smithi179.stdout:Total 957 kB/s | 311 kB 00:00 2024-04-20T02:11:55.850 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:11:55.865 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:11:55.865 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:11:55.902 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:11:55.911 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:11:56.112 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:11:56.491 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-20T02:11:56.801 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T02:11:57.214 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T02:11:57.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 1/2 2024-04-20T02:11:57.360 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:11:57.360 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:11:57.360 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:11:57.360 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:11:57.360 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout: x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 859 k 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout:Install 2 Packages 2024-04-20T02:11:57.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:11:57.362 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 942 k 2024-04-20T02:11:57.362 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 3.0 M 2024-04-20T02:11:57.362 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:11:57.530 INFO:teuthology.orchestra.run.smithi047.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 496 kB/s | 83 kB 00:00 2024-04-20T02:11:57.638 INFO:teuthology.orchestra.run.smithi047.stdout:(2/2): ceph-fuse-18.2.2-1252.g15eecde2.el8.x86_ 3.0 MB/s | 859 kB 00:00 2024-04-20T02:11:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:11:57.639 INFO:teuthology.orchestra.run.smithi047.stdout:Total 3.3 MB/s | 942 kB 00:00 2024-04-20T02:11:57.654 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:11:57.679 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:11:57.679 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:11:57.780 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:11:57.790 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:11:57.974 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:11:58.230 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-20T02:11:58.413 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T02:11:58.834 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T02:11:58.835 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/2 2024-04-20T02:12:11.983 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-20T02:12:11.983 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:12:12.545 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:12.545 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:12:12.545 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:12:12.545 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-20T02:12:12.545 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:12.545 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:12:12.696 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados-devel 2024-04-20T02:12:13.057 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:12:15.408 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:12:15.656 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-20T02:12:15.656 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:12:15.784 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T02:12:16.311 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:16.311 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:12:16.311 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-20T02:12:16.311 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:16.312 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:12:16.494 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-volume 2024-04-20T02:12:16.911 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:12:18.377 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 139 k 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:18.378 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 139 k 2024-04-20T02:12:18.379 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 513 k 2024-04-20T02:12:18.379 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:12:18.709 INFO:teuthology.orchestra.run.smithi179.stdout:librados-devel-18.2.2-1252.g15eecde2.el8.x86_64 421 kB/s | 139 kB 00:00 2024-04-20T02:12:18.710 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:12:18.710 INFO:teuthology.orchestra.run.smithi179.stdout:Total 420 kB/s | 139 kB 00:00 2024-04-20T02:12:18.710 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:12:18.715 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:12:18.716 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:12:18.774 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:12:18.782 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:12:18.908 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:12:19.104 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-20T02:12:19.155 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:19.417 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:19.467 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T02:12:22.256 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:12:22.256 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================== 2024-04-20T02:12:22.256 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================== 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 263 k 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================================== 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout:Install 2 Packages 2024-04-20T02:12:22.257 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:22.258 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 311 k 2024-04-20T02:12:22.258 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 1.4 M 2024-04-20T02:12:22.258 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:12:22.435 INFO:teuthology.orchestra.run.smithi047.stdout:(1/2): python3-packaging-16.8-9.el8.noarch.rpm 273 kB/s | 48 kB 00:00 2024-04-20T02:12:22.819 INFO:teuthology.orchestra.run.smithi047.stdout:(2/2): ceph-volume-18.2.2-1252.g15eecde2.el8.no 469 kB/s | 263 kB 00:00 2024-04-20T02:12:22.819 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:12:22.820 INFO:teuthology.orchestra.run.smithi047.stdout:Total 553 kB/s | 311 kB 00:00 2024-04-20T02:12:22.835 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:12:22.849 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:12:22.850 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:12:22.897 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:12:22.906 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:12:23.099 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:12:23.460 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-20T02:12:23.718 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T02:12:24.184 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T02:12:24.184 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 1/2 2024-04-20T02:12:36.678 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:36.678 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:12:36.922 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-20T02:12:36.922 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:12:37.264 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:37.264 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:12:37.264 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:12:37.264 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:37.264 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:12:37.433 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs2 2024-04-20T02:12:37.576 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:37.576 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:12:37.576 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T02:12:37.576 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-20T02:12:37.576 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:37.576 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:12:37.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados-devel 2024-04-20T02:12:37.794 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:12:38.180 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:12:39.702 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-20T02:12:40.130 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 23 kB/s | 4.5 kB 00:00 2024-04-20T02:12:40.740 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 kB/s | 4.1 kB 00:00 2024-04-20T02:12:41.149 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-20T02:12:42.557 INFO:teuthology.orchestra.run.smithi179.stdout:Package libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:12:42.746 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:12:42.747 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:12:42.747 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:12:42.921 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs-devel 2024-04-20T02:12:43.278 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:12:43.911 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:12:43.912 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 139 k 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 139 k 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 513 k 2024-04-20T02:12:43.913 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:12:44.099 INFO:teuthology.orchestra.run.smithi047.stdout:librados-devel-18.2.2-1252.g15eecde2.el8.x86_64 749 kB/s | 139 kB 00:00 2024-04-20T02:12:44.100 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:12:44.100 INFO:teuthology.orchestra.run.smithi047.stdout:Total 744 kB/s | 139 kB 00:00 2024-04-20T02:12:44.100 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:12:44.107 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:12:44.107 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:12:44.176 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:12:44.185 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:12:44.330 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:12:44.575 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:44.869 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:45.170 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 kB/s | 4.1 kB 00:00 2024-04-20T02:12:45.546 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T02:12:48.173 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:12:48.174 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:12:48.174 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-04-20T02:12:48.174 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:12:48.174 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 31 k 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 31 k 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 139 k 2024-04-20T02:12:48.175 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:12:48.463 INFO:teuthology.orchestra.run.smithi179.stdout:libcephfs-devel-18.2.2-1252.g15eecde2.el8.x86_6 107 kB/s | 31 kB 00:00 2024-04-20T02:12:48.464 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:12:48.464 INFO:teuthology.orchestra.run.smithi179.stdout:Total 107 kB/s | 31 kB 00:00 2024-04-20T02:12:48.464 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:12:48.468 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:12:48.468 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:12:48.481 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:12:48.489 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:12:48.547 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:12:48.775 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:49.025 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:54.830 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:12:54.831 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:12:55.455 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:55.455 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:12:55.455 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:12:55.455 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:12:55.455 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:12:55.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs2 2024-04-20T02:12:56.060 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:12:58.141 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T02:12:58.492 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 kB/s | 4.5 kB 00:00 2024-04-20T02:13:01.056 INFO:teuthology.orchestra.run.smithi047.stdout:Package libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:01.276 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:01.278 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:01.278 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:01.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs-devel 2024-04-20T02:13:01.846 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:02.146 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:02.146 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:13:02.712 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:13:02.712 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:13:02.712 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:13:02.712 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:13:02.712 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:02.913 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados2 2024-04-20T02:13:03.273 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:03.707 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T02:13:04.104 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-20T02:13:05.211 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T02:13:05.599 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T02:13:06.837 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 31 k 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:13:06.838 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:13:06.839 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:13:06.839 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 31 k 2024-04-20T02:13:06.839 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 139 k 2024-04-20T02:13:06.839 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:13:07.070 INFO:teuthology.orchestra.run.smithi047.stdout:libcephfs-devel-18.2.2-1252.g15eecde2.el8.x86_6 134 kB/s | 31 kB 00:00 2024-04-20T02:13:07.071 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:13:07.071 INFO:teuthology.orchestra.run.smithi047.stdout:Total 133 kB/s | 31 kB 00:00 2024-04-20T02:13:07.071 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:13:07.076 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:13:07.076 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:13:07.092 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:13:07.203 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:13:07.370 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:07.643 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:08.017 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:08.206 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:08.207 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:13:08.207 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:08.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librbd1 2024-04-20T02:13:08.712 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:11.010 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-20T02:13:11.361 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T02:13:13.789 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:13.975 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:13.977 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:13:13.977 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:14.125 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rados 2024-04-20T02:13:14.484 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:16.512 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T02:13:16.887 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T02:13:19.300 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:19.488 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:19.489 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:13:19.489 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:19.627 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rgw 2024-04-20T02:13:19.987 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:21.399 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:21.400 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:13:21.667 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-04-20T02:13:22.021 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T02:13:22.023 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:13:22.024 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:13:22.024 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:13:22.024 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:13:22.024 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:22.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados2 2024-04-20T02:13:22.611 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:24.550 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:24.739 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:24.740 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:13:24.740 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:24.887 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-cephfs 2024-04-20T02:13:25.164 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 kB/s | 4.1 kB 00:00 2024-04-20T02:13:25.244 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:25.545 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-04-20T02:13:27.049 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 4.1 kB 00:00 2024-04-20T02:13:27.457 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 25 kB/s | 4.5 kB 00:00 2024-04-20T02:13:28.115 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:28.337 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:28.338 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:28.338 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:28.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librbd1 2024-04-20T02:13:28.910 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:29.908 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:30.094 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:30.095 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:13:30.095 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:30.234 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rbd 2024-04-20T02:13:30.593 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:31.090 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 kB/s | 4.1 kB 00:00 2024-04-20T02:13:31.445 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T02:13:32.687 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:13:33.043 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T02:13:34.007 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:34.222 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:34.224 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:34.224 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:34.385 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rados 2024-04-20T02:13:34.798 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:35.564 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:35.752 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:35.753 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-04-20T02:13:35.754 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:35.896 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-fuse 2024-04-20T02:13:36.255 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:36.801 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T02:13:37.282 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 21 kB/s | 4.5 kB 00:00 2024-04-20T02:13:38.150 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:13:38.517 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-20T02:13:39.835 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:40.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rgw 2024-04-20T02:13:40.623 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:41.108 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:13:41.108 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:13:41.108 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-20T02:13:41.108 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:13:41.108 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:13:41.108 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 95 k 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 95 k 2024-04-20T02:13:41.109 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 280 k 2024-04-20T02:13:41.110 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:13:41.660 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-fuse-18.2.2-1252.g15eecde2.el8.x86_64.rpm 173 kB/s | 95 kB 00:00 2024-04-20T02:13:41.661 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:13:41.661 INFO:teuthology.orchestra.run.smithi179.stdout:Total 172 kB/s | 95 kB 00:00 2024-04-20T02:13:41.661 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:13:41.672 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:13:41.672 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:13:41.731 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:13:41.739 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:13:41.858 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:13:42.060 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:42.325 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:42.503 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T02:13:42.924 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 25 kB/s | 4.5 kB 00:00 2024-04-20T02:13:45.455 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:45.665 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:45.666 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:45.666 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:45.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-cephfs 2024-04-20T02:13:46.238 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:48.219 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-20T02:13:48.579 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-20T02:13:51.125 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:51.339 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:51.340 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:51.340 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:51.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rbd 2024-04-20T02:13:51.909 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:53.882 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T02:13:54.204 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 79 kB/s | 4.5 kB 00:00 2024-04-20T02:13:56.101 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:13:56.102 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:13:56.715 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:13:56.715 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:13:56.715 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:13:56.715 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:13:56.715 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:13:56.780 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T02:13:56.874 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-mirror 2024-04-20T02:13:57.002 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:13:57.003 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T02:13:57.003 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:13:57.151 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-fuse 2024-04-20T02:13:57.232 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:57.564 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:13:59.512 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T02:13:59.770 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T02:13:59.857 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-20T02:14:00.177 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-04-20T02:14:02.451 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.5 M 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 3.5 M 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 16 M 2024-04-20T02:14:02.452 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:14:02.920 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 95 k 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:14:02.921 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:02.922 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 95 k 2024-04-20T02:14:02.922 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 280 k 2024-04-20T02:14:02.922 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:14:03.375 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-fuse-18.2.2-1252.g15eecde2.el8.x86_64.rpm 210 kB/s | 95 kB 00:00 2024-04-20T02:14:03.375 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:14:03.376 INFO:teuthology.orchestra.run.smithi047.stdout:Total 210 kB/s | 95 kB 00:00 2024-04-20T02:14:03.376 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:14:03.389 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:14:03.389 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:14:03.458 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:14:03.468 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:14:03.592 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:14:03.779 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:03.822 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-mirror-18.2.2-1252.g15eecde2.el8.x86_64.rpm 2.5 MB/s | 3.5 MB 00:01 2024-04-20T02:14:03.822 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:14:03.823 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.5 MB/s | 3.5 MB 00:01 2024-04-20T02:14:03.823 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:14:03.836 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:14:03.836 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:14:03.907 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:14:03.915 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:14:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:04.377 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:14:04.574 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:04.979 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:14.114 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:14.114 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:14:14.684 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:14.684 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:14:14.684 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:14:14.684 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:14.684 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:14:14.856 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-nbd 2024-04-20T02:14:15.232 INFO:teuthology.orchestra.run.smithi179.stdout:Updating Subscription Management repositories. 2024-04-20T02:14:16.966 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T02:14:17.357 INFO:teuthology.orchestra.run.smithi179.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T02:14:17.449 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:17.449 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:14:18.065 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:18.065 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:14:18.065 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:14:18.066 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:18.066 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:14:18.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-mirror 2024-04-20T02:14:18.659 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:14:20.079 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-20T02:14:20.079 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:14:20.079 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-04-20T02:14:20.079 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:14:20.079 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 182 k 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:20.080 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 182 k 2024-04-20T02:14:20.081 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 572 k 2024-04-20T02:14:20.081 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-20T02:14:20.435 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-nbd-18.2.2-1252.g15eecde2.el8.x86_64.rpm 513 kB/s | 182 kB 00:00 2024-04-20T02:14:20.436 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:14:20.436 INFO:teuthology.orchestra.run.smithi179.stdout:Total 511 kB/s | 182 kB 00:00 2024-04-20T02:14:20.436 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-20T02:14:20.447 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-20T02:14:20.447 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-20T02:14:20.505 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-20T02:14:20.514 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-20T02:14:20.662 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-20T02:14:20.677 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-20T02:14:20.873 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:21.062 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T02:14:21.152 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:23.867 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:14:23.867 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.5 M 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:14:23.868 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:23.869 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 3.5 M 2024-04-20T02:14:23.869 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 16 M 2024-04-20T02:14:23.869 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:14:24.980 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-mirror-18.2.2-1252.g15eecde2.el8.x86_64.rpm 3.1 MB/s | 3.5 MB 00:01 2024-04-20T02:14:24.981 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:14:24.981 INFO:teuthology.orchestra.run.smithi047.stdout:Total 3.1 MB/s | 3.5 MB 00:01 2024-04-20T02:14:24.982 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:14:24.997 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:14:24.997 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:14:25.077 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:14:25.087 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:14:25.570 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:14:25.719 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:26.189 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:37.596 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:37.597 INFO:teuthology.orchestra.run.smithi179.stdout:Installed products updated. 2024-04-20T02:14:37.634 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:37.635 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:14:38.183 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:38.183 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-20T02:14:38.184 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:14:38.184 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:14:38.184 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-20T02:14:38.258 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:38.258 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:14:38.259 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:14:38.259 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:38.259 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:14:38.348 DEBUG:teuthology.parallel:result is None 2024-04-20T02:14:38.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-nbd 2024-04-20T02:14:38.889 INFO:teuthology.orchestra.run.smithi047.stdout:Updating Subscription Management repositories. 2024-04-20T02:14:41.023 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 23 kB/s | 4.1 kB 00:00 2024-04-20T02:14:41.372 INFO:teuthology.orchestra.run.smithi047.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-20T02:14:44.135 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 182 k 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-04-20T02:14:44.136 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:14:44.137 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 182 k 2024-04-20T02:14:44.137 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 572 k 2024-04-20T02:14:44.137 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T02:14:44.709 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-nbd-18.2.2-1252.g15eecde2.el8.x86_64.rpm 318 kB/s | 182 kB 00:00 2024-04-20T02:14:44.710 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T02:14:44.711 INFO:teuthology.orchestra.run.smithi047.stdout:Total 317 kB/s | 182 kB 00:00 2024-04-20T02:14:44.711 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T02:14:44.725 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T02:14:44.725 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T02:14:44.795 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T02:14:44.804 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T02:14:44.939 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T02:14:45.116 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:59.766 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T02:14:59.766 INFO:teuthology.orchestra.run.smithi047.stdout:Installed products updated. 2024-04-20T02:15:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:00.383 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T02:15:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T02:15:00.383 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:00.383 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T02:15:00.564 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:00.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T02:15:00.809 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T02:15:00.838 INFO:teuthology.orchestra.run.smithi047.stdout:18.2.2-1252.g15eecde2.el8 2024-04-20T02:15:00.838 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1252.g15eecde2.el8 2024-04-20T02:15:00.838 INFO:teuthology.task.install:The correct ceph version 18.2.2-1252.g15eecde2 is installed. 2024-04-20T02:15:00.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T02:15:01.071 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T02:15:01.099 INFO:teuthology.orchestra.run.smithi179.stdout:18.2.2-1252.g15eecde2.el8 2024-04-20T02:15:01.099 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1252.g15eecde2.el8 2024-04-20T02:15:01.099 INFO:teuthology.task.install:The correct ceph version 18.2.2-1252.g15eecde2 is installed. 2024-04-20T02:15:01.101 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T02:15:01.101 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:01.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T02:15:01.138 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:01.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T02:15:01.173 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T02:15:01.173 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:01.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T02:15:01.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T02:15:01.286 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:01.286 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T02:15:01.317 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T02:15:01.386 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T02:15:01.387 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:01.387 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T02:15:01.422 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T02:15:01.498 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:01.498 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T02:15:01.527 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T02:15:01.596 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T02:15:01.596 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:01.596 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T02:15:01.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T02:15:01.704 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:01.704 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T02:15:01.732 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T02:15:01.800 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T02:15:01.888 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T02:15:01.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-04-20T02:15:01.890 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /var/log/ceph 2024-04-20T02:15:01.922 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T02:15:01.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T02:15:01.965 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T02:15:01.997 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T02:15:01.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T02:15:02.040 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T02:15:02.082 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T02:15:02.082 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 5}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_FULL\\)', '\\(SLOW_OPS\\)', '\\(MON_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T02:15:02.082 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191', 'branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7664191', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2024-04-18-1351-reef'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 5}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_FULL\\)', '\\(SLOW_OPS\\)', '\\(MON_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'slow request'], 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 6867, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8EhPJpNg17ofpe396Fg19EHzoBdX1wmf3XBCqPWWDyrELXEamWW2QFMe3lZkKrzZyhM43plttJP5i42k2gZ9g=', 'smithi179.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAU+o5zdjLSk5hRO93XP+kIsoKn9Mr6zn7rU+cIPoMs6ca/AQxNR4dTX8Fni+/Fi0gXMqndJPbAxG1yMJidQozQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'store_thrash': True, 'thrash_delay': 1, 'thrash_many': True}}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_01:10:46', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T02:15:02.083 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T02:15:02.115 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T02:15:02.135 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T02:15:02.177 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T02:15:02.210 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:02.210 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T02:15:02.273 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T02:15:02.273 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 181802 Links: 1 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 02:14:25.911121664 +0000 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 02:05:13.386137080 +0000 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 02:05:13.386137080 +0000 2024-04-20T02:15:02.335 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-04-20T02:15:02.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T02:15:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T02:15:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T02:15:02.408 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000107129 s, 4.8 MB/s 2024-04-20T02:15:02.409 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T02:15:02.473 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 179786 Links: 1 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 02:14:25.911121664 +0000 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 02:05:13.764128727 +0000 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 02:05:13.764128727 +0000 2024-04-20T02:15:02.536 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-04-20T02:15:02.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T02:15:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T02:15:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T02:15:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000124898 s, 4.1 MB/s 2024-04-20T02:15:02.611 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T02:15:02.673 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 179883 Links: 1 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 02:14:25.911121664 +0000 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 02:05:14.142120373 +0000 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 02:05:14.142120373 +0000 2024-04-20T02:15:02.735 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-04-20T02:15:02.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T02:15:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T02:15:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T02:15:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000127033 s, 4.0 MB/s 2024-04-20T02:15:02.809 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T02:15:02.873 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-04-20T02:15:02.935 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T02:15:02.935 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:02.935 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 179943 Links: 1 2024-04-20T02:15:02.935 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:02.935 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:02.936 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 02:14:25.911121664 +0000 2024-04-20T02:15:02.936 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 02:05:14.506112329 +0000 2024-04-20T02:15:02.936 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 02:05:14.506112329 +0000 2024-04-20T02:15:02.936 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: - 2024-04-20T02:15:02.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T02:15:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T02:15:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T02:15:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00011374 s, 4.5 MB/s 2024-04-20T02:15:03.010 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T02:15:03.072 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:03.072 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:03.073 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T02:15:03.092 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T02:15:03.092 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 176797 Links: 1 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-20 02:14:04.745483510 +0000 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-20 02:04:45.018516421 +0000 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-20 02:04:45.018516421 +0000 2024-04-20T02:15:03.151 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-04-20T02:15:03.152 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T02:15:03.218 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-20T02:15:03.218 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-20T02:15:03.218 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.00024807 s, 2.1 MB/s 2024-04-20T02:15:03.219 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T02:15:03.278 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 179671 Links: 1 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-20 02:14:04.745483510 +0000 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-20 02:04:45.371508652 +0000 2024-04-20T02:15:03.339 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-20 02:04:45.371508652 +0000 2024-04-20T02:15:03.340 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-04-20T02:15:03.340 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T02:15:03.405 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-20T02:15:03.406 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-20T02:15:03.406 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000243026 s, 2.1 MB/s 2024-04-20T02:15:03.407 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T02:15:03.465 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-04-20T02:15:03.523 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T02:15:03.523 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:03.523 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 179797 Links: 1 2024-04-20T02:15:03.523 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:03.524 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:03.524 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-20 02:14:04.745483510 +0000 2024-04-20T02:15:03.525 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-20 02:04:45.755500201 +0000 2024-04-20T02:15:03.525 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-20 02:04:45.755500201 +0000 2024-04-20T02:15:03.525 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-04-20T02:15:03.525 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T02:15:03.589 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-20T02:15:03.589 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-20T02:15:03.589 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000190193 s, 2.7 MB/s 2024-04-20T02:15:03.590 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T02:15:03.648 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-04-20T02:15:03.705 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 179896 Links: 1 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-20 02:14:04.745483510 +0000 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-20 02:04:46.116492256 +0000 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-20 02:04:46.116492256 +0000 2024-04-20T02:15:03.706 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2024-04-20T02:15:03.706 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T02:15:03.770 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-20T02:15:03.771 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-20T02:15:03.771 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000238912 s, 2.1 MB/s 2024-04-20T02:15:03.772 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T02:15:03.829 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:03.829 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-04-20T02:15:03.829 INFO:tasks.ceph:Generating config... 2024-04-20T02:15:03.831 INFO:tasks.ceph:[client] debug monc = 20 2024-04-20T02:15:03.831 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] debug ms = 1 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] debug objecter = 20 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] debug rados = 20 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-20T02:15:03.831 INFO:tasks.ceph:[global] ms type = async 2024-04-20T02:15:03.831 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T02:15:03.831 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T02:15:03.831 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T02:15:03.831 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-04-20T02:15:03.832 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-04-20T02:15:03.832 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-04-20T02:15:03.832 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-04-20T02:15:03.832 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T02:15:03.832 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T02:15:03.832 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd class default list = * 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd class load list = * 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-20T02:15:03.833 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T02:15:03.834 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-20T02:15:03.834 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-20T02:15:03.834 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T02:15:03.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:03.905 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T02:15:03.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T02:15:03.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T02:15:03.993 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]'), ('mon.c', '[v2:172.21.15.47:3302,v1:172.21.15.47:6791]'), ('mon.d', '[v2:172.21.15.47:3303,v1:172.21.15.47:6792]'), ('mon.e', '[v2:172.21.15.47:3304,v1:172.21.15.47:6793]'), ('mon.f', '172.21.15.179'), ('mon.g', '[v2:172.21.15.179:3301,v1:172.21.15.179:6790]'), ('mon.h', '[v2:172.21.15.179:3302,v1:172.21.15.179:6791]'), ('mon.i', '[v2:172.21.15.179:3303,v1:172.21.15.179:6792]')] 2024-04-20T02:15:03.994 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],[v2:172.21.15.47:3302,v1:172.21.15.47:6791],[v2:172.21.15.47:3303,v1:172.21.15.47:6792],[v2:172.21.15.47:3304,v1:172.21.15.47:6793],172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790],[v2:172.21.15.179:3302,v1:172.21.15.179:6791],[v2:172.21.15.179:3303,v1:172.21.15.179:6792]', 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-04-20T02:15:03.995 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:03.995 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T02:15:04.055 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --addv b '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]' --addv c '[v2:172.21.15.47:3302,v1:172.21.15.47:6791]' --addv d '[v2:172.21.15.47:3303,v1:172.21.15.47:6792]' --addv e '[v2:172.21.15.47:3304,v1:172.21.15.47:6793]' --add f 172.21.15.179 --addv g '[v2:172.21.15.179:3301,v1:172.21.15.179:6790]' --addv h '[v2:172.21.15.179:3302,v1:172.21.15.179:6791]' --addv i '[v2:172.21.15.179:3303,v1:172.21.15.179:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = pacific 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:04.175 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 16 (pacific) 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 3 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:3: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:4: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:5: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:6: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:7: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:8: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:04.176 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-04-20T02:15:04.177 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T02:15:04.197 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 72abca9e-d588-4ab7-bb3a-a08105100daf... 2024-04-20T02:15:04.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T02:15:04.239 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T02:15:04.285 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2024-04-20T02:15:04.286 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2024-04-20T02:15:04.287 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.288 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],[v2:172.21.15.47:3302,v1:172.21.15.47:6791],[v2:172.21.15.47:3303,v1:172.21.15.47:6792],[v2:172.21.15.47:3304,v1:172.21.15.47:6793],172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790],[v2:172.21.15.179:3302,v1:172.21.15.179:6791],[v2:172.21.15.179:3303,v1:172.21.15.179:6792]" 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: debug objecter = 20 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: debug rados = 20 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: mon client directed command retry = 5 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: mon election default strategy = 3 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: ms inject socket failures = 5000 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: ms type = async 2024-04-20T02:15:04.289 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock profile = high_recovery_ops 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2024-04-20T02:15:04.290 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.291 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: bluefs allocator = hybrid 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore allocator = hybrid 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore zero block detection = True 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 20 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 20 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2024-04-20T02:15:04.292 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 10 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd class default list = * 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd class load list = * 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify cached snaps = True 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify missing on start = True 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd mclock override recovery settings = True 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd op queue = debug_random 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: osd op queue cut off = debug_random 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2024-04-20T02:15:04.293 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2024-04-20T02:15:04.294 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: # 1m isn't quite enough 2024-04-20T02:15:04.295 INFO:teuthology.orchestra.run.smithi179.stdout: mon_down_mkfs_grace = 2m 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_filestore_osds = false 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon min osdmap epochs = 25 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon osdmap full prune interval = 2 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon osdmap full prune min = 15 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon osdmap full prune txsize = 2 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon scrub inject crc mismatch = 0.01 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon scrub inject missing keys = 0.05 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon scrub interval = 300 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on pool no app = False 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: paxos service trim min = 5 2024-04-20T02:15:04.296 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: debug monc = 20 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.d] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.e] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.f] 2024-04-20T02:15:04.297 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.g] 2024-04-20T02:15:04.298 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.h] 2024-04-20T02:15:04.298 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.i] 2024-04-20T02:15:04.298 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-04-20T02:15:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-04-20T02:15:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T02:15:04.298 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T02:15:04.299 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-04-20T02:15:04.300 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-04-20T02:15:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],[v2:172.21.15.47:3302,v1:172.21.15.47:6791],[v2:172.21.15.47:3303,v1:172.21.15.47:6792],[v2:172.21.15.47:3304,v1:172.21.15.47:6793],172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790],[v2:172.21.15.179:3302,v1:172.21.15.179:6791],[v2:172.21.15.179:3303,v1:172.21.15.179:6792]" 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: debug objecter = 20 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: debug rados = 20 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: mon client directed command retry = 5 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: mon election default strategy = 3 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: ms inject socket failures = 5000 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: ms type = async 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.302 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-04-20T02:15:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: bluefs allocator = hybrid 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = hybrid 2024-04-20T02:15:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore zero block detection = True 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: osd class default list = * 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: osd class load list = * 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify cached snaps = True 2024-04-20T02:15:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify missing on start = True 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock override recovery settings = True 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: osd op queue = debug_random 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: osd op queue cut off = debug_random 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-04-20T02:15:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T02:15:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon min osdmap epochs = 25 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon osdmap full prune interval = 2 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon osdmap full prune min = 15 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon osdmap full prune txsize = 2 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon scrub inject crc mismatch = 0.01 2024-04-20T02:15:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: mon scrub inject missing keys = 0.05 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: mon scrub interval = 300 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on pool no app = False 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: paxos service trim min = 5 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: debug monc = 20 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-04-20T02:15:04.309 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.d] 2024-04-20T02:15:04.310 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.e] 2024-04-20T02:15:04.310 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.f] 2024-04-20T02:15:04.310 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.g] 2024-04-20T02:15:04.310 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.h] 2024-04-20T02:15:04.310 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.i] 2024-04-20T02:15:04.311 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T02:15:04.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T02:15:04.379 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T02:15:04.380 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:04.380 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T02:15:04.399 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:04.399 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T02:15:04.459 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:15:04.460 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:04.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T02:15:04.460 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T02:15:04.547 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:04.547 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:04.609 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:15:04.610 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:04.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T02:15:04.610 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T02:15:04.648 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:04.648 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:04.705 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T02:15:04.705 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T02:15:04.706 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T02:15:04.810 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T02:15:04.813 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T02:15:04.813 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T02:15:04.813 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T02:15:04.857 INFO:teuthology.orchestra.run.smithi179.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T02:15:04.870 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T02:15:04.870 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-04-20T02:15:04.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T02:15:04.906 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:04.906 INFO:tasks.ceph:role: osd.0 2024-04-20T02:15:04.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:15:04.907 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T02:15:04.981 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T02:15:04.982 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T02:15:05.004 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T02:15:05.006 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T02:15:05.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T02:15:05.096 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T02:15:05.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T02:15:05.211 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:05.211 INFO:tasks.ceph:role: osd.1 2024-04-20T02:15:05.211 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:15:05.211 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T02:15:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T02:15:05.286 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T02:15:05.308 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T02:15:05.310 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T02:15:05.310 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T02:15:05.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T02:15:05.437 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T02:15:05.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:05.512 INFO:tasks.ceph:role: osd.2 2024-04-20T02:15:05.513 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:15:05.513 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T02:15:05.587 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T02:15:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T02:15:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T02:15:05.588 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T02:15:05.610 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T02:15:05.612 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T02:15:05.612 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T02:15:05.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T02:15:05.737 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:05.838+0000 7f35336b47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T02:15:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:05.839+0000 7f35336b47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T02:15:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:05.839+0000 7f35336b47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T02:15:05.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:05.839+0000 7f35336b47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T02:15:05.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:05.839+0000 7f35336b47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T02:15:05.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:05.839+0000 7f35336b47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T02:15:08.690 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T02:15:08.727 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:08.827+0000 7f86b5f297c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T02:15:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:08.827+0000 7f86b5f297c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T02:15:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:08.827+0000 7f86b5f297c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T02:15:08.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:08.827+0000 7f86b5f297c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T02:15:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:08.827+0000 7f86b5f297c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T02:15:08.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:08.828+0000 7f86b5f297c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T02:15:11.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T02:15:11.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:11.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:11.811+0000 7f50f99f47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T02:15:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:11.811+0000 7f50f99f47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T02:15:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:11.811+0000 7f50f99f47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T02:15:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:11.811+0000 7f50f99f47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T02:15:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:11.811+0000 7f50f99f47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T02:15:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:11.811+0000 7f50f99f47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T02:15:14.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T02:15:14.691 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T02:15:14.725 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:14.725 INFO:tasks.ceph:role: osd.3 2024-04-20T02:15:14.725 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:15:14.725 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T02:15:14.802 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T02:15:14.803 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T02:15:15.229 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-04-20T02:15:15.231 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-04-20T02:15:15.231 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-04-20T02:15:15.421 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T02:15:15.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T02:15:15.533 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:15.533 INFO:tasks.ceph:role: osd.4 2024-04-20T02:15:15.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:15:15.533 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T02:15:15.610 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T02:15:15.610 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T02:15:15.610 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T02:15:15.611 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T02:15:16.030 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-04-20T02:15:16.032 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-04-20T02:15:16.032 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-04-20T02:15:16.145 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T02:15:16.179 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T02:15:16.247 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-04-20T02:15:16.248 INFO:tasks.ceph:role: osd.5 2024-04-20T02:15:16.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:15:16.248 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T02:15:16.323 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T02:15:16.324 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T02:15:16.770 INFO:teuthology.orchestra.run.smithi179.stdout:Discarding blocks...Done. 2024-04-20T02:15:16.771 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2024-04-20T02:15:16.772 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-04-20T02:15:16.942 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T02:15:16.981 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:17.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:17.072+0000 7fc611e527c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T02:15:17.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:17.075+0000 7fc611e527c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T02:15:17.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:17.075+0000 7fc611e527c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T02:15:17.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:17.075+0000 7fc611e527c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T02:15:17.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:17.075+0000 7fc611e527c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T02:15:17.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:17.082+0000 7fc611e527c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T02:15:19.933 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T02:15:19.965 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:20.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:20.057+0000 7f1f5829c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T02:15:20.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:20.057+0000 7f1f5829c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T02:15:20.057 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:20.057+0000 7f1f5829c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T02:15:20.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:20.057+0000 7f1f5829c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T02:15:20.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:20.057+0000 7f1f5829c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T02:15:20.058 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:20.058+0000 7f1f5829c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T02:15:22.905 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T02:15:22.937 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:23.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:23.030+0000 7f3903cc57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T02:15:23.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:23.030+0000 7f3903cc57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T02:15:23.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:23.030+0000 7f3903cc57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T02:15:23.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:23.030+0000 7f3903cc57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T02:15:23.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:23.030+0000 7f3903cc57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T02:15:23.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:23.031+0000 7f3903cc57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T02:15:25.878 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T02:15:25.910 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T02:15:25.910 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:25.910 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T02:15:25.947 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:25.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T02:15:26.021 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:26.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T02:15:26.097 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:26.097 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T02:15:26.129 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:26.129 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T02:15:26.200 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:26.200 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T02:15:26.270 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:26.270 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T02:15:26.339 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:26.339 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T02:15:26.397 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T02:15:26.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T02:15:26.400 DEBUG:teuthology.orchestra.run.smithi179:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T02:15:26.441 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-04-20T02:15:26.441 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQApJSNmEZ0SMhAAnw9+craIXSZm7HcsX/cweg== 2024-04-20T02:15:26.441 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-04-20T02:15:26.441 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAsJSNmgbxlMRAAGU0Htd2cHJrJL+M3gdVQeQ== 2024-04-20T02:15:26.441 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-04-20T02:15:26.441 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAvJSNmvB1tMBAAMsBbnhPdXfWzlP70mUjPRg== 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAoJSNmxjxzMBAAv5Zcu4KYTq5ZMTYfn3mFXw== 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQA1JSNmR8hsBBAA26bsZypaHLYtlJb2Uo7xJA== 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQA4JSNmkZeIAxAAqXK7E+nRZnX0hitvj1Tt4w== 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQA7JSNmGNLqARAABn3pVqRcSj0WpGg6q3T1sg== 2024-04-20T02:15:26.442 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-20T02:15:26.443 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQAoJSNm8I05MxAAKBene8zH1yYEyTHk8wwDpw== 2024-04-20T02:15:26.464 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.0] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQApJSNmEZ0SMhAAnw9+craIXSZm7HcsX/cweg== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.1] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQAsJSNmgbxlMRAAGU0Htd2cHJrJL+M3gdVQeQ== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.2] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQAvJSNmvB1tMBAAMsBbnhPdXfWzlP70mUjPRg== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.x] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQAoJSNmxjxzMBAAv5Zcu4KYTq5ZMTYfn3mFXw== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.3] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQA1JSNmR8hsBBAA26bsZypaHLYtlJb2Uo7xJA== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.4] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQA4JSNmkZeIAxAAqXK7E+nRZnX0hitvj1Tt4w== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.5] 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQA7JSNmGNLqARAABn3pVqRcSj0WpGg6q3T1sg== 2024-04-20T02:15:26.465 INFO:teuthology.orchestra.run.smithi179.stdout:[client.0] 2024-04-20T02:15:26.466 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQAoJSNm8I05MxAAKBene8zH1yYEyTHk8wwDpw== 2024-04-20T02:15:26.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.487 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.564 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.624 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.626 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.685 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T02:15:26.687 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T02:15:26.745 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.748 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.805 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.807 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.867 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T02:15:26.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T02:15:26.928 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T02:15:26.988 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T02:15:26.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T02:15:27.018 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:27.127 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T02:15:27.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T02:15:27.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:27.344 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T02:15:27.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T02:15:27.452 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:27.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T02:15:27.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-04-20T02:15:27.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:27.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-04-20T02:15:27.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-04-20T02:15:27.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:28.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-04-20T02:15:28.034 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-04-20T02:15:28.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:28.174 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-04-20T02:15:28.202 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-04-20T02:15:28.271 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:28.372 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-04-20T02:15:28.398 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-04-20T02:15:28.465 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:28.564 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-04-20T02:15:28.590 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-04-20T02:15:28.657 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T02:15:28.756 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-04-20T02:15:28.784 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:28.786 DEBUG:teuthology.orchestra.run.smithi179:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T02:15:28.840 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T02:15:28.841 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T02:15:28.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T02:15:28.848 INFO:tasks.ceph.mon.a:Started 2024-04-20T02:15:28.848 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T02:15:28.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T02:15:28.850 INFO:tasks.ceph.mon.b:Started 2024-04-20T02:15:28.850 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T02:15:28.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T02:15:28.853 INFO:tasks.ceph.mon.c:Started 2024-04-20T02:15:28.853 INFO:tasks.ceph.mon.d:Restarting daemon 2024-04-20T02:15:28.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-04-20T02:15:28.855 INFO:tasks.ceph.mon.d:Started 2024-04-20T02:15:28.855 INFO:tasks.ceph.mon.e:Restarting daemon 2024-04-20T02:15:28.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-04-20T02:15:28.858 INFO:tasks.ceph.mon.e:Started 2024-04-20T02:15:28.858 INFO:tasks.ceph.mon.f:Restarting daemon 2024-04-20T02:15:28.858 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-04-20T02:15:28.883 INFO:tasks.ceph.mon.f:Started 2024-04-20T02:15:28.883 INFO:tasks.ceph.mon.g:Restarting daemon 2024-04-20T02:15:28.883 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-04-20T02:15:28.885 INFO:tasks.ceph.mon.g:Started 2024-04-20T02:15:28.885 INFO:tasks.ceph.mon.h:Restarting daemon 2024-04-20T02:15:28.885 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-04-20T02:15:28.887 INFO:tasks.ceph.mon.h:Started 2024-04-20T02:15:28.887 INFO:tasks.ceph.mon.i:Restarting daemon 2024-04-20T02:15:28.887 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-04-20T02:15:28.889 INFO:tasks.ceph.mon.i:Started 2024-04-20T02:15:28.889 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T02:15:28.889 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T02:15:28.889 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T02:15:28.891 INFO:tasks.ceph.mgr.x:Started 2024-04-20T02:15:28.891 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T02:15:28.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.064+0000 7f745f0a1700 10 monclient: get_monmap_and_config 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.064+0000 7f745f0a1700 10 monclient: build_initial_monmap 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.064+0000 7f745f0a1700 10 monclient: monmap: 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:29.066046+0000 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:29.066046+0000 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:29.064 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:29.065 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.065+0000 7f745f0a1700 1 Processor -- start 2024-04-20T02:15:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.065+0000 7f745f0a1700 1 -- start start 2024-04-20T02:15:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.065+0000 7f745f0a1700 10 monclient: init 2024-04-20T02:15:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient: _add_conns ranks=[5,4,2,9,0,8,10,7,1,3,6] 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): picked mon.noname-h con 0x7f745809fc10 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): picked mon.noname-c con 0x7f745809fff0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): picked mon.noname-b con 0x7f74580a03d0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745809def0 con 0x7f74580a03d0 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74580a3dd0 con 0x7f745809fff0 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f745809dc10 con 0x7f745809fc10 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745f0a1700 10 monclient(hunting): authenticate will time out at 1608.467294s 2024-04-20T02:15:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54618/0 (socket says 172.21.15.47:54618) 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 learned_addr learned my addr 172.21.15.47:0/1265490336 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 10 monclient(hunting): get_auth_request con 0x7f74580a03d0 auth_method 0 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f745809fff0 auth_method 0 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f745809fc10 auth_method 0 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.066+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f745809fff0 auth_method 0 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:29.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 10 monclient(hunting): get_auth_request con 0x7f74580a03d0 auth_method 0 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f745809fc10 auth_method 0 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:15:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:15:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:15:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.267+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:15:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f745809fff0 auth_method 0 2024-04-20T02:15:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 10 monclient(hunting): get_auth_request con 0x7f74580a03d0 auth_method 0 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:29.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f745809fc10 auth_method 0 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:29.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:29.668+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 10 monclient(hunting): get_auth_request con 0x7f74580a03d0 auth_method 0 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f745809fff0 auth_method 0 2024-04-20T02:15:30.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f745809fc10 auth_method 0 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:30.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:15:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.469+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.470+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:30.470+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:15:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): tick 2024-04-20T02:15:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:15:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 msgr2=0x7f7458135d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f745809fc10 0x7f7458135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 msgr2=0x7f74580a1f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f745809fff0 0x7f74580a1f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 msgr2=0x7f74580a2450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a03d0 0x7f74580a2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient: _add_conns ranks=[7,8,0,10,6,9,3,1] 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): picked mon.noname-i con 0x7f74400038b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): picked mon.noname-e con 0x7f74400062e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7440008db0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:32.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 -- 172.21.15.47:0/1265490336 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580751e0 con 0x7f7440008db0 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 -- 172.21.15.47:0/1265490336 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f745809def0 con 0x7f74400038b0 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 1 -- 172.21.15.47:0/1265490336 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f74580a3dd0 con 0x7f74400062e0 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f74567fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f74400062e0 auth_method 0 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.066+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:32.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f74400038b0 auth_method 0 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:32.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.067+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f74400062e0 auth_method 0 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f74400038b0 auth_method 0 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:32.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:32.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.268+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f74400062e0 auth_method 0 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:32.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f74400038b0 auth_method 0 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:32.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:32.669+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:15:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f74400062e0 auth_method 0 2024-04-20T02:15:33.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f74400038b0 auth_method 0 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.470+0000 7f745ce3d700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:15:33.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.471+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:15:33.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:33.471+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:15:34.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.267+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630763249 0 0) 0x7f74580751e0 con 0x7f7440008db0 2024-04-20T02:15:34.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.267+0000 7f74577fe700 10 monclient(hunting): _negotiate 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.267+0000 7f74577fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.267+0000 7f74577fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.267+0000 7f74577fe700 10 monclient(hunting): my global_id is 4104 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.267+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74440035b0 con 0x7f7440008db0 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1060636259 0 0) 0x7f74440035b0 con 0x7f7440008db0 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 msgr2=0x7f7440008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74400062e0 0x7f7440008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 msgr2=0x7f7440005d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74400038b0 0x7f7440005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:15:34.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f745d63e700 1 -- 172.21.15.47:0/1265490336 reap_dead start 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744000dc60 con 0x7f7440008db0 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f7450009240 con 0x7f7440008db0 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: dump: 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.273 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 >> v1:172.21.15.47:6789/0 conn(0x7f7440008db0 legacy=0x7f744000b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient: _add_conns ranks=[6,8,4,7,0,1,2,5,3] 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7444004250 0x7f7444004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): picked mon.d con 0x7f7444004250 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:34.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7444004e60 0x7f74440052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): picked mon.e con 0x7f7444004e60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444009c40 0x7f744400c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): picked mon.c con 0x7f7444009c40 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f745809def0 con 0x7f7444009c40 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f74580a3dd0 con 0x7f7444004250 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f744400c9e0 con 0x7f7444004e60 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f74577fe700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7444004e60 0x7f74440052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f7444004e60 auth_method 0 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f7457fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.268+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7444004250 0x7f7444004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f7444004250 auth_method 0 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444009c40 0x7f744400c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): get_auth_request con 0x7f7444009c40 auth_method 0 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:15:34.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745d63e700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745ce3d700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient(hunting): handle_auth_done global_id 4104 payload 293 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7444004250 msgr2=0x7f7444004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7444004250 0x7f7444004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444009c40 msgr2=0x7f744400c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444009c40 0x7f744400c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 monclient: found mon.e 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 -- 172.21.15.47:0/1265490336 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440035b0 con 0x7f7444004e60 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745f0a1700 5 monclient: authenticate success, global_id 4104 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f745f0a1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.269+0000 7f7457fff700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7444004e60 0x7f74440052e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f744c0198b0 tx=0x7f7444005880 comp rx=0 tx=0).ready entity=mon.8 client_cookie=225f1aac1a246046 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.270+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c016d30 con 0x7f7444004e60 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.270+0000 7f74577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.270+0000 7f74577fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:34.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.270+0000 7f74577fe700 10 monclient: dump: 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.279 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f745f0a1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c00c040 con 0x7f7444004e60 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f74577fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f74577fe700 1 -- 172.21.15.47:0/1265490336 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c016e90 con 0x7f7444004e60 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f745f0a1700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f745f0a1700 4 set_mon_vals no callback set 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f74577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f74577fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.271+0000 7f74577fe700 10 monclient: dump: 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 10 monclient: shutdown 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 -- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7444004e60 msgr2=0x7f74440052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7444004e60 0x7f74440052e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f744c0198b0 tx=0x7f7444005880 comp rx=0 tx=0).stop 2024-04-20T02:15:34.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 -- 172.21.15.47:0/1265490336 shutdown_connections 2024-04-20T02:15:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444009c40 0x7f744400c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7444004e60 0x7f74440052e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1265490336 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7444004250 0x7f7444004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 -- 172.21.15.47:0/1265490336 >> 172.21.15.47:0/1265490336 conn(0x7f74580a3000 msgr2=0x7f74580a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 -- 172.21.15.47:0/1265490336 shutdown_connections 2024-04-20T02:15:34.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.272+0000 7f745f0a1700 1 -- 172.21.15.47:0/1265490336 wait complete. 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.273+0000 7f745f0a1700 10 monclient: build_initial_monmap 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.273+0000 7f745f0a1700 10 monclient: monmap: 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:34.287 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 1 librados: starting msgr at 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 1 librados: starting objecter 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 1 Processor -- start 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 1 -- start start 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 1 librados: setting wanted keys 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 1 librados: calling monclient init 2024-04-20T02:15:34.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.274+0000 7f745f0a1700 10 monclient: init 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient: _add_conns ranks=[8,4,3,2,7,5,0,6,1] 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 0x7f7458097120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): picked mon.noname-e con 0x7f7440008db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 0x7f7458097660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): picked mon.noname-c con 0x7f7444004250 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7444004e60 0x7f745808bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): picked mon.noname-g con 0x7f7444004e60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74581d15f0 con 0x7f7444004e60 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7458097ba0 con 0x7f7444004250 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7458134d90 con 0x7f7440008db0 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745f0a1700 10 monclient(hunting): authenticate will time out at 1613.676375s 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 0x7f7458097120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 0x7f7458097660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 0x7f7458097120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:38868/0 (socket says 172.21.15.47:38868) 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 0x7f7458097660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37966/0 (socket says 172.21.15.47:37966) 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 1 -- 172.21.15.47:0/1195654932 learned_addr learned my addr 172.21.15.47:0/1195654932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): get_auth_request con 0x7f7440008db0 auth_method 0 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): get_auth_request con 0x7f7444004250 auth_method 0 2024-04-20T02:15:34.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7444004e60 0x7f745808bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745ce3d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): get_auth_request con 0x7f7444004e60 auth_method 0 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f745d63e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient(hunting): handle_auth_done global_id 4117 payload 824 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 -- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 msgr2=0x7f7458097120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 0x7f7458097120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 -- 172.21.15.47:0/1195654932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7444004e60 msgr2=0x7f745808bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7444004e60 0x7f745808bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.275+0000 7f7457fff700 1 monclient: found mon.noname-c 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f7457fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f7457fff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f7457fff700 1 -- 172.21.15.47:0/1195654932 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745809cae0 con 0x7f7444004250 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745ce3d700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 0x7f7458097120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:34.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f7457fff700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 0x7f7458097660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f744c028000 tx=0x7f744c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 client.?.objecter ms_handle_connect 0x7f7444004250 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c0044a0 con 0x7f7444004250 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: dump: 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.293 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c0036a0 con 0x7f7444004250 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 5 monclient: authenticate success, global_id 4117 2024-04-20T02:15:34.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 10 monclient: _renew_subs 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745808e0b0 con 0x7f7444004250 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744c003c80 con 0x7f7444004250 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 monclient: dump: 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.295 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f7456ffd700 4 set_mon_vals no callback set 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 10 client.4117.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 10 monclient: _renew_subs 2024-04-20T02:15:34.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745808d150 con 0x7f7444004250 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f745f0a1700 1 librados: init done 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f744c026070 con 0x7f7444004250 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f744c029a00 con 0x7f7444004250 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 10 client.4117.objecter ms_dispatch 0x7f7458001550 osd_map(1..1 src has 1..1) v4 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 3 client.4117.objecter handle_osd_map got epochs [1,1] > 0 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 3 client.4117.objecter handle_osd_map decoding full epoch 1 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.276+0000 7f74557fa700 20 client.4117.objecter dump_active .. 0 homeless 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.277+0000 7f745f0a1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.277+0000 7f745f0a1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.277+0000 7f745f0a1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.277+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745808e5d0 con 0x7f7444004250 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.280+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f745808e5d0 con 0x7f7444004250 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.280+0000 7f74557fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:34.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.280+0000 7f74557fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:34.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.362+0000 7f745f0a1700 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-04-20T02:15:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.362+0000 7f745f0a1700 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-04-20T02:15:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.362+0000 7f745f0a1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:34.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.362+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f745808e790 con 0x7f7444004250 2024-04-20T02:15:34.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.366+0000 7f74557fa700 1 -- 172.21.15.47:0/1195654932 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f744c036020 con 0x7f7444004250 2024-04-20T02:15:34.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.366+0000 7f74557fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-04-20T02:15:34.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.366+0000 7f74557fa700 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 10 librados: watch_flush enter 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 10 librados: watch_flush exit 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 20 client.4117.objecter shutdown clearing up homeless session... 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 10 client.4117.objecter successfully canceled tick 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 10 monclient: shutdown 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 msgr2=0x7f7458097660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 0x7f7458097660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f744c028000 tx=0x7f744c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 shutdown_connections 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7444004e60 0x7f745808bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7440008db0 0x7f7458097120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 --2- 172.21.15.47:0/1195654932 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7444004250 0x7f7458097660 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 >> 172.21.15.47:0/1195654932 conn(0x7f74580a3000 msgr2=0x7f74580994c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 shutdown_connections 2024-04-20T02:15:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 -- 172.21.15.47:0/1195654932 wait complete. 2024-04-20T02:15:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:34.367+0000 7f745f0a1700 1 librados: shutdown 2024-04-20T02:15:34.368 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-04-20T02:15:34.380 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T02:15:34.380 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T02:15:34.380 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:34.380 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T02:15:34.411 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:34.411 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T02:15:34.483 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T02:15:34.483 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T02:15:34.559 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:34.559 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T02:15:34.585 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:34.585 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T02:15:34.652 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:15:34.653 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T02:15:34.705 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:34.707+0000 7f86c2c98200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T02:15:34.719 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new a1c1b335-9be6-43d2-94b3-8cd5fb042a10 0 2024-04-20T02:15:34.789 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:34.790+0000 7f86c2c98200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T02:15:34.866 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:34.867+0000 7f86c2c98200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T02:15:34.878 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.880+0000 7fbaee2d9700 10 monclient: get_monmap_and_config 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.880+0000 7fbaee2d9700 10 monclient: build_initial_monmap 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.880+0000 7fbaee2d9700 10 monclient: monmap: 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:34.881089+0000 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:34.881089+0000 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:34.879 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:34.880 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 Processor -- start 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 -- start start 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient: init 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient: _add_conns ranks=[10,7,1,2,5,8,6,9,4,0,3] 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbae8134ad0 0x7fbae8134eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): picked mon.noname-f con 0x7fbae8134ad0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae80a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): picked mon.noname-i con 0x7fbae8072070 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbae80a2d40 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:34.889 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae80751e0 con 0x7fbae80a2d40 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae80a12a0 con 0x7fbae8134ad0 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbae809ec90 con 0x7fbae8072070 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaee2d9700 10 monclient(hunting): authenticate will time out at 1598.288743s 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbae8134ad0 0x7fbae8134eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaec876700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbae80a2d40 0x7fbae813b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:45396/0 (socket says 172.21.15.179:45396) 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbaec876700 1 -- 172.21.15.179:0/2386506391 learned_addr learned my addr 172.21.15.179:0/2386506391 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 10 monclient(hunting): get_auth_request con 0x7fbae8134ad0 auth_method 0 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae77fe700 1 --2- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae80a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae77fe700 10 monclient(hunting): get_auth_request con 0x7fbae8072070 auth_method 0 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.881+0000 7fbae7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 10 monclient(hunting): handle_auth_done global_id 4123 payload 274 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 1 -- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 msgr2=0x7fbae80a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 1 --2- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae80a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 1 -- 172.21.15.179:0/2386506391 >> v1:172.21.15.179:6789/0 conn(0x7fbae80a2d40 legacy=0x7fbae813b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 1 monclient: found mon.noname-f 2024-04-20T02:15:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 1 -- 172.21.15.179:0/2386506391 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae809ff40 con 0x7fbae8134ad0 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae7fff700 1 --2- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8134eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009840 tx=0x7fbad8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e99c2af6747fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 1 -- 172.21.15.179:0/2386506391 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbad8011aa0 con 0x7fbae8134ad0 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: dump: 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:34.893 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbaee2d9700 5 monclient: authenticate success, global_id 4123 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 1 -- 172.21.15.179:0/2386506391 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad8011c00 con 0x7fbae8134ad0 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbaee2d9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbaee2d9700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbaee2d9700 4 set_mon_vals no callback set 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 1 -- 172.21.15.179:0/2386506391 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbad8011d70 con 0x7fbae8134ad0 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.882+0000 7fbae6ffd700 10 monclient: dump: 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 10 monclient: shutdown 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 -- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 msgr2=0x7fbae8134eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8134eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009840 tx=0x7fbad8009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 -- 172.21.15.179:0/2386506391 shutdown_connections 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8134eb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/2386506391 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae80a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 -- 172.21.15.179:0/2386506391 >> 172.21.15.179:0/2386506391 conn(0x7fbae80a3f20 msgr2=0x7fbae806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 -- 172.21.15.179:0/2386506391 shutdown_connections 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 1 -- 172.21.15.179:0/2386506391 wait complete. 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 10 monclient: build_initial_monmap 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.883+0000 7fbaee2d9700 10 monclient: monmap: 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 0.000000 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:created 0.000000 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 1 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:34.897 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 librados: starting msgr at 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 librados: starting objecter 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 Processor -- start 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 -- start start 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 librados: setting wanted keys 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 librados: calling monclient init 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient: init 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient: _add_conns ranks=[7,5,1,3,0,2,6,8,4] 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae81cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): picked mon.noname-i con 0x7fbae8072070 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbae80a2d40 0x7fbae81339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): picked mon.noname-h con 0x7fbae80a2d40 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): picked mon.noname-f con 0x7fbae8134ad0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.898 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae813cb20 con 0x7fbae8134ad0 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbae81307f0 con 0x7fbae80a2d40 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbae813ad80 con 0x7fbae8072070 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaee2d9700 10 monclient(hunting): authenticate will time out at 1598.291841s 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8133ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38482/0 (socket says 172.21.15.179:38482) 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 1 -- 172.21.15.179:0/339261861 learned_addr learned my addr 172.21.15.179:0/339261861 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae81cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 10 monclient(hunting): get_auth_request con 0x7fbae8134ad0 auth_method 0 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.884+0000 7fbaec876700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.899 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): get_auth_request con 0x7fbae8072070 auth_method 0 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbae80a2d40 0x7fbae81339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): get_auth_request con 0x7fbae80a2d40 auth_method 0 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 10 monclient(hunting): handle_auth_done global_id 4141 payload 824 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 -- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 msgr2=0x7fbae81cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae81cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 -- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbae80a2d40 msgr2=0x7fbae81339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbae80a2d40 0x7fbae81339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 10 monclient: _finish_hunting 0 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 monclient: found mon.noname-f 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 -- 172.21.15.179:0/339261861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80094a0 con 0x7fbae8134ad0 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae7fff700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae81cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaec876700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8133ef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbae000cbc0 tx=0x7fbae000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 client.?.objecter ms_handle_connect 0x7fbae8134ad0 2024-04-20T02:15:34.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbae000ac50 con 0x7fbae8134ad0 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: dump: 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.902 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 5 monclient: authenticate success, global_id 4141 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae000adb0 con 0x7fbae8134ad0 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 10 monclient: _renew_subs 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae809f350 con 0x7fbae8134ad0 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbae0011810 con 0x7fbae8134ad0 2024-04-20T02:15:34.903 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae67fc700 4 set_mon_vals no callback set 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 10 client.4141.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 monclient: dump: 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:34.904 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 10 monclient: _renew_subs 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae813c840 con 0x7fbae8134ad0 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbaee2d9700 1 librados: init done 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fbae00113a0 con 0x7fbae8134ad0 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbae0011db0 con 0x7fbae8134ad0 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 10 client.4141.objecter ms_dispatch 0x7fbae8001550 osd_map(2..2 src has 1..2) v4 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 3 client.4141.objecter handle_osd_map got epochs [2,2] > 0 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.885+0000 7fbae4ff9700 3 client.4141.objecter handle_osd_map decoding full epoch 2 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.886+0000 7fbae4ff9700 20 client.4141.objecter dump_active .. 0 homeless 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.886+0000 7fbaee2d9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:34.905 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.886+0000 7fbaee2d9700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:34.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.886+0000 7fbaee2d9700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:34.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.886+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae806aac0 con 0x7fbae8134ad0 2024-04-20T02:15:34.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.888+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fbae0017070 con 0x7fbae8134ad0 2024-04-20T02:15:34.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.888+0000 7fbae4ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:34.906 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.888+0000 7fbae4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:34.940 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:34.941+0000 7f86c2c98200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T02:15:34.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.973+0000 7fbaee2d9700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a1c1b335-9be6-43d2-94b3-8cd5fb042a10", "id": 0}] 2024-04-20T02:15:34.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.973+0000 7fbaee2d9700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a1c1b335-9be6-43d2-94b3-8cd5fb042a10", "id": 0}] 2024-04-20T02:15:34.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.973+0000 7fbaee2d9700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:34.973 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.973+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a1c1b335-9be6-43d2-94b3-8cd5fb042a10", "id": 0} v 0) v1 -- 0x7fbae806ace0 con 0x7fbae8134ad0 2024-04-20T02:15:34.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.980+0000 7fbae4ff9700 1 -- 172.21.15.179:0/339261861 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1c1b335-9be6-43d2-94b3-8cd5fb042a10", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbae000a7e0 con 0x7fbae8134ad0 2024-04-20T02:15:34.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.980+0000 7fbae4ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a1c1b335-9be6-43d2-94b3-8cd5fb042a10", "id": 0}] 2024-04-20T02:15:34.979 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.980+0000 7fbae4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:34.979 INFO:teuthology.orchestra.run.smithi179.stdout:0 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 10 librados: watch_flush enter 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 10 librados: watch_flush exit 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 20 client.4141.objecter shutdown clearing up homeless session... 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 10 client.4141.objecter successfully canceled tick 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 10 monclient: shutdown 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 msgr2=0x7fbae8133ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8133ef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbae000cbc0 tx=0x7fbae000ced0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 shutdown_connections 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbae8134ad0 0x7fbae8133ef0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.981 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbae8072070 0x7fbae81cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 --2- 172.21.15.179:0/339261861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbae80a2d40 0x7fbae81339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:34.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 >> 172.21.15.179:0/339261861 conn(0x7fbae80a3f20 msgr2=0x7fbae8070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:34.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 shutdown_connections 2024-04-20T02:15:34.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 -- 172.21.15.179:0/339261861 wait complete. 2024-04-20T02:15:34.982 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:34.981+0000 7fbaee2d9700 1 librados: shutdown 2024-04-20T02:15:34.992 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new ced35bb3-1a5e-416a-9d1d-c68163082c91 1 2024-04-20T02:15:35.124 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.125+0000 7f86c2c98200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T02:15:35.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.154+0000 7f7af7bc8700 10 monclient: get_monmap_and_config 2024-04-20T02:15:35.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.154+0000 7f7af7bc8700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.154+0000 7f7af7bc8700 10 monclient: monmap: 2024-04-20T02:15:35.153 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:35.155344+0000 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:35.155344+0000 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.154 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.155 INFO:teuthology.orchestra.run.smithi179.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:35.155 INFO:teuthology.orchestra.run.smithi179.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:35.155 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 Processor -- start 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 -- start start 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient: init 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient: _add_conns ranks=[3,8,5,2,4,9,7,6,10,1,0] 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 0x7f7af00a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): picked mon.noname-g con 0x7f7af00a27f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7af00a2bd0 0x7f7af009c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): picked mon.noname-e con 0x7f7af00a2bd0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 0x7f7af009cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): picked mon.noname-h con 0x7f7af009c8a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:35.165 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0066c80 con 0x7f7af00a27f0 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7af009b980 con 0x7f7af009c8a0 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7af009d260 con 0x7f7af00a2bd0 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.155+0000 7f7af7bc8700 10 monclient(hunting): authenticate will time out at 1598.562913s 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 0x7f7af009cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 0x7f7af00a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 0x7f7af00a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:39974/0 (socket says 172.21.15.179:39974) 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 0x7f7af009cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.179:51810/0 (socket says 172.21.15.179:51810) 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 1 -- 172.21.15.179:0/178205614 learned_addr learned my addr 172.21.15.179:0/178205614 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): get_auth_request con 0x7f7af00a27f0 auth_method 0 2024-04-20T02:15:35.166 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): get_auth_request con 0x7f7af009c8a0 auth_method 0 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5163700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7af00a2bd0 0x7f7af009c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5163700 10 monclient(hunting): get_auth_request con 0x7f7af00a2bd0 auth_method 0 2024-04-20T02:15:35.167 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5163700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5163700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5163700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient(hunting): handle_auth_done global_id 4118 payload 274 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 -- 172.21.15.179:0/178205614 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7af00a2bd0 msgr2=0x7f7af009c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7af00a2bd0 0x7f7af009c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 -- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 msgr2=0x7f7af00a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 0x7f7af00a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 monclient: found mon.noname-h 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 -- 172.21.15.179:0/178205614 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af00751e0 con 0x7f7af009c8a0 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af5964700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 0x7f7af00a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:35.168 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af6165700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 0x7f7af009cd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009dc0 tx=0x7f7ae800b330 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b826e16ca09aba98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 1 -- 172.21.15.179:0/178205614 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ae800e070 con 0x7f7af009c8a0 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: dump: 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.169 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 1 -- 172.21.15.179:0/178205614 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae80098f0 con 0x7f7af009c8a0 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af7bc8700 5 monclient: authenticate success, global_id 4118 2024-04-20T02:15:35.170 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af7bc8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 1 -- 172.21.15.179:0/178205614 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ae80128a0 con 0x7f7af009c8a0 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af7bc8700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af7bc8700 4 set_mon_vals no callback set 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.156+0000 7f7af4962700 10 monclient: dump: 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.171 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 10 monclient: shutdown 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 -- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 msgr2=0x7f7af009cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 0x7f7af009cd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009dc0 tx=0x7f7ae800b330 comp rx=0 tx=0).stop 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 -- 172.21.15.179:0/178205614 shutdown_connections 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7af009c8a0 0x7f7af009cd20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af00a27f0 0x7f7af00a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.172 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/178205614 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7af00a2bd0 0x7f7af009c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 -- 172.21.15.179:0/178205614 >> 172.21.15.179:0/178205614 conn(0x7f7af00a2fe0 msgr2=0x7f7af00a33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 -- 172.21.15.179:0/178205614 shutdown_connections 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.157+0000 7f7af7bc8700 1 -- 172.21.15.179:0/178205614 wait complete. 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient: monmap: 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 0.000000 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:created 0.000000 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 1 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.173 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 librados: starting msgr at 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 librados: starting objecter 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 Processor -- start 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 -- start start 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 librados: setting wanted keys 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 librados: calling monclient init 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient: init 2024-04-20T02:15:35.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient: _add_conns ranks=[3,7,2,6,4,0,1,5,8] 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 0x7f7af01cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): picked mon.noname-g con 0x7f7af009c8a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7af00a27f0 0x7f7af0098c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): picked mon.noname-i con 0x7f7af00a27f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7af00a2bd0 0x7f7af01c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): picked mon.noname-b con 0x7f7af00a2bd0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00729f0 con 0x7f7af00a2bd0 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01cd3a0 con 0x7f7af009c8a0 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7af009ace0 con 0x7f7af00a27f0 2024-04-20T02:15:35.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af7bc8700 10 monclient(hunting): authenticate will time out at 1598.565855s 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.158+0000 7f7af5964700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 0x7f7af01cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7af00a27f0 0x7f7af0098c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 0x7f7af01cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:39982/0 (socket says 172.21.15.179:39982) 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 -- 172.21.15.179:0/1300101877 learned_addr learned my addr 172.21.15.179:0/1300101877 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): get_auth_request con 0x7f7af009c8a0 auth_method 0 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): get_auth_request con 0x7f7af00a27f0 auth_method 0 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5163700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af6165700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7af00a2bd0 0x7f7af01c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af6165700 10 monclient(hunting): get_auth_request con 0x7f7af00a2bd0 auth_method 0 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af6165700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af6165700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af6165700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient(hunting): handle_auth_done global_id 4125 payload 824 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 -- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7af00a27f0 msgr2=0x7f7af0098c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7af00a27f0 0x7f7af0098c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 -- 172.21.15.179:0/1300101877 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7af00a2bd0 msgr2=0x7f7af01c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7af00a2bd0 0x7f7af01c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 monclient: found mon.noname-g 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 -- 172.21.15.179:0/1300101877 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009550 con 0x7f7af009c8a0 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af6165700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7af00a2bd0 0x7f7af01c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af5964700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 0x7f7af01cc450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00ec10 tx=0x7f7aec00ef20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 client.?.objecter ms_handle_connect 0x7f7af009c8a0 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7aec00ccc0 con 0x7f7af009c8a0 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: dump: 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.178 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 5 monclient: authenticate success, global_id 4125 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec00ce20 con 0x7f7af009c8a0 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7aec0138e0 con 0x7f7af009c8a0 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae7fff700 4 set_mon_vals no callback set 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7ae67fc700 10 monclient: dump: 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.180 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 10 monclient: _renew_subs 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae8009a70 con 0x7f7af009c8a0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 10 client.4125.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 10 monclient: _renew_subs 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af0091de0 con 0x7f7af009c8a0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.159+0000 7f7af7bc8700 1 librados: init done 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7aec00c850 con 0x7f7af009c8a0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f7aec0043d0 con 0x7f7af009c8a0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7ae67fc700 10 client.4125.objecter ms_dispatch 0x7f7af0001550 osd_map(3..3 src has 1..3) v4 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7ae67fc700 3 client.4125.objecter handle_osd_map got epochs [3,3] > 0 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7ae67fc700 3 client.4125.objecter handle_osd_map decoding full epoch 3 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7ae67fc700 20 client.4125.objecter dump_active .. 0 homeless 2024-04-20T02:15:35.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7af7bc8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7af7bc8700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7af7bc8700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:35.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.160+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af006aac0 con 0x7f7af009c8a0 2024-04-20T02:15:35.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.164+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f7aec017070 con 0x7f7af009c8a0 2024-04-20T02:15:35.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.164+0000 7f7ae67fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.183 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.164+0000 7f7ae67fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:35.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.250+0000 7f7af7bc8700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ced35bb3-1a5e-416a-9d1d-c68163082c91", "id": 1}] 2024-04-20T02:15:35.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.250+0000 7f7af7bc8700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ced35bb3-1a5e-416a-9d1d-c68163082c91", "id": 1}] 2024-04-20T02:15:35.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.250+0000 7f7af7bc8700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:35.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.250+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ced35bb3-1a5e-416a-9d1d-c68163082c91", "id": 1} v 0) v1 -- 0x7f7af006ace0 con 0x7f7af009c8a0 2024-04-20T02:15:35.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.255+0000 7f7ae67fc700 1 -- 172.21.15.179:0/1300101877 <== mon.3 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ced35bb3-1a5e-416a-9d1d-c68163082c91", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7aec013a40 con 0x7f7af009c8a0 2024-04-20T02:15:35.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.255+0000 7f7ae67fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ced35bb3-1a5e-416a-9d1d-c68163082c91", "id": 1}] 2024-04-20T02:15:35.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.255+0000 7f7ae67fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:35.254 INFO:teuthology.orchestra.run.smithi179.stdout:1 2024-04-20T02:15:35.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 10 librados: watch_flush enter 2024-04-20T02:15:35.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 10 librados: watch_flush exit 2024-04-20T02:15:35.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 20 client.4125.objecter shutdown clearing up homeless session... 2024-04-20T02:15:35.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 10 client.4125.objecter successfully canceled tick 2024-04-20T02:15:35.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 10 monclient: shutdown 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 msgr2=0x7f7af01cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 0x7f7af01cc450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00ec10 tx=0x7f7aec00ef20 comp rx=0 tx=0).stop 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 shutdown_connections 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7af00a2bd0 0x7f7af01c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7af009c8a0 0x7f7af01cc450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 --2- 172.21.15.179:0/1300101877 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7af00a27f0 0x7f7af0098c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 >> 172.21.15.179:0/1300101877 conn(0x7f7af00a2fe0 msgr2=0x7f7af01bf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 shutdown_connections 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 -- 172.21.15.179:0/1300101877 wait complete. 2024-04-20T02:15:35.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.256+0000 7f7af7bc8700 1 librados: shutdown 2024-04-20T02:15:35.267 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new c932cb96-511e-4863-8792-aa8c0bbb0e84 2 2024-04-20T02:15:35.307 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.308+0000 7f86c2c98200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T02:15:35.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.393+0000 7f00828f7700 10 monclient: get_monmap_and_config 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.393+0000 7f00828f7700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.393+0000 7f00828f7700 10 monclient: monmap: 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:35.393746+0000 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:35.393746+0000 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.392 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:35.393 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.393+0000 7f00828f7700 1 Processor -- start 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.393+0000 7f00828f7700 1 -- start start 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.393+0000 7f00828f7700 10 monclient: init 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient: _add_conns ranks=[3,4,10,5,9,6,0,8,2,1,7] 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 0x7f007c137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): picked mon.noname-g con 0x7f007c137ab0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c09d9f0 0x7f007c1383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): picked mon.noname-c con 0x7f007c09d9f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f007c136a30 0x7f007c136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): picked mon.noname-f con 0x7f007c136a30 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:35.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c09e6c0 con 0x7f007c136a30 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c09fba0 con 0x7f007c137ab0 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f007c138910 con 0x7f007c09d9f0 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f00828f7700 10 monclient(hunting): authenticate will time out at 1598.801289s 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 0x7f007c137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f007c136a30 0x7f007c136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 0x7f007c137e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:39996/0 (socket says 172.21.15.179:39996) 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 -- 172.21.15.179:0/2891248545 learned_addr learned my addr 172.21.15.179:0/2891248545 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): get_auth_request con 0x7f007c136a30 auth_method 0 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): get_auth_request con 0x7f007c137ab0 auth_method 0 2024-04-20T02:15:35.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007b7fe700 1 --2- 172.21.15.179:0/2891248545 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c09d9f0 0x7f007c1383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007b7fe700 10 monclient(hunting): get_auth_request con 0x7f007c09d9f0 auth_method 0 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient(hunting): handle_auth_done global_id 4134 payload 274 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 -- 172.21.15.179:0/2891248545 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c09d9f0 msgr2=0x7f007c1383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.405 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 --2- 172.21.15.179:0/2891248545 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c09d9f0 0x7f007c1383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 -- 172.21.15.179:0/2891248545 >> v2:172.21.15.179:3300/0 conn(0x7f007c136a30 msgr2=0x7f007c136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 --2- 172.21.15.179:0/2891248545 >> v2:172.21.15.179:3300/0 conn(0x7f007c136a30 0x7f007c136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 monclient: found mon.noname-g 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 -- 172.21.15.179:0/2891248545 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c0751e0 con 0x7f007c137ab0 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f0080e94700 1 --2- 172.21.15.179:0/2891248545 >> v2:172.21.15.179:3300/0 conn(0x7f007c136a30 0x7f007c136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.394+0000 7f007bfff700 1 --2- 172.21.15.179:0/2891248545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 0x7f007c137e90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f006c0098f0 tx=0x7f006c009c00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=e3defe8ffabd0180 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 1 -- 172.21.15.179:0/2891248545 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f006c011aa0 con 0x7f007c137ab0 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: dump: 2024-04-20T02:15:35.406 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 5 monclient: authenticate success, global_id 4134 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 1 -- 172.21.15.179:0/2891248545 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c011c00 con 0x7f007c137ab0 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 1 -- 172.21.15.179:0/2891248545 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f006c011d70 con 0x7f007c137ab0 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 4 set_mon_vals no callback set 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f007affd700 10 monclient: dump: 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.408 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 10 monclient: shutdown 2024-04-20T02:15:35.409 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 -- 172.21.15.179:0/2891248545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 msgr2=0x7f007c137e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 --2- 172.21.15.179:0/2891248545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 0x7f007c137e90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f006c0098f0 tx=0x7f006c009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 -- 172.21.15.179:0/2891248545 shutdown_connections 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 --2- 172.21.15.179:0/2891248545 >> v2:172.21.15.179:3300/0 conn(0x7f007c136a30 0x7f007c136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 --2- 172.21.15.179:0/2891248545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f007c137ab0 0x7f007c137e90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 --2- 172.21.15.179:0/2891248545 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c09d9f0 0x7f007c1383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 -- 172.21.15.179:0/2891248545 >> 172.21.15.179:0/2891248545 conn(0x7f007c0aa280 msgr2=0x7f007c0aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 -- 172.21.15.179:0/2891248545 shutdown_connections 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.395+0000 7f00828f7700 1 -- 172.21.15.179:0/2891248545 wait complete. 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.396+0000 7f00828f7700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.396+0000 7f00828f7700 10 monclient: monmap: 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 0.000000 2024-04-20T02:15:35.410 INFO:teuthology.orchestra.run.smithi179.stderr:created 0.000000 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 1 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.396+0000 7f00828f7700 1 librados: starting msgr at 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.396+0000 7f00828f7700 1 librados: starting objecter 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.396+0000 7f00828f7700 1 Processor -- start 2024-04-20T02:15:35.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 -- start start 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 librados: setting wanted keys 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 librados: calling monclient init 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient: init 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient: _add_conns ranks=[2,6,0,5,3,8,4,7,1] 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 0x7f007c0a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): picked mon.noname-b con 0x7f007c09d9f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 0x7f007c1208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): picked mon.noname-d con 0x7f007c136a30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c137ab0 0x7f007c120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): picked mon.noname-a con 0x7f007c137ab0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c136770 con 0x7f007c137ab0 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c123ba0 con 0x7f007c09d9f0 2024-04-20T02:15:35.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f007c09ecf0 con 0x7f007c136a30 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f00828f7700 10 monclient(hunting): authenticate will time out at 1598.804589s 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 0x7f007c0a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 0x7f007c1208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f0080e94700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c137ab0 0x7f007c120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 0x7f007c0a3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.179:58002/0 (socket says 172.21.15.179:58002) 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 0x7f007c1208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.179:52814/0 (socket says 172.21.15.179:52814) 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 1 -- 172.21.15.179:0/265923030 learned_addr learned my addr 172.21.15.179:0/265923030 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 10 monclient(hunting): get_auth_request con 0x7f007c09d9f0 auth_method 0 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f007bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f0080e94700 10 monclient(hunting): get_auth_request con 0x7f007c137ab0 auth_method 0 2024-04-20T02:15:35.413 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f0080e94700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f0080e94700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.397+0000 7f0080e94700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): get_auth_request con 0x7f007c136a30 auth_method 0 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0080e94700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0080e94700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.414 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0080e94700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient(hunting): handle_auth_done global_id 4110 payload 824 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 -- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 msgr2=0x7f007c0a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 0x7f007c0a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 -- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c137ab0 msgr2=0x7f007c120e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c137ab0 0x7f007c120e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 monclient: found mon.noname-d 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 -- 172.21.15.179:0/265923030 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074009a70 con 0x7f007c136a30 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007bfff700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 0x7f007c0a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0080e94700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c137ab0 0x7f007c120e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007b7fe700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 0x7f007c1208d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f007c0a6660 tx=0x7f007000e8b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 client.?.objecter ms_handle_connect 0x7f007c136a30 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:35.415 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00700044a0 con 0x7f007c136a30 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: dump: 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.416 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 5 monclient: authenticate success, global_id 4110 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 10 monclient: _renew_subs 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00700036a0 con 0x7f007c136a30 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c009550 con 0x7f007c136a30 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0070003c80 con 0x7f007c136a30 2024-04-20T02:15:35.417 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 10 client.4110.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f007a7fc700 4 set_mon_vals no callback set 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f0078ff9700 10 monclient: dump: 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.418 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 10 monclient: _renew_subs 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c123300 con 0x7f007c136a30 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.398+0000 7f00828f7700 1 librados: init done 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0070004030 con 0x7f007c136a30 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f00700042e0 con 0x7f007c136a30 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f0078ff9700 10 client.4110.objecter ms_dispatch 0x7f007c001550 osd_map(4..4 src has 1..4) v4 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f0078ff9700 3 client.4110.objecter handle_osd_map got epochs [4,4] > 0 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f0078ff9700 3 client.4110.objecter handle_osd_map decoding full epoch 4 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f0078ff9700 20 client.4110.objecter dump_active .. 0 homeless 2024-04-20T02:15:35.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f00828f7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f00828f7700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f00828f7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:35.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.399+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c067f90 con 0x7f007c136a30 2024-04-20T02:15:35.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.402+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f007000c070 con 0x7f007c136a30 2024-04-20T02:15:35.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.402+0000 7f0078ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.402+0000 7f0078ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:35.425 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.426+0000 7f86c2c98200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T02:15:35.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.487+0000 7f00828f7700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c932cb96-511e-4863-8792-aa8c0bbb0e84", "id": 2}] 2024-04-20T02:15:35.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.487+0000 7f00828f7700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c932cb96-511e-4863-8792-aa8c0bbb0e84", "id": 2}] 2024-04-20T02:15:35.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.487+0000 7f00828f7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:35.486 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.487+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "osd new", "uuid": "c932cb96-511e-4863-8792-aa8c0bbb0e84", "id": 2} v 0) v1 -- 0x7f007c09ea70 con 0x7f007c136a30 2024-04-20T02:15:35.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.491+0000 7f0078ff9700 1 -- 172.21.15.179:0/265923030 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c932cb96-511e-4863-8792-aa8c0bbb0e84", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f007000f040 con 0x7f007c136a30 2024-04-20T02:15:35.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.491+0000 7f0078ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c932cb96-511e-4863-8792-aa8c0bbb0e84", "id": 2}] 2024-04-20T02:15:35.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.491+0000 7f0078ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:35.491 INFO:teuthology.orchestra.run.smithi179.stdout:2 2024-04-20T02:15:35.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 10 librados: watch_flush enter 2024-04-20T02:15:35.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 10 librados: watch_flush exit 2024-04-20T02:15:35.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 20 client.4110.objecter shutdown clearing up homeless session... 2024-04-20T02:15:35.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 10 client.4110.objecter successfully canceled tick 2024-04-20T02:15:35.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 10 monclient: shutdown 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 msgr2=0x7f007c1208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.492+0000 7f00828f7700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 0x7f007c1208d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f007c0a6660 tx=0x7f007000e8b0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 shutdown_connections 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c137ab0 0x7f007c120e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c09d9f0 0x7f007c0a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 --2- 172.21.15.179:0/265923030 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f007c136a30 0x7f007c1208d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 >> 172.21.15.179:0/265923030 conn(0x7f007c0aa280 msgr2=0x7f007c1bf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 shutdown_connections 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 -- 172.21.15.179:0/265923030 wait complete. 2024-04-20T02:15:35.493 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.493+0000 7f00828f7700 1 librados: shutdown 2024-04-20T02:15:35.503 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 60f24ee2-e337-4b56-a45d-98a0cb6188bf 3 2024-04-20T02:15:35.524 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.525+0000 7f86c2c98200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T02:15:35.607 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.608+0000 7f86c2c98200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.648+0000 7f5a2242a700 10 monclient: get_monmap_and_config 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.648+0000 7f5a2242a700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.648+0000 7f5a2242a700 10 monclient: monmap: 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:35.649124+0000 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:35.649124+0000 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.647 INFO:teuthology.orchestra.run.smithi179.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:35.648 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 Processor -- start 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 -- start start 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient: init 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient: _add_conns ranks=[8,5,9,3,0,1,6,4,10,7,2] 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a1c0a0a90 0x7f5a1c0a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): picked mon.noname-e con 0x7f5a1c0a0a90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 0x7f5a1c0a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): picked mon.noname-h con 0x7f5a1c0a00c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a1c0a2d40 0x7f5a1c0a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): picked mon.noname-a con 0x7f5a1c0a2d40 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:35.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c09f4e0 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c09de20 con 0x7f5a1c0a00c0 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c09ee40 con 0x7f5a1c0a0a90 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a2242a700 10 monclient(hunting): authenticate will time out at 1599.056708s 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 0x7f5a1c0a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 0x7f5a1c0a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.179:51824/0 (socket says 172.21.15.179:51824) 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 1 -- 172.21.15.179:0/3524415845 learned_addr learned my addr 172.21.15.179:0/3524415845 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a00c0 auth_method 0 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a209c7700 1 --2- 172.21.15.179:0/3524415845 >> v2:172.21.15.47:3300/0 conn(0x7f5a1c0a2d40 0x7f5a1c0a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.649+0000 7f5a1bfff700 1 --2- 172.21.15.179:0/3524415845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a1c0a0a90 0x7f5a1c0a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a209c7700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a2d40 auth_method 0 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a209c7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a209c7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a209c7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a0a90 auth_method 0 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_done global_id 4127 payload 274 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 -- 172.21.15.179:0/3524415845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a1c0a0a90 msgr2=0x7f5a1c0a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 --2- 172.21.15.179:0/3524415845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a1c0a0a90 0x7f5a1c0a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.660 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 -- 172.21.15.179:0/3524415845 >> v2:172.21.15.47:3300/0 conn(0x7f5a1c0a2d40 msgr2=0x7f5a1c0a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 --2- 172.21.15.179:0/3524415845 >> v2:172.21.15.47:3300/0 conn(0x7f5a1c0a2d40 0x7f5a1c0a31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 monclient: found mon.noname-h 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 -- 172.21.15.179:0/3524415845 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0751e0 con 0x7f5a1c0a00c0 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a209c7700 1 --2- 172.21.15.179:0/3524415845 >> v2:172.21.15.47:3300/0 conn(0x7f5a1c0a2d40 0x7f5a1c0a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1b7fe700 1 --2- 172.21.15.179:0/3524415845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 0x7f5a1c0a2800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009920 tx=0x7f5a10009c30 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d36d7cc62482cc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 1 -- 172.21.15.179:0/3524415845 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a10009e70 con 0x7f5a1c0a00c0 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: dump: 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 5 monclient: authenticate success, global_id 4127 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 1 -- 172.21.15.179:0/3524415845 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1000bc90 con 0x7f5a1c0a00c0 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 4 set_mon_vals no callback set 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 1 -- 172.21.15.179:0/3524415845 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a1000be00 con 0x7f5a1c0a00c0 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a1affd700 10 monclient: dump: 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.663 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.664 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 10 monclient: shutdown 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 1 -- 172.21.15.179:0/3524415845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 msgr2=0x7f5a1c0a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.650+0000 7f5a2242a700 1 --2- 172.21.15.179:0/3524415845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 0x7f5a1c0a2800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009920 tx=0x7f5a10009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 -- 172.21.15.179:0/3524415845 shutdown_connections 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 --2- 172.21.15.179:0/3524415845 >> v2:172.21.15.47:3300/0 conn(0x7f5a1c0a2d40 0x7f5a1c0a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 --2- 172.21.15.179:0/3524415845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a1c0a0a90 0x7f5a1c0a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 --2- 172.21.15.179:0/3524415845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a00c0 0x7f5a1c0a2800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 -- 172.21.15.179:0/3524415845 >> 172.21.15.179:0/3524415845 conn(0x7f5a1c0a3f20 msgr2=0x7f5a1c06c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 -- 172.21.15.179:0/3524415845 shutdown_connections 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 -- 172.21.15.179:0/3524415845 wait complete. 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 10 monclient: monmap: 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 0.000000 2024-04-20T02:15:35.665 INFO:teuthology.orchestra.run.smithi179.stderr:created 0.000000 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 1 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 librados: starting msgr at 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 librados: starting objecter 2024-04-20T02:15:35.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.651+0000 7f5a2242a700 1 Processor -- start 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 -- start start 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 librados: setting wanted keys 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 librados: calling monclient init 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient: init 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient: _add_conns ranks=[7,6,5,3,0,4,8,2,1] 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a1c0a00c0 0x7f5a1c1cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): picked mon.noname-i con 0x7f5a1c0a00c0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a1c0a0a90 0x7f5a1c1bc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): picked mon.noname-d con 0x7f5a1c0a0a90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 0x7f5a1c1bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): picked mon.noname-h con 0x7f5a1c0a2d40 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c1a8970 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.667 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c071f70 con 0x7f5a1c0a0a90 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c09e510 con 0x7f5a1c0a00c0 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a2242a700 10 monclient(hunting): authenticate will time out at 1599.059659s 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 0x7f5a1c1bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 0x7f5a1c1bcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.179:51832/0 (socket says 172.21.15.179:51832) 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 1 -- 172.21.15.179:0/1667360975 learned_addr learned my addr 172.21.15.179:0/1667360975 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a2d40 auth_method 0 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a1c0a00c0 0x7f5a1c1cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a00c0 auth_method 0 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.668 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a1c0a0a90 0x7f5a1c1bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a209c7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a0a90 auth_method 0 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.669 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.652+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 10 monclient(hunting): handle_auth_done global_id 4136 payload 824 2024-04-20T02:15:35.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 -- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a1c0a00c0 msgr2=0x7f5a1c1cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a1c0a00c0 0x7f5a1c1cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.670 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 -- 172.21.15.179:0/1667360975 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a1c0a0a90 msgr2=0x7f5a1c1bc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a1c0a0a90 0x7f5a1c1bc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 monclient: found mon.noname-h 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 -- 172.21.15.179:0/1667360975 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10009550 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a1bfff700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a1c0a00c0 0x7f5a1c1cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a209c7700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 0x7f5a1c1bcec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5a14007e70 tx=0x7f5a1400d3b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 client.?.objecter ms_handle_connect 0x7f5a1c0a2d40 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a14003a20 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: dump: 2024-04-20T02:15:35.671 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.672 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 5 monclient: authenticate success, global_id 4136 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1400f040 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a14003bc0 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 monclient: dump: 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.673 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a1a7fc700 4 set_mon_vals no callback set 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 10 monclient: _renew_subs 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c1aa340 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.674 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 10 client.4136.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 10 monclient: _renew_subs 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c1a7f90 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a2242a700 1 librados: init done 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5a14003d20 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5a140048a0 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 10 client.4136.objecter ms_dispatch 0x7f5a1c001550 osd_map(5..5 src has 1..5) v4 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 3 client.4136.objecter handle_osd_map got epochs [5,5] > 0 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 3 client.4136.objecter handle_osd_map decoding full epoch 5 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.653+0000 7f5a18ff9700 20 client.4136.objecter dump_active .. 0 homeless 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.654+0000 7f5a2242a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.654+0000 7f5a2242a700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.654+0000 7f5a2242a700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.654+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c067cd0 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.656+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f5a1c067cd0 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.656+0000 7f5a18ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.656+0000 7f5a18ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:35.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.742+0000 7f5a2242a700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "60f24ee2-e337-4b56-a45d-98a0cb6188bf", "id": 3}] 2024-04-20T02:15:35.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.742+0000 7f5a2242a700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "60f24ee2-e337-4b56-a45d-98a0cb6188bf", "id": 3}] 2024-04-20T02:15:35.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.742+0000 7f5a2242a700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.742+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_command({"prefix": "osd new", "uuid": "60f24ee2-e337-4b56-a45d-98a0cb6188bf", "id": 3} v 0) v1 -- 0x7f5a1c06aac0 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a18ff9700 1 -- 172.21.15.179:0/1667360975 <== mon.5 v2:172.21.15.179:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "60f24ee2-e337-4b56-a45d-98a0cb6188bf", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a1401a070 con 0x7f5a1c0a2d40 2024-04-20T02:15:35.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a18ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "60f24ee2-e337-4b56-a45d-98a0cb6188bf", "id": 3}] 2024-04-20T02:15:35.745 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a18ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:35.746 INFO:teuthology.orchestra.run.smithi179.stdout:3 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 10 librados: watch_flush enter 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 10 librados: watch_flush exit 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 20 client.4136.objecter shutdown clearing up homeless session... 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 10 client.4136.objecter successfully canceled tick 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 10 monclient: shutdown 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.747+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 msgr2=0x7f5a1c1bcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 0x7f5a1c1bcec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5a14007e70 tx=0x7f5a1400d3b0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 shutdown_connections 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a1c0a00c0 0x7f5a1c1cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a1c0a2d40 0x7f5a1c1bcec0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 --2- 172.21.15.179:0/1667360975 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a1c0a0a90 0x7f5a1c1bc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 >> 172.21.15.179:0/1667360975 conn(0x7f5a1c0a3f20 msgr2=0x7f5a1c0a25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 shutdown_connections 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 -- 172.21.15.179:0/1667360975 wait complete. 2024-04-20T02:15:35.748 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.748+0000 7f5a2242a700 1 librados: shutdown 2024-04-20T02:15:35.758 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 78fd6b14-e8e8-420e-bb89-17b57d0b613e 4 2024-04-20T02:15:35.848 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.849+0000 7f86c2c98200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.902+0000 7f3cedb7d700 10 monclient: get_monmap_and_config 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.902+0000 7f3cedb7d700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.902+0000 7f3cedb7d700 10 monclient: monmap: 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:35.902941+0000 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:35.902941+0000 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.901 INFO:teuthology.orchestra.run.smithi179.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:35.902 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.912 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 Processor -- start 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 -- start start 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient: init 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient: _add_conns ranks=[5,9,8,4,1,10,2,3,7,6,0] 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 0x7f3ce809c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): picked mon.noname-h con 0x7f3ce809c500 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3ce809ce20 0x7f3ce80a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): picked mon.noname-a con 0x7f3ce809ce20 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3ce8134cb0 0x7f3ce80a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): picked mon.noname-e con 0x7f3ce8134cb0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce80a0300 con 0x7f3ce809ce20 2024-04-20T02:15:35.913 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce809efc0 con 0x7f3ce809c500 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce809fcb0 con 0x7f3ce8134cb0 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3cedb7d700 10 monclient(hunting): authenticate will time out at 1599.310726s 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 0x7f3ce809c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 0x7f3ce809c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.179:51838/0 (socket says 172.21.15.179:51838) 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 1 -- 172.21.15.179:0/206991622 learned_addr learned my addr 172.21.15.179:0/206991622 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): get_auth_request con 0x7f3ce809c500 auth_method 0 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.914 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.903+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce7fff700 1 --2- 172.21.15.179:0/206991622 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3ce8134cb0 0x7f3ce80a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cdffff700 1 --2- 172.21.15.179:0/206991622 >> v2:172.21.15.47:3300/0 conn(0x7f3ce809ce20 0x7f3ce80a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce7fff700 10 monclient(hunting): get_auth_request con 0x7f3ce8134cb0 auth_method 0 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cdffff700 10 monclient(hunting): get_auth_request con 0x7f3ce809ce20 auth_method 0 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cdffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cdffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cdffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_done global_id 4145 payload 274 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 -- 172.21.15.179:0/206991622 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3ce8134cb0 msgr2=0x7f3ce80a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 --2- 172.21.15.179:0/206991622 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3ce8134cb0 0x7f3ce80a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 -- 172.21.15.179:0/206991622 >> v2:172.21.15.47:3300/0 conn(0x7f3ce809ce20 msgr2=0x7f3ce80a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:35.915 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 --2- 172.21.15.179:0/206991622 >> v2:172.21.15.47:3300/0 conn(0x7f3ce809ce20 0x7f3ce80a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 monclient: found mon.noname-h 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 -- 172.21.15.179:0/206991622 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80751e0 con 0x7f3ce809c500 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce77fe700 1 --2- 172.21.15.179:0/206991622 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 0x7f3ce809c8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80098f0 tx=0x7f3cd8009c00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a9a68ba8439e11c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 1 -- 172.21.15.179:0/206991622 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3cd800bb10 con 0x7f3ce809c500 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: dump: 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.916 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cedb7d700 5 monclient: authenticate success, global_id 4145 2024-04-20T02:15:35.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 1 -- 172.21.15.179:0/206991622 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd800bc70 con 0x7f3ce809c500 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cedb7d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cedb7d700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 1 -- 172.21.15.179:0/206991622 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3cd800bde0 con 0x7f3ce809c500 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3cedb7d700 4 set_mon_vals no callback set 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.904+0000 7f3ce6ffd700 10 monclient: dump: 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.918 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 10 monclient: shutdown 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 -- 172.21.15.179:0/206991622 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 msgr2=0x7f3ce809c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/206991622 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 0x7f3ce809c8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80098f0 tx=0x7f3cd8009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:35.919 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 -- 172.21.15.179:0/206991622 shutdown_connections 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/206991622 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3ce8134cb0 0x7f3ce80a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/206991622 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ce809c500 0x7f3ce809c8e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/206991622 >> v2:172.21.15.47:3300/0 conn(0x7f3ce809ce20 0x7f3ce80a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 -- 172.21.15.179:0/206991622 >> 172.21.15.179:0/206991622 conn(0x7f3ce80a2fe0 msgr2=0x7f3ce80a33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 -- 172.21.15.179:0/206991622 shutdown_connections 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 -- 172.21.15.179:0/206991622 wait complete. 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 10 monclient: build_initial_monmap 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 10 monclient: monmap: 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 0.000000 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:created 0.000000 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 1 2024-04-20T02:15:35.920 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 librados: starting msgr at 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.905+0000 7f3cedb7d700 1 librados: starting objecter 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 Processor -- start 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 -- start start 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 librados: setting wanted keys 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 librados: calling monclient init 2024-04-20T02:15:35.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient: init 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient: _add_conns ranks=[4,2,0,1,6,3,8,5,7] 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3ce809c500 0x7f3ce81c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): picked mon.noname-c con 0x7f3ce809c500 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ce8134cb0 0x7f3ce813d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): picked mon.noname-b con 0x7f3ce8134cb0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 0x7f3ce81cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): picked mon.noname-a con 0x7f3ce813d8c0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce81c97a0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce8099fa0 con 0x7f3ce8134cb0 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80729c0 con 0x7f3ce809c500 2024-04-20T02:15:35.922 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cedb7d700 10 monclient(hunting): authenticate will time out at 1599.313606s 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 0x7f3ce81cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3ce809c500 0x7f3ce81c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ce8134cb0 0x7f3ce813d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 0x7f3ce81cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.179:36952/0 (socket says 172.21.15.179:36952) 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 1 -- 172.21.15.179:0/188198494 learned_addr learned my addr 172.21.15.179:0/188198494 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): get_auth_request con 0x7f3ce813d8c0 auth_method 0 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): get_auth_request con 0x7f3ce809c500 auth_method 0 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.923 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): get_auth_request con 0x7f3ce8134cb0 auth_method 0 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3ce77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.906+0000 7f3cdffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 10 monclient(hunting): handle_auth_done global_id 4140 payload 824 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 -- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3ce809c500 msgr2=0x7f3ce81c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3ce809c500 0x7f3ce81c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.924 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 -- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ce8134cb0 msgr2=0x7f3ce813d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ce8134cb0 0x7f3ce813d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 monclient: found mon.noname-a 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 -- 172.21.15.179:0/188198494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8009550 con 0x7f3ce813d8c0 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce7fff700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 0x7f3ce81cc240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0007e60 tx=0x7f3ce000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 client.?.objecter ms_handle_connect 0x7f3ce813d8c0 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ce0014420 con 0x7f3ce813d8c0 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: dump: 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.925 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.926 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 5 monclient: authenticate success, global_id 4140 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce00145c0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 10 monclient: _renew_subs 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce813bb70 con 0x7f3ce813d8c0 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ce0014ba0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce67fc700 4 set_mon_vals no callback set 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 10 client.4140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 monclient: dump: 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:35.927 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 10 monclient: _renew_subs 2024-04-20T02:15:35.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce8070ec0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3cedb7d700 1 librados: init done 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3ce0014d20 con 0x7f3ce813d8c0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3ce000eb30 con 0x7f3ce813d8c0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 10 client.4140.objecter ms_dispatch 0x7f3ce8001550 osd_map(6..6 src has 1..6) v4 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 3 client.4140.objecter handle_osd_map got epochs [6,6] > 0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 3 client.4140.objecter handle_osd_map decoding full epoch 6 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.907+0000 7f3ce4ff9700 20 client.4140.objecter dump_active .. 0 homeless 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.908+0000 7f3cedb7d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.908+0000 7f3cedb7d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.908+0000 7f3cedb7d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.908+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce806aac0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.911+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f3ce000edd0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.911+0000 7f3ce4ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:35.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.911+0000 7f3ce4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:35.930 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:35.925+0000 7f86c2c98200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T02:15:35.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.993+0000 7f3cedb7d700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "78fd6b14-e8e8-420e-bb89-17b57d0b613e", "id": 4}] 2024-04-20T02:15:35.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.993+0000 7f3cedb7d700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "78fd6b14-e8e8-420e-bb89-17b57d0b613e", "id": 4}] 2024-04-20T02:15:35.992 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.993+0000 7f3cedb7d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:35.993 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.993+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd new", "uuid": "78fd6b14-e8e8-420e-bb89-17b57d0b613e", "id": 4} v 0) v1 -- 0x7f3ce806ace0 con 0x7f3ce813d8c0 2024-04-20T02:15:35.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.997+0000 7f3ce4ff9700 1 -- 172.21.15.179:0/188198494 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "78fd6b14-e8e8-420e-bb89-17b57d0b613e", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ce001e820 con 0x7f3ce813d8c0 2024-04-20T02:15:35.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.997+0000 7f3ce4ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "78fd6b14-e8e8-420e-bb89-17b57d0b613e", "id": 4}] 2024-04-20T02:15:35.996 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.997+0000 7f3ce4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:35.996 INFO:teuthology.orchestra.run.smithi179.stdout:4 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 10 librados: watch_flush enter 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 10 librados: watch_flush exit 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 20 client.4140.objecter shutdown clearing up homeless session... 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 10 client.4140.objecter successfully canceled tick 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 10 monclient: shutdown 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 msgr2=0x7f3ce81cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 0x7f3ce81cc240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0007e60 tx=0x7f3ce000e3c0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 shutdown_connections 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ce813d8c0 0x7f3ce81cc240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3ce809c500 0x7f3ce81c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 --2- 172.21.15.179:0/188198494 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ce8134cb0 0x7f3ce813d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:35.998 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 >> 172.21.15.179:0/188198494 conn(0x7f3ce80a2fe0 msgr2=0x7f3ce80a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:35.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 shutdown_connections 2024-04-20T02:15:35.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 -- 172.21.15.179:0/188198494 wait complete. 2024-04-20T02:15:35.999 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:35.998+0000 7f3cedb7d700 1 librados: shutdown 2024-04-20T02:15:36.009 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new fad619aa-0c5c-4c2d-82b3-510ce37c7c4c 5 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.134+0000 7f91b875e700 10 monclient: get_monmap_and_config 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.134+0000 7f91b875e700 10 monclient: build_initial_monmap 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.134+0000 7f91b875e700 10 monclient: monmap: 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:36.134967+0000 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:36.134967+0000 2024-04-20T02:15:36.133 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.134+0000 7f91b875e700 1 Processor -- start 2024-04-20T02:15:36.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.134+0000 7f91b875e700 1 -- start start 2024-04-20T02:15:36.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.134+0000 7f91b875e700 10 monclient: init 2024-04-20T02:15:36.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:36.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient: _add_conns ranks=[10,4,1,0,5,2,3,6,9,8,7] 2024-04-20T02:15:36.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91b00993e0 0x7f91b00997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.144 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): picked mon.noname-f con 0x7f91b00993e0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91b0135f70 0x7f91b00a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): picked mon.noname-c con 0x7f91b0135f70 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f91b01363f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00751e0 con 0x7f91b01363f0 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b0066c20 con 0x7f91b00993e0 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91b009dbf0 con 0x7f91b0135f70 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b875e700 10 monclient(hunting): authenticate will time out at 1599.542568s 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91b00993e0 0x7f91b00997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b6cfb700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f91b01363f0 0x7f91b00a2430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:45404/0 (socket says 172.21.15.179:45404) 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f91b00993e0 0x7f91b00997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:38508/0 (socket says 172.21.15.179:38508) 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b6cfb700 1 -- 172.21.15.179:0/457669356 learned_addr learned my addr 172.21.15.179:0/457669356 (peer_addr_for_me v1:172.21.15.179:0/0) 2024-04-20T02:15:36.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): get_auth_request con 0x7f91b00993e0 auth_method 0 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient(hunting): handle_auth_done global_id 4159 payload 274 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 -- 172.21.15.179:0/457669356 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91b0135f70 msgr2=0x7f91b00a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b5cf9700 1 --2- 172.21.15.179:0/457669356 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91b0135f70 0x7f91b00a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 --2- 172.21.15.179:0/457669356 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91b0135f70 0x7f91b00a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 -- 172.21.15.179:0/457669356 >> v1:172.21.15.179:6789/0 conn(0x7f91b01363f0 legacy=0x7f91b00a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b54f8700 1 -- 172.21.15.179:0/457669356 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572276888 0 0) 0x7f91b00751e0 con 0x7f91b01363f0 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient: _finish_hunting 0 2024-04-20T02:15:36.146 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 monclient: found mon.noname-f 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b64fa700 1 -- 172.21.15.179:0/457669356 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b009ded0 con 0x7f91b00993e0 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.135+0000 7f91b54f8700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b64fa700 1 --2- 172.21.15.179:0/457669356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00993e0 0x7f91b00997c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009840 tx=0x7f91ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37272ea07e3057c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 1 -- 172.21.15.179:0/457669356 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91ac011aa0 con 0x7f91b00993e0 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: dump: 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.147 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: _finish_auth 0 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: _check_auth_tickets 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:36.148 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 1 -- 172.21.15.179:0/457669356 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac011c00 con 0x7f91b00993e0 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 5 monclient: authenticate success, global_id 4159 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 1 -- 172.21.15.179:0/457669356 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91ac011d70 con 0x7f91b00993e0 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 4 set_mon_vals no callback set 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b54f8700 10 monclient: dump: 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.149 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 10 monclient: shutdown 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 -- 172.21.15.179:0/457669356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00993e0 msgr2=0x7f91b00997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.150 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 --2- 172.21.15.179:0/457669356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00993e0 0x7f91b00997c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009840 tx=0x7f91ac009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 -- 172.21.15.179:0/457669356 shutdown_connections 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 --2- 172.21.15.179:0/457669356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91b00993e0 0x7f91b00997c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 --2- 172.21.15.179:0/457669356 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91b0135f70 0x7f91b00a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 -- 172.21.15.179:0/457669356 >> 172.21.15.179:0/457669356 conn(0x7f91b00a2fe0 msgr2=0x7f91b00a33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 -- 172.21.15.179:0/457669356 shutdown_connections 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.136+0000 7f91b875e700 1 -- 172.21.15.179:0/457669356 wait complete. 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 10 monclient: build_initial_monmap 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 10 monclient: monmap: 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 0 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 0.000000 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:created 0.000000 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:36.151 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 1 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 1 librados: starting msgr at 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 1 librados: starting objecter 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 1 Processor -- start 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 1 -- start start 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 1 librados: setting wanted keys 2024-04-20T02:15:36.152 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 1 librados: calling monclient init 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.137+0000 7f91b875e700 10 monclient: init 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient: _add_conns ranks=[7,5,0,1,4,2,3,6,8] 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 0x7f91b0134860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): picked mon.noname-i con 0x7f91b00993e0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91b0135f70 0x7f91b0134da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): picked mon.noname-h con 0x7f91b0135f70 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b01363f0 0x7f91b012b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): picked mon.noname-a con 0x7f91b01363f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b012dd20 con 0x7f91b01363f0 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.153 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91b009cf10 con 0x7f91b0135f70 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f91b009cac0 con 0x7f91b00993e0 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b875e700 10 monclient(hunting): authenticate will time out at 1599.545369s 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 0x7f91b0134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 0x7f91b0134860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.179:54940/0 (socket says 172.21.15.179:54940) 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91b0135f70 0x7f91b0134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 -- 172.21.15.179:0/1381089990 learned_addr learned my addr 172.21.15.179:0/1381089990 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): get_auth_request con 0x7f91b00993e0 auth_method 0 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): get_auth_request con 0x7f91b0135f70 auth_method 0 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.154 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b5cf9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b6cfb700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b01363f0 0x7f91b012b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b6cfb700 10 monclient(hunting): get_auth_request con 0x7f91b01363f0 auth_method 0 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b6cfb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b6cfb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b6cfb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient(hunting): handle_auth_done global_id 4147 payload 824 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 -- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91b0135f70 msgr2=0x7f91b0134da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.155 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91b0135f70 0x7f91b0134da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 -- 172.21.15.179:0/1381089990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b01363f0 msgr2=0x7f91b012b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b01363f0 0x7f91b012b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient: _finish_hunting 0 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 monclient: found mon.noname-i 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 -- 172.21.15.179:0/1381089990 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0094a0 con 0x7f91b00993e0 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b6cfb700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b01363f0 0x7f91b012b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91b64fa700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 0x7f91b0134860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91b006e830 tx=0x7f91ac004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91a6ffd700 10 client.?.objecter ms_handle_connect 0x7f91b00993e0 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.138+0000 7f91a6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91ac003b10 con 0x7f91b00993e0 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:36.156 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: dump: 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.157 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 5 monclient: authenticate success, global_id 4147 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac01f070 con 0x7f91b00993e0 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 10 monclient: _renew_subs 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b012da40 con 0x7f91b00993e0 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91ac003cd0 con 0x7f91b00993e0 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:36.158 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b4cf7700 4 set_mon_vals no callback set 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 monclient: dump: 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:epoch 1 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:election_strategy: 3 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.159 INFO:teuthology.orchestra.run.smithi179.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 10 client.4147.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 10 monclient: _renew_subs 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b012cb50 con 0x7f91b00993e0 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91b875e700 1 librados: init done 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f91ac026480 con 0x7f91b00993e0 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f91ac021070 con 0x7f91b00993e0 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 10 client.4147.objecter ms_dispatch 0x7f91b0001550 osd_map(7..7 src has 1..7) v4 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 3 client.4147.objecter handle_osd_map got epochs [7,7] > 0 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 3 client.4147.objecter handle_osd_map decoding full epoch 7 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.139+0000 7f91a6ffd700 20 client.4147.objecter dump_active .. 0 homeless 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.140+0000 7f91b875e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.140+0000 7f91b875e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:36.160 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.140+0000 7f91b875e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.140+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b006aac0 con 0x7f91b00993e0 2024-04-20T02:15:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.142+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f91ac015020 con 0x7f91b00993e0 2024-04-20T02:15:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.142+0000 7f91a6ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:36.161 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.142+0000 7f91a6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:36.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.227+0000 7f91b875e700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "fad619aa-0c5c-4c2d-82b3-510ce37c7c4c", "id": 5}] 2024-04-20T02:15:36.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.227+0000 7f91b875e700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "fad619aa-0c5c-4c2d-82b3-510ce37c7c4c", "id": 5}] 2024-04-20T02:15:36.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.227+0000 7f91b875e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:36.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.227+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "osd new", "uuid": "fad619aa-0c5c-4c2d-82b3-510ce37c7c4c", "id": 5} v 0) v1 -- 0x7f91b0099c80 con 0x7f91b00993e0 2024-04-20T02:15:36.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.232+0000 7f91a6ffd700 1 -- 172.21.15.179:0/1381089990 <== mon.7 v2:172.21.15.179:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fad619aa-0c5c-4c2d-82b3-510ce37c7c4c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f91ac003e30 con 0x7f91b00993e0 2024-04-20T02:15:36.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.232+0000 7f91a6ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "fad619aa-0c5c-4c2d-82b3-510ce37c7c4c", "id": 5}] 2024-04-20T02:15:36.231 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.232+0000 7f91a6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:36.231 INFO:teuthology.orchestra.run.smithi179.stdout:5 2024-04-20T02:15:36.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 10 librados: watch_flush enter 2024-04-20T02:15:36.232 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 10 librados: watch_flush exit 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 20 client.4147.objecter shutdown clearing up homeless session... 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 10 client.4147.objecter successfully canceled tick 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 10 monclient: shutdown 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 msgr2=0x7f91b0134860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 0x7f91b0134860 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91b006e830 tx=0x7f91ac004750 comp rx=0 tx=0).stop 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 shutdown_connections 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b01363f0 0x7f91b012b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b00993e0 0x7f91b0134860 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 --2- 172.21.15.179:0/1381089990 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91b0135f70 0x7f91b0134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 >> 172.21.15.179:0/1381089990 conn(0x7f91b00a2fe0 msgr2=0x7f91b00a49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 shutdown_connections 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 -- 172.21.15.179:0/1381089990 wait complete. 2024-04-20T02:15:36.233 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-20T02:15:36.233+0000 7f91b875e700 1 librados: shutdown 2024-04-20T02:15:36.244 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T02:15:36.244 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T02:15:36.246 INFO:tasks.ceph.osd.0:Started 2024-04-20T02:15:36.247 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T02:15:36.247 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T02:15:36.249 INFO:tasks.ceph.osd.1:Started 2024-04-20T02:15:36.249 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T02:15:36.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T02:15:36.251 INFO:tasks.ceph.osd.2:Started 2024-04-20T02:15:36.251 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T02:15:36.251 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T02:15:36.253 INFO:tasks.ceph.osd.3:Started 2024-04-20T02:15:36.253 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T02:15:36.253 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T02:15:36.255 INFO:tasks.ceph.osd.4:Started 2024-04-20T02:15:36.255 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T02:15:36.255 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T02:15:36.257 INFO:tasks.ceph.osd.5:Started 2024-04-20T02:15:36.257 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T02:15:36.325 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:36.326+0000 7f86c2c98200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.411+0000 7fd407a34700 10 monclient: get_monmap_and_config 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.411+0000 7fd407a34700 10 monclient: build_initial_monmap 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.411+0000 7fd407a34700 10 monclient: monmap: 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:36.412879+0000 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:36.412879+0000 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:36.411 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:36.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.412+0000 7fd407a34700 1 Processor -- start 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.412+0000 7fd407a34700 1 -- start start 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.412+0000 7fd407a34700 10 monclient: init 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient: _add_conns ranks=[3,5,1,8,0,9,6,10,2,4,7] 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 0x7fd4000a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): picked mon.noname-g con 0x7fd4000a0730 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 0x7fd4000a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): picked mon.noname-h con 0x7fd4000a1050 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd40009ffd0 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4000751e0 con 0x7fd40009ffd0 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40009f000 con 0x7fd4000a0730 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd40009ed20 con 0x7fd4000a1050 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd407a34700 10 monclient(hunting): authenticate will time out at 1615.814249s 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 0x7fd4000a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 0x7fd4000a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd405fd1700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd40009ffd0 0x7fd4000a2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:40094/0 (socket says 172.21.15.47:40094) 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 0x7fd4000a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:59224/0 (socket says 172.21.15.47:59224) 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 0x7fd4000a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:55428/0 (socket says 172.21.15.47:55428) 2024-04-20T02:15:36.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd405fd1700 1 -- 172.21.15.47:0/2238012483 learned_addr learned my addr 172.21.15.47:0/2238012483 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): get_auth_request con 0x7fd4000a1050 auth_method 0 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): get_auth_request con 0x7fd4000a0730 auth_method 0 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient(hunting): handle_auth_done global_id 4172 payload 274 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd3fffff700 1 -- 172.21.15.47:0/2238012483 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691380764 0 0) 0x7fd4000751e0 con 0x7fd40009ffd0 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 -- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 msgr2=0x7fd4000a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 --2- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 0x7fd4000a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 -- 172.21.15.47:0/2238012483 >> v1:172.21.15.179:6789/0 conn(0x7fd40009ffd0 legacy=0x7fd4000a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient: _finish_hunting 0 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 monclient: found mon.noname-h 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd404fcf700 1 -- 172.21.15.47:0/2238012483 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40009f2e0 con 0x7fd4000a1050 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd4057d0700 1 --2- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 0x7fd4000a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.413+0000 7fd3fffff700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd404fcf700 1 --2- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 0x7fd4000a1d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009870 tx=0x7fd3f4009b80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6c4ec40c9d18c874 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 1 -- 172.21.15.47:0/2238012483 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3f4011aa0 con 0x7fd4000a1050 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: dump: 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.425 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: _finish_auth 0 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 1 -- 172.21.15.47:0/2238012483 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4011c00 con 0x7fd4000a1050 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 1 -- 172.21.15.47:0/2238012483 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3f4011d70 con 0x7fd4000a1050 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd3fffff700 10 monclient: dump: 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:36.426 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.427 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 5 monclient: authenticate success, global_id 4172 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 4 set_mon_vals no callback set 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 10 monclient: shutdown 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 1 -- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 msgr2=0x7fd4000a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 1 --2- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 0x7fd4000a1d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009870 tx=0x7fd3f4009b80 comp rx=0 tx=0).stop 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 1 -- 172.21.15.47:0/2238012483 shutdown_connections 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 1 --2- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd4000a0730 0x7fd4000a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 1 --2- 172.21.15.47:0/2238012483 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd4000a1050 0x7fd4000a1d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.414+0000 7fd407a34700 1 -- 172.21.15.47:0/2238012483 >> 172.21.15.47:0/2238012483 conn(0x7fd40006cbf0 msgr2=0x7fd40006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.415+0000 7fd407a34700 1 -- 172.21.15.47:0/2238012483 shutdown_connections 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.415+0000 7fd407a34700 1 -- 172.21.15.47:0/2238012483 wait complete. 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.415+0000 7fd407a34700 10 monclient: build_initial_monmap 2024-04-20T02:15:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.415+0000 7fd407a34700 10 monclient: monmap: 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.415+0000 7fd407a34700 1 librados: starting msgr at 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.415+0000 7fd407a34700 1 librados: starting objecter 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 Processor -- start 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 -- start start 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 librados: setting wanted keys 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 librados: calling monclient init 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient: init 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient: _add_conns ranks=[4,6,8,2,3,1,0,7,5] 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 0x7fd4001cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): picked mon.noname-c con 0x7fd40009ffd0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 0x7fd400137e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): picked mon.noname-d con 0x7fd4000a0730 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd4000a1050 0x7fd400138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): picked mon.noname-e con 0x7fd4000a1050 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd400125320 con 0x7fd40009ffd0 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd4001cd1c0 con 0x7fd4000a0730 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd4001388a0 con 0x7fd4000a1050 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd407a34700 10 monclient(hunting): authenticate will time out at 1615.817615s 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 0x7fd4001cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 0x7fd400137e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 0x7fd400137e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:55136/0 (socket says 172.21.15.47:55136) 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd4000a1050 0x7fd400138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 0x7fd4001cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37988/0 (socket says 172.21.15.47:37988) 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 1 -- 172.21.15.47:0/3731613843 learned_addr learned my addr 172.21.15.47:0/3731613843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): get_auth_request con 0x7fd4000a0730 auth_method 0 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): get_auth_request con 0x7fd40009ffd0 auth_method 0 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): get_auth_request con 0x7fd4000a1050 auth_method 0 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd404fcf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd4057d0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.416+0000 7fd405fd1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 10 monclient(hunting): handle_auth_done global_id 4157 payload 824 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 -- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 msgr2=0x7fd400137e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 0x7fd400137e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 -- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 msgr2=0x7fd4001cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 0x7fd4001cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 10 monclient: _finish_hunting 0 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 monclient: found mon.noname-e 2024-04-20T02:15:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 -- 172.21.15.47:0/3731613843 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0009a70 con 0x7fd4000a1050 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd404fcf700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 0x7fd400137e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd4057d0700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 0x7fd4001cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd405fd1700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd4000a1050 0x7fd400138360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8000c00 tx=0x7fd3f800c730 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 client.?.objecter ms_handle_connect 0x7fd4000a1050 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3f8004ce0 con 0x7fd4000a1050 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: dump: 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f8004e60 con 0x7fd4000a1050 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 5 monclient: authenticate success, global_id 4157 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 10 monclient: _renew_subs 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f40094a0 con 0x7fd4000a1050 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3f8007540 con 0x7fd4000a1050 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3fdffb700 10 monclient: dump: 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd3ff7fe700 4 set_mon_vals no callback set 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 10 client.4157.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 10 monclient: _renew_subs 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd400123410 con 0x7fd4000a1050 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.417+0000 7fd407a34700 1 librados: init done 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd3f8004870 con 0x7fd4000a1050 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fd3f80139b0 con 0x7fd4000a1050 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd3fdffb700 10 client.4157.objecter ms_dispatch 0x7fd400001550 osd_map(8..8 src has 1..8) v4 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd3fdffb700 3 client.4157.objecter handle_osd_map got epochs [8,8] > 0 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd3fdffb700 3 client.4157.objecter handle_osd_map decoding full epoch 8 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd3fdffb700 20 client.4157.objecter dump_active .. 0 homeless 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd407a34700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd407a34700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd407a34700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.418+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd400125460 con 0x7fd4000a1050 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.421+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd400125460 con 0x7fd4000a1050 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.421+0000 7fd3fdffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.421+0000 7fd3fdffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd407a34700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd407a34700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd407a34700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd400066610 con 0x7fd4000a1050 2024-04-20T02:15:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd3fdffb700 1 -- 172.21.15.47:0/3731613843 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5604 (secure 0 0 0) 0x7fd3f80076a0 con 0x7fd4000a1050 2024-04-20T02:15:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd3fdffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.496+0000 7fd3fdffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:36.496 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":8,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:15:36.229103+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:15:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 10 librados: watch_flush enter 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 10 librados: watch_flush exit 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 20 client.4157.objecter shutdown clearing up homeless session... 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 10 client.4157.objecter successfully canceled tick 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 10 monclient: shutdown 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd4000a1050 msgr2=0x7fd400138360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd4000a1050 0x7fd400138360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8000c00 tx=0x7fd3f800c730 comp rx=0 tx=0).stop 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 shutdown_connections 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd4000a1050 0x7fd400138360 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd40009ffd0 0x7fd4001cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 --2- 172.21.15.47:0/3731613843 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd4000a0730 0x7fd400137e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 >> 172.21.15.47:0/3731613843 conn(0x7fd40006cbf0 msgr2=0x7fd400071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 shutdown_connections 2024-04-20T02:15:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 -- 172.21.15.47:0/3731613843 wait complete. 2024-04-20T02:15:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:36.497+0000 7fd407a34700 1 librados: shutdown 2024-04-20T02:15:36.509 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T02:15:36.510 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T02:15:36.654 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:36.655+0000 7f86c2c98200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T02:15:36.813 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:36.814+0000 7f86c2c98200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T02:15:36.853 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-04-20T02:15:36.854+0000 7f8332e5c7c0 -1 Falling back to public interface 2024-04-20T02:15:36.858 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-04-20T02:15:36.859+0000 7fd3fdf717c0 -1 Falling back to public interface 2024-04-20T02:15:36.858 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-04-20T02:15:36.860+0000 7f8feba957c0 -1 Falling back to public interface 2024-04-20T02:15:36.871 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T02:15:36.872+0000 7f55066137c0 -1 Falling back to public interface 2024-04-20T02:15:36.871 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:15:36.873+0000 7ffb3b7c97c0 -1 Falling back to public interface 2024-04-20T02:15:36.872 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T02:15:36.873+0000 7fd4a33167c0 -1 Falling back to public interface 2024-04-20T02:15:36.896 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:36.897+0000 7f86c2c98200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T02:15:37.116 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.252+0000 7f7558a10700 10 monclient: get_monmap_and_config 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.252+0000 7f7558a10700 10 monclient: build_initial_monmap 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.252+0000 7f7558a10700 10 monclient: monmap: 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:epoch 0 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:37.253245+0000 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:37.253245+0000 2024-04-20T02:15:37.251 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:37.252 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:37.261 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.252+0000 7f7558a10700 1 Processor -- start 2024-04-20T02:15:37.261 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.252+0000 7f7558a10700 1 -- start start 2024-04-20T02:15:37.261 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.252+0000 7f7558a10700 10 monclient: init 2024-04-20T02:15:37.261 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:37.261 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient: _add_conns ranks=[4,6,9,0,2,5,7,3,1,10,8] 2024-04-20T02:15:37.261 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 0x7f75540a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): picked mon.noname-c con 0x7f75540a2790 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 0x7f75540a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): picked mon.noname-d con 0x7f7554072070 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7554133530 0x7f75541339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): picked mon.noname-a con 0x7f7554133530 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755409efe0 con 0x7f7554133530 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75540a0450 con 0x7f75540a2790 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f75540a35f0 con 0x7f7554072070 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7558a10700 10 monclient(hunting): authenticate will time out at 1616.654569s 2024-04-20T02:15:37.262 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 0x7f75540a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 0x7f75540a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7554133530 0x7f75541339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 0x7f75540a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:55150/0 (socket says 172.21.15.47:55150) 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 1 -- 172.21.15.47:0/2547353433 learned_addr learned my addr 172.21.15.47:0/2547353433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 0x7f75540a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37992/0 (socket says 172.21.15.47:37992) 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): get_auth_request con 0x7f7554072070 auth_method 0 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): get_auth_request con 0x7f75540a2790 auth_method 0 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): get_auth_request con 0x7f7554133530 auth_method 0 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:37.263 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f755259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7551d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.253+0000 7f7552d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 10 monclient(hunting): handle_auth_done global_id 4155 payload 274 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 -- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 msgr2=0x7f75540a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 --2- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 0x7f75540a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 -- 172.21.15.47:0/2547353433 >> v2:172.21.15.47:3300/0 conn(0x7f7554133530 msgr2=0x7f75541339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:37.264 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 --2- 172.21.15.47:0/2547353433 >> v2:172.21.15.47:3300/0 conn(0x7f7554133530 0x7f75541339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 monclient: found mon.noname-d 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 -- 172.21.15.47:0/2547353433 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755404fb40 con 0x7f7554072070 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7552d9d700 1 --2- 172.21.15.47:0/2547353433 >> v2:172.21.15.47:3300/0 conn(0x7f7554133530 0x7f75541339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755259c700 1 --2- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 0x7f75540a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7551d9b700 1 --2- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 0x7f75540a30b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7548009920 tx=0x7f7548009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=52af8062356edf63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 1 -- 172.21.15.47:0/2547353433 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7548011aa0 con 0x7f7554072070 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: dump: 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:37.265 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:37.266 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: _finish_auth 0 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: _check_auth_tickets 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 1 -- 172.21.15.47:0/2547353433 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548011c00 con 0x7f7554072070 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7558a10700 5 monclient: authenticate success, global_id 4155 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7558a10700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 1 -- 172.21.15.47:0/2547353433 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7548011d70 con 0x7f7554072070 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7558a10700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f7558a10700 4 set_mon_vals no callback set 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.254+0000 7f755159a700 10 monclient: dump: 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:37.267 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 10 monclient: shutdown 2024-04-20T02:15:37.268 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 -- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 msgr2=0x7f75540a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 --2- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 0x7f75540a30b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7548009920 tx=0x7f7548009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 -- 172.21.15.47:0/2547353433 shutdown_connections 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 --2- 172.21.15.47:0/2547353433 >> v2:172.21.15.47:3300/0 conn(0x7f7554133530 0x7f75541339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 --2- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75540a2790 0x7f75540a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 --2- 172.21.15.47:0/2547353433 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7554072070 0x7f75540a30b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 -- 172.21.15.47:0/2547353433 >> 172.21.15.47:0/2547353433 conn(0x7f75540a73b0 msgr2=0x7f75540a3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 -- 172.21.15.47:0/2547353433 shutdown_connections 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.255+0000 7f7558a10700 1 -- 172.21.15.47:0/2547353433 wait complete. 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 10 monclient: build_initial_monmap 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 10 monclient: monmap: 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:epoch 0 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:37.269 INFO:teuthology.misc.health.smithi047.stderr:created 0.000000 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 librados: starting msgr at 2024-04-20T02:15:37.270 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 librados: starting objecter 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 Processor -- start 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 -- start start 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 librados: setting wanted keys 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 librados: calling monclient init 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 10 monclient: init 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 10 monclient: _add_conns ranks=[8,2,3,7,1,5,4,0,6] 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7554072070 0x7f75541cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.256+0000 7f7558a10700 10 monclient(hunting): picked mon.noname-e con 0x7f7554072070 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 0x7f75541bc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): picked mon.noname-b con 0x7f75540a2790 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7554133530 0x7f75541c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): picked mon.noname-g con 0x7f7554133530 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:37.271 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75541bb290 con 0x7f75540a2790 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75540714a0 con 0x7f7554133530 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f75541b95f0 con 0x7f7554072070 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient(hunting): authenticate will time out at 1616.658065s 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7554072070 0x7f75541cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 0x7f75541bc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7554072070 0x7f75541cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:38894/0 (socket says 172.21.15.47:38894) 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 0x7f75541bc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45260/0 (socket says 172.21.15.47:45260) 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 1 -- 172.21.15.47:0/2620588769 learned_addr learned my addr 172.21.15.47:0/2620588769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): get_auth_request con 0x7f7554072070 auth_method 0 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:37.272 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): get_auth_request con 0x7f75540a2790 auth_method 0 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7554133530 0x7f75541c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f755259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): get_auth_request con 0x7f7554133530 auth_method 0 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:37.273 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7552d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient(hunting): handle_auth_done global_id 4151 payload 824 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 -- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7554072070 msgr2=0x7f75541cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7554072070 0x7f75541cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 -- 172.21.15.47:0/2620588769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7554133530 msgr2=0x7f75541c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7554133530 0x7f75541c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 monclient: found mon.noname-b 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:37.274 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 -- 172.21.15.47:0/2620588769 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440094a0 con 0x7f75540a2790 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7551d9b700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 0x7f75541bc750 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7548016000 tx=0x7f7548004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 client.?.objecter ms_handle_connect 0x7f75540a2790 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75480043d0 con 0x7f75540a2790 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: dump: 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:37.275 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480045b0 con 0x7f75540a2790 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 5 monclient: authenticate success, global_id 4151 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient: _renew_subs 2024-04-20T02:15:37.276 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c009a70 con 0x7f75540a2790 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f754801a450 con 0x7f75540a2790 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7542ffd700 10 monclient: dump: 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:37.277 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.257+0000 7f7550d99700 4 set_mon_vals no callback set 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 10 client.4151.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 10 monclient: _renew_subs 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7548009550 con 0x7f75540a2790 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 1 librados: init done 2024-04-20T02:15:37.278 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7548021020 con 0x7f75540a2790 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f754801abc0 con 0x7f75540a2790 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7542ffd700 10 client.4151.objecter ms_dispatch 0x7f7554001550 osd_map(8..8 src has 1..8) v4 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7542ffd700 3 client.4151.objecter handle_osd_map got epochs [8,8] > 0 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7542ffd700 3 client.4151.objecter handle_osd_map decoding full epoch 8 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7542ffd700 20 client.4151.objecter dump_active .. 0 homeless 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.258+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755406afd0 con 0x7f75540a2790 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.261+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f7548029060 con 0x7f75540a2790 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.261+0000 7f7542ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:37.279 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.261+0000 7f7542ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:37.337 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7558a10700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:37.337 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7558a10700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:37.337 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7558a10700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:37.337 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7554071a80 con 0x7f75540a2790 2024-04-20T02:15:37.338 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7542ffd700 1 -- 172.21.15.47:0/2620588769 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5604 (secure 0 0 0) 0x7f7548015020 con 0x7f75540a2790 2024-04-20T02:15:37.338 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7542ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:37.338 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.338+0000 7f7542ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:37.338 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-20T02:15:37.338 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":8,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:15:36.229103+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:15:37.339 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 10 librados: watch_flush enter 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 10 librados: watch_flush exit 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 20 client.4151.objecter shutdown clearing up homeless session... 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 10 client.4151.objecter successfully canceled tick 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 10 monclient: shutdown 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 msgr2=0x7f75541bc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 0x7f75541bc750 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7548016000 tx=0x7f7548004750 comp rx=0 tx=0).stop 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 shutdown_connections 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7554072070 0x7f75541cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7554133530 0x7f75541c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 --2- 172.21.15.47:0/2620588769 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f75540a2790 0x7f75541bc750 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 >> 172.21.15.47:0/2620588769 conn(0x7f75540a73b0 msgr2=0x7f75540a5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 shutdown_connections 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 -- 172.21.15.47:0/2620588769 wait complete. 2024-04-20T02:15:37.340 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:37.339+0000 7f7558a10700 1 librados: shutdown 2024-04-20T02:15:37.350 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-04-20T02:15:37.599 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:37.600+0000 7f86c2c98200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T02:15:37.681 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:37.682+0000 7f86c2c98200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T02:15:37.815 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:37.816+0000 7f86c2c98200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T02:15:38.365 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:38.367+0000 7f86c2c98200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T02:15:38.532 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:38.533+0000 7f86c2c98200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T02:15:38.696 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:38.697+0000 7f86c2c98200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T02:15:39.535 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:39.536+0000 7f86c2c98200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T02:15:39.797 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:39.798+0000 7f86c2c98200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T02:15:39.874 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:39.875+0000 7f86c2c98200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T02:15:40.130 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-04-20T02:15:40.131+0000 7fd3fdf717c0 -1 osd.5 0 log_to_monitors true 2024-04-20T02:15:40.263 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:40.264+0000 7f86c2c98200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T02:15:40.331 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:15:40.332+0000 7ffb3b7c97c0 -1 osd.1 0 log_to_monitors true 2024-04-20T02:15:40.372 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T02:15:40.373+0000 7f55066137c0 -1 osd.0 0 log_to_monitors true 2024-04-20T02:15:40.374 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-04-20T02:15:40.376+0000 7f8feba957c0 -1 osd.4 0 log_to_monitors true 2024-04-20T02:15:40.374 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-04-20T02:15:40.376+0000 7f8332e5c7c0 -1 osd.3 0 log_to_monitors true 2024-04-20T02:15:40.406 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T02:15:40.407+0000 7fd4a33167c0 -1 osd.2 0 log_to_monitors true 2024-04-20T02:15:41.305 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.305+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T02:15:41.305 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.306+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.d 2024-04-20T02:15:41.305 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.306+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T02:15:41.306 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.306+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.e 2024-04-20T02:15:41.306 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.307+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.g 2024-04-20T02:15:41.306 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.307+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.h 2024-04-20T02:15:41.307 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.308+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.f 2024-04-20T02:15:41.309 INFO:tasks.ceph.mgr.x.smithi179.stderr:2024-04-20T02:15:41.310+0000 7f867df0e700 -1 mgr.server handle_report got status from non-daemon mon.i 2024-04-20T02:15:41.956 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-04-20T02:15:41.957+0000 7fd3f2329700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T02:15:41.971 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-04-20T02:15:41.972+0000 7fd3e9f03700 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T02:15:42.205 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-04-20T02:15:42.207+0000 7f8327214700 -1 osd.3 0 waiting for initial osdmap 2024-04-20T02:15:42.206 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T02:15:42.207+0000 7f54fa9cb700 -1 osd.0 0 waiting for initial osdmap 2024-04-20T02:15:42.206 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:15:42.207+0000 7ffb2fb81700 -1 osd.1 0 waiting for initial osdmap 2024-04-20T02:15:42.207 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-04-20T02:15:42.208+0000 7f8fdfe4d700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T02:15:42.209 INFO:tasks.ceph.osd.3.smithi179.stderr:2024-04-20T02:15:42.210+0000 7f8322df6700 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T02:15:42.210 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T02:15:42.211+0000 7f54f65ad700 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T02:15:42.211 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-04-20T02:15:42.213+0000 7f8fdba2f700 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T02:15:42.213 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:15:42.214+0000 7ffb2b763700 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T02:15:42.821 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T02:15:42.822+0000 7fd4976ce700 -1 osd.2 0 waiting for initial osdmap 2024-04-20T02:15:42.839 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T02:15:42.840+0000 7fd48f2a8700 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T02:15:43.956 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T02:15:44.089 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.090+0000 7f8ea5862700 10 monclient: get_monmap_and_config 2024-04-20T02:15:44.089 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.090+0000 7f8ea5862700 10 monclient: build_initial_monmap 2024-04-20T02:15:44.089 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.090+0000 7f8ea5862700 10 monclient: monmap: 2024-04-20T02:15:44.089 INFO:teuthology.misc.health.smithi047.stderr:epoch 0 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:44.091444+0000 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:44.091444+0000 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:44.090 INFO:teuthology.misc.health.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:44.091 INFO:teuthology.misc.health.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:44.091 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.090+0000 7f8ea5862700 1 Processor -- start 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.090+0000 7f8ea5862700 1 -- start start 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.090+0000 7f8ea5862700 10 monclient: init 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient: _add_conns ranks=[5,9,3,0,2,8,1,6,10,4,7] 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8ea00a3ae0 0x7f8ea0137d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): picked mon.noname-h con 0x7f8ea00a3ae0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ea00a3ec0 0x7f8ea00a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.101 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): picked mon.noname-a con 0x7f8ea00a3ec0 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea00a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): picked mon.noname-g con 0x7f8ea00a42a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea00a1700 con 0x7f8ea00a3ec0 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea009f6e0 con 0x7f8ea00a42a0 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea00a0bc0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8ea5862700 10 monclient(hunting): authenticate will time out at 1623.492520s 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ea00a3ec0 0x7f8ea00a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ea00a3ec0 0x7f8ea00a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56820/0 (socket says 172.21.15.47:56820) 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 1 -- 172.21.15.47:0/60836147 learned_addr learned my addr 172.21.15.47:0/60836147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:44.102 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): get_auth_request con 0x7f8ea00a3ec0 auth_method 0 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9effd700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8ea00a3ae0 0x7f8ea0137d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9effd700 10 monclient(hunting): get_auth_request con 0x7f8ea00a3ae0 auth_method 0 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9f7fe700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea00a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9f7fe700 10 monclient(hunting): get_auth_request con 0x7f8ea00a42a0 auth_method 0 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.103 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_done global_id 4248 payload 274 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 1 -- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8ea00a3ae0 msgr2=0x7f8ea0137d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8ea00a3ae0 0x7f8ea0137d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.091+0000 7f8e9e7fc700 1 -- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 msgr2=0x7f8ea00a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea00a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 1 monclient: found mon.noname-a 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 1 -- 172.21.15.47:0/60836147 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea004fb40 con 0x7f8ea00a3ec0 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9e7fc700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea00a3ec0 0x7f8ea00a5a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009920 tx=0x7f8e94009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92eb983107307b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 1 -- 172.21.15.47:0/60836147 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e94009e70 con 0x7f8ea00a3ec0 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.104 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: dump: 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.105 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 1 -- 172.21.15.47:0/60836147 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9400bc90 con 0x7f8ea00a3ec0 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 5 monclient: authenticate success, global_id 4248 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 1 -- 172.21.15.47:0/60836147 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e9400be00 con 0x7f8ea00a3ec0 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 4 set_mon_vals no callback set 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:44.106 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8e9dffb700 10 monclient: dump: 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.107 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 10 monclient: shutdown 2024-04-20T02:15:44.108 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 1 -- 172.21.15.47:0/60836147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea00a3ec0 msgr2=0x7f8ea00a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.092+0000 7f8ea5862700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea00a3ec0 0x7f8ea00a5a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009920 tx=0x7f8e94009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 -- 172.21.15.47:0/60836147 shutdown_connections 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea00a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8ea00a3ae0 0x7f8ea0137d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 --2- 172.21.15.47:0/60836147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ea00a3ec0 0x7f8ea00a5a90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 -- 172.21.15.47:0/60836147 >> 172.21.15.47:0/60836147 conn(0x7f8ea00aa6b0 msgr2=0x7f8ea00a6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 -- 172.21.15.47:0/60836147 shutdown_connections 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 -- 172.21.15.47:0/60836147 wait complete. 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 10 monclient: build_initial_monmap 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 10 monclient: monmap: 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:epoch 0 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:44.109 INFO:teuthology.misc.health.smithi047.stderr:created 0.000000 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 librados: starting msgr at 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.093+0000 7f8ea5862700 1 librados: starting objecter 2024-04-20T02:15:44.110 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 Processor -- start 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 -- start start 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 librados: setting wanted keys 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 librados: calling monclient init 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient: init 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient: _add_conns ranks=[2,7,3,8,5,4,0,6,1] 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 0x7f8ea0136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): picked mon.noname-b con 0x7f8ea00a3ae0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8ea00a3ec0 0x7f8ea01371a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): picked mon.noname-i con 0x7f8ea00a3ec0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea01c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): picked mon.noname-g con 0x7f8ea00a42a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.111 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea01376e0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0124060 con 0x7f8ea00a42a0 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea009fd60 con 0x7f8ea00a3ec0 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8ea5862700 10 monclient(hunting): authenticate will time out at 1623.495868s 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.094+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 0x7f8ea0136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 0x7f8ea0136c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52970/0 (socket says 172.21.15.47:52970) 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 -- 172.21.15.47:0/3505058851 learned_addr learned my addr 172.21.15.47:0/3505058851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:44.112 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): get_auth_request con 0x7f8ea00a3ae0 auth_method 0 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8ea00a3ec0 0x7f8ea01371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea01c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): get_auth_request con 0x7f8ea00a3ec0 auth_method 0 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): get_auth_request con 0x7f8ea00a42a0 auth_method 0 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.113 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient(hunting): handle_auth_done global_id 4187 payload 824 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 -- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8ea00a3ec0 msgr2=0x7f8ea01371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8ea00a3ec0 0x7f8ea01371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 -- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 msgr2=0x7f8ea01c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea01c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:44.114 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 monclient: found mon.noname-b 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 -- 172.21.15.47:0/3505058851 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94009550 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9effd700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 0x7f8ea0136c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000e810 tx=0x7f8e9000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 client.?.objecter ms_handle_connect 0x7f8ea00a3ae0 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e9000c070 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: dump: 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.115 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90013880 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 5 monclient: authenticate success, global_id 4187 2024-04-20T02:15:44.116 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 10 monclient: _renew_subs 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea0072380 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e90013e60 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e9d7fa700 4 set_mon_vals no callback set 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 10 client.4187.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8e87fff700 10 monclient: dump: 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:epoch 1 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.117 INFO:teuthology.misc.health.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr: 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 10 monclient: _renew_subs 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea0121c10 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.118 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.095+0000 7f8ea5862700 1 librados: init done 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f8e900139f0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8ea5862700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8ea5862700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8ea5862700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea006afd0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e8c03a1c0 0x7f8e8c03c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3607+0+0 (secure 0 0 0) 0x7f8e900498e0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 10 client.4187.objecter ms_dispatch 0x7f8ea0001550 osd_map(12..12 src has 1..12) v4 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 3 client.4187.objecter handle_osd_map got epochs [12,12] > 0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 3 client.4187.objecter handle_osd_map decoding full epoch 12 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.096+0000 7f8e87fff700 20 client.4187.objecter dump_active .. 0 homeless 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.097+0000 7f8e9e7fc700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e8c03a1c0 0x7f8e8c03c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.097+0000 7f8e9e7fc700 10 monclient: get_auth_request con 0x7f8e8c03a1c0 auth_method 0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.097+0000 7f8e87fff700 10 client.4187.objecter ms_handle_connect 0x7f8e8c03a1c0 2024-04-20T02:15:44.119 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.097+0000 7f8e9e7fc700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e8c03a1c0 0x7f8e8c03c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e94000c00 tx=0x7f8e94008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:44.120 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.102+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f8e90013ca0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.120 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.102+0000 7f8e87fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:44.120 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.102+0000 7f8e87fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:44.324 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.324+0000 7f8ea5862700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:44.324 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.324+0000 7f8ea5862700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:44.324 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.324+0000 7f8ea5862700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:44.324 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.324+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ea01244d0 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.325 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.325+0000 7f8e87fff700 1 -- 172.21.15.47:0/3505058851 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8286 (secure 0 0 0) 0x7f8e90017070 con 0x7f8ea00a3ae0 2024-04-20T02:15:44.325 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.325+0000 7f8e87fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:44.325 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.325+0000 7f8e87fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:44.325 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-20T02:15:44.325 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":13,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:15:44.218250+0000","last_up_change":"2024-04-20T02:15:43.202903+0000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":42309},{"type":"v1","addr":"172.21.15.47:6801","nonce":42309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":42309},{"type":"v1","addr":"172.21.15.47:6803","nonce":42309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":42309},{"type":"v1","addr":"172.21.15.47:6811","nonce":42309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":42309},{"type":"v1","addr":"172.21.15.47:6806","nonce":42309}]},"public_addr":"172.21.15.47:6801/42309","cluster_addr":"172.21.15.47:6803/42309","heartbeat_back_addr":"172.21.15.47:6811/42309","heartbeat_front_addr":"172.21.15.47:6806/42309","state":["exists","up"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":42310},{"type":"v1","addr":"172.21.15.47:6807","nonce":42310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":42310},{"type":"v1","addr":"172.21.15.47:6813","nonce":42310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6819","nonce":42310},{"type":"v1","addr":"172.21.15.47:6821","nonce":42310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":42310},{"type":"v1","addr":"172.21.15.47:6817","nonce":42310}]},"public_addr":"172.21.15.47:6807/42310","cluster_addr":"172.21.15.47:6813/42310","heartbeat_back_addr":"172.21.15.47:6821/42310","heartbeat_front_addr":"172.21.15.47:6817/42310","state":["exists","up"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":42311},{"type":"v1","addr":"172.21.15.47:6812","nonce":42311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":42311},{"type":"v1","addr":"172.21.15.47:6816","nonce":42311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":42311},{"type":"v1","addr":"172.21.15.47:6823","nonce":42311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":42311},{"type":"v1","addr":"172.21.15.47:6820","nonce":42311}]},"public_addr":"172.21.15.47:6812/42311","cluster_addr":"172.21.15.47:6816/42311","heartbeat_back_addr":"172.21.15.47:6823/42311","heartbeat_front_addr":"172.21.15.47:6820/42311","state":["exists","up"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":42294},{"type":"v1","addr":"172.21.15.179:6801","nonce":42294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":42294},{"type":"v1","addr":"172.21.15.179:6803","nonce":42294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":42294},{"type":"v1","addr":"172.21.15.179:6807","nonce":42294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":42294},{"type":"v1","addr":"172.21.15.179:6805","nonce":42294}]},"public_addr":"172.21.15.179:6801/42294","cluster_addr":"172.21.15.179:6803/42294","heartbeat_back_addr":"172.21.15.179:6807/42294","heartbeat_front_addr":"172.21.15.179:6805/42294","state":["exists","up"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":42295},{"type":"v1","addr":"172.21.15.179:6812","nonce":42295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":42295},{"type":"v1","addr":"172.21.15.179:6816","nonce":42295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":42295},{"type":"v1","addr":"172.21.15.179:6823","nonce":42295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":42295},{"type":"v1","addr":"172.21.15.179:6820","nonce":42295}]},"public_addr":"172.21.15.179:6812/42295","cluster_addr":"172.21.15.179:6816/42295","heartbeat_back_addr":"172.21.15.179:6823/42295","heartbeat_front_addr":"172.21.15.179:6820/42295","state":["exists","up"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":42296},{"type":"v1","addr":"172.21.15.179:6809","nonce":42296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":42296},{"type":"v1","addr":"172.21.15.179:6813","nonce":42296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":42296},{"type":"v1","addr":"172.21.15.179:6821","nonce":42296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":42296},{"type":"v1","addr":"172.21.15.179:6817","nonce":42296}]},"public_addr":"172.21.15.179:6809/42296","cluster_addr":"172.21.15.179:6813/42296","heartbeat_back_addr":"172.21.15.179:6821/42296","heartbeat_front_addr":"172.21.15.179:6817/42296","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.384773+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.348973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.447042+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.151528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 10 librados: watch_flush enter 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 10 librados: watch_flush exit 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 20 client.4187.objecter shutdown clearing up homeless session... 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 10 client.4187.objecter successfully canceled tick 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e8c03a1c0 msgr2=0x7f8e8c03c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e8c03a1c0 0x7f8e8c03c680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e94000c00 tx=0x7f8e94008030 comp rx=0 tx=0).stop 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 10 monclient: shutdown 2024-04-20T02:15:44.328 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 msgr2=0x7f8ea0136c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 0x7f8ea0136c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000e810 tx=0x7f8e9000eb20 comp rx=0 tx=0).stop 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 shutdown_connections 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e8c03a1c0 0x7f8e8c03c680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ea00a42a0 0x7f8ea01c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8ea00a3ae0 0x7f8ea0136c60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 --2- 172.21.15.47:0/3505058851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8ea00a3ec0 0x7f8ea01371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 >> 172.21.15.47:0/3505058851 conn(0x7f8ea00aa6b0 msgr2=0x7f8ea00a5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 shutdown_connections 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 -- 172.21.15.47:0/3505058851 wait complete. 2024-04-20T02:15:44.329 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T02:15:44.328+0000 7f8ea5862700 1 librados: shutdown 2024-04-20T02:15:44.340 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-04-20T02:15:44.340 INFO:tasks.ceph:Creating RBD pool 2024-04-20T02:15:44.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-20T02:15:44.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.476+0000 7f778b840700 10 monclient: get_monmap_and_config 2024-04-20T02:15:44.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.476+0000 7f778b840700 10 monclient: build_initial_monmap 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.476+0000 7f778b840700 10 monclient: monmap: 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:44.477760+0000 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:44.477760+0000 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:44.477 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.477+0000 7f778b840700 1 Processor -- start 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.477+0000 7f778b840700 1 -- start start 2024-04-20T02:15:44.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.477+0000 7f778b840700 10 monclient: init 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient: _add_conns ranks=[7,3,4,6,1,10,5,0,9,8,2] 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7784071860 0x7f7784071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): picked mon.noname-i con 0x7f7784071860 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841344a0 0x7f77840a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): picked mon.noname-g con 0x7f77841344a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 0x7f77840a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): picked mon.noname-c con 0x7f77841348a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778409ee80 con 0x7f77841344a0 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77840a0900 con 0x7f77841348a0 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77840a0060 con 0x7f7784071860 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f778b840700 10 monclient(hunting): authenticate will time out at 1623.879262s 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 0x7f77840a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 0x7f77840a2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44376/0 (socket says 172.21.15.47:44376) 2024-04-20T02:15:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 -- 172.21.15.47:0/3477963506 learned_addr learned my addr 172.21.15.47:0/3477963506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): get_auth_request con 0x7f77841348a0 auth_method 0 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f77895dc700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7784071860 0x7f7784071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f77895dc700 10 monclient(hunting): get_auth_request con 0x7f7784071860 auth_method 0 2024-04-20T02:15:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f77895dc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f77895dc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f77895dc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7788ddb700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841344a0 0x7f77840a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7788ddb700 10 monclient(hunting): get_auth_request con 0x7f77841344a0 auth_method 0 2024-04-20T02:15:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7788ddb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7788ddb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7788ddb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient(hunting): handle_auth_done global_id 4216 payload 274 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 -- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7784071860 msgr2=0x7f7784071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7784071860 0x7f7784071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 -- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841344a0 msgr2=0x7f77840a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841344a0 0x7f77840a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 monclient: found mon.noname-c 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 -- 172.21.15.47:0/3477963506 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840751e0 con 0x7f77841348a0 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f77895dc700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7784071860 0x7f7784071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.478+0000 7f7789ddd700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 0x7f77840a2660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f777c009e80 tx=0x7f777c00b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9f8f112355228df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 1 -- 172.21.15.47:0/3477963506 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f777c00e070 con 0x7f77841348a0 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: dump: 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: _finish_auth 0 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 1 -- 172.21.15.47:0/3477963506 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0099b0 con 0x7f77841348a0 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 5 monclient: authenticate success, global_id 4216 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 4 set_mon_vals no callback set 2024-04-20T02:15:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 1 -- 172.21.15.47:0/3477963506 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f777c00be30 con 0x7f77841348a0 2024-04-20T02:15:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f777bfff700 10 monclient: dump: 2024-04-20T02:15:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 10 monclient: shutdown 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 -- 172.21.15.47:0/3477963506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 msgr2=0x7f77840a2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 0x7f77840a2660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f777c009e80 tx=0x7f777c00b330 comp rx=0 tx=0).stop 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 -- 172.21.15.47:0/3477963506 shutdown_connections 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841348a0 0x7f77840a2660 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7784071860 0x7f7784071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 --2- 172.21.15.47:0/3477963506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841344a0 0x7f77840a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.479+0000 7f778b840700 1 -- 172.21.15.47:0/3477963506 >> 172.21.15.47:0/3477963506 conn(0x7f77840a6d40 msgr2=0x7f77840a3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.480+0000 7f778b840700 1 -- 172.21.15.47:0/3477963506 shutdown_connections 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.480+0000 7f778b840700 1 -- 172.21.15.47:0/3477963506 wait complete. 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.480+0000 7f778b840700 10 monclient: build_initial_monmap 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.480+0000 7f778b840700 10 monclient: monmap: 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:44.504 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.480+0000 7f778b840700 1 librados: starting msgr at 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.480+0000 7f778b840700 1 librados: starting objecter 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 Processor -- start 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 -- start start 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 librados: setting wanted keys 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 librados: calling monclient init 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient: init 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient: _add_conns ranks=[6,4,3,7,2,1,5,8,0] 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 0x7f77841ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): picked mon.noname-d con 0x7f7784071860 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:44.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 0x7f7784131570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): picked mon.noname-c con 0x7f77841344a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841348a0 0x7f7784131ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): picked mon.noname-g con 0x7f77841348a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77841cd0f0 con 0x7f77841348a0 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77841383c0 con 0x7f77841344a0 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f778411e460 con 0x7f7784071860 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f778b840700 10 monclient(hunting): authenticate will time out at 1623.882565s 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 0x7f77841ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 0x7f7784131570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 0x7f77841ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52506/0 (socket says 172.21.15.47:52506) 2024-04-20T02:15:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 0x7f7784131570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44378/0 (socket says 172.21.15.47:44378) 2024-04-20T02:15:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 1 -- 172.21.15.47:0/2576159074 learned_addr learned my addr 172.21.15.47:0/2576159074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): get_auth_request con 0x7f7784071860 auth_method 0 2024-04-20T02:15:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): get_auth_request con 0x7f77841344a0 auth_method 0 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841348a0 0x7f7784131ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): get_auth_request con 0x7f77841348a0 auth_method 0 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f77895dc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7788ddb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.481+0000 7f7789ddd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 10 monclient(hunting): handle_auth_done global_id 4225 payload 824 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 -- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 msgr2=0x7f77841ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 0x7f77841ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 -- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841348a0 msgr2=0x7f7784131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841348a0 0x7f7784131ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:44.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 10 monclient: _finish_hunting 0 2024-04-20T02:15:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 monclient: found mon.noname-c 2024-04-20T02:15:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 -- 172.21.15.47:0/2576159074 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c009b30 con 0x7f77841344a0 2024-04-20T02:15:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f77895dc700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 0x7f77841ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:44.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7788ddb700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 0x7f7784131570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f777400e7e0 tx=0x7f777400eaf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 client.?.objecter ms_handle_connect 0x7f77841344a0 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f777400c070 con 0x7f77841344a0 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: dump: 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.511 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774013880 con 0x7f77841344a0 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 5 monclient: authenticate success, global_id 4225 2024-04-20T02:15:44.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 10 monclient: _renew_subs 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778411e740 con 0x7f77841344a0 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7774013e60 con 0x7f77841344a0 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f7779ffb700 10 monclient: dump: 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:44.513 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f777b7fe700 4 set_mon_vals no callback set 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 10 client.4225.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 10 monclient: _renew_subs 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778411ed00 con 0x7f77841344a0 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.482+0000 7f778b840700 1 librados: init done 2024-04-20T02:15:44.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f77740139f0 con 0x7f77841344a0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f776803a260 0x7f776803c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3607+0+0 (secure 0 0 0) 0x7f7774048ed0 con 0x7f77841344a0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 10 client.4225.objecter ms_dispatch 0x7f7784001550 osd_map(13..13 src has 1..13) v4 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 3 client.4225.objecter handle_osd_map got epochs [13,13] > 0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 3 client.4225.objecter handle_osd_map decoding full epoch 13 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f7779ffb700 20 client.4225.objecter dump_active .. 0 homeless 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f778b840700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f778b840700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f778b840700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778406aac0 con 0x7f77841344a0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f77895dc700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f776803a260 0x7f776803c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.483+0000 7f77895dc700 10 monclient: get_auth_request con 0x7f776803a260 auth_method 0 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.484+0000 7f7779ffb700 10 client.4225.objecter ms_handle_connect 0x7f776803a260 2024-04-20T02:15:44.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.484+0000 7f77895dc700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f776803a260 0x7f776803c720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7780004040 tx=0x7f7780008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.489+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7774013ca0 con 0x7f77841344a0 2024-04-20T02:15:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.489+0000 7f7779ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:44.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.489+0000 7f7779ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:44.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.707+0000 7f778b840700 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-04-20T02:15:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.707+0000 7f778b840700 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-04-20T02:15:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.707+0000 7f778b840700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:44.707+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f778406ace0 con 0x7f77841344a0 2024-04-20T02:15:45.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.237+0000 7f7779ffb700 1 -- 172.21.15.47:0/2576159074 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f77740112f0 con 0x7f77841344a0 2024-04-20T02:15:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.237+0000 7f7779ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-04-20T02:15:45.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.237+0000 7f7779ffb700 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 10 librados: watch_flush enter 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 10 librados: watch_flush exit 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 20 client.4225.objecter shutdown clearing up homeless session... 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 10 client.4225.objecter successfully canceled tick 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f776803a260 msgr2=0x7f776803c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f776803a260 0x7f776803c720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7780004040 tx=0x7f7780008040 comp rx=0 tx=0).stop 2024-04-20T02:15:45.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 10 monclient: shutdown 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 msgr2=0x7f7784131570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 0x7f7784131570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f777400e7e0 tx=0x7f777400eaf0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 shutdown_connections 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f776803a260 0x7f776803c720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f77841348a0 0x7f7784131ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7784071860 0x7f77841ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 --2- 172.21.15.47:0/2576159074 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77841344a0 0x7f7784131570 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 >> 172.21.15.47:0/2576159074 conn(0x7f77840a6d40 msgr2=0x7f77840a11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 shutdown_connections 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.242+0000 7f778b840700 1 -- 172.21.15.47:0/2576159074 wait complete. 2024-04-20T02:15:45.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.243+0000 7f778b840700 1 librados: shutdown 2024-04-20T02:15:45.244 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'rbd' created 2024-04-20T02:15:45.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.398+0000 7f8804334700 10 monclient: get_monmap_and_config 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.398+0000 7f8804334700 10 monclient: build_initial_monmap 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.398+0000 7f8804334700 10 monclient: monmap: 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:45.399983+0000 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:45.399983+0000 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:45.399 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.399+0000 7f8804334700 1 Processor -- start 2024-04-20T02:15:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.399+0000 7f8804334700 1 -- start start 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.399+0000 7f8804334700 10 monclient: init 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient: _add_conns ranks=[3,7,4,1,8,2,6,0,5,9,10] 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87fc071b50 0x7f87fc071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): picked mon.noname-g con 0x7f87fc071b50 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87fc072a70 0x7f87fc072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): picked mon.noname-i con 0x7f87fc072a70 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 0x7f87fc06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): picked mon.noname-c con 0x7f87fc137fe0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0a4090 con 0x7f87fc071b50 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc06e610 con 0x7f87fc137fe0 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc06e980 con 0x7f87fc072a70 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f8804334700 10 monclient(hunting): authenticate will time out at 1624.801277s 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 0x7f87fc06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 0x7f87fc06c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44394/0 (socket says 172.21.15.47:44394) 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 -- 172.21.15.47:0/402083422 learned_addr learned my addr 172.21.15.47:0/402083422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): get_auth_request con 0x7f87fc137fe0 auth_method 0 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88018cf700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87fc072a70 0x7f87fc072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88020d0700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87fc071b50 0x7f87fc071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88020d0700 10 monclient(hunting): get_auth_request con 0x7f87fc071b50 auth_method 0 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88020d0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88020d0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88020d0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88018cf700 10 monclient(hunting): get_auth_request con 0x7f87fc072a70 auth_method 0 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88018cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88018cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88018cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient(hunting): handle_auth_done global_id 4234 payload 274 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 -- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87fc072a70 msgr2=0x7f87fc072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87fc072a70 0x7f87fc072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 -- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87fc071b50 msgr2=0x7f87fc071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87fc071b50 0x7f87fc071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient: _finish_hunting 0 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 monclient: found mon.noname-c 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.400+0000 7f88028d1700 1 -- 172.21.15.47:0/402083422 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0751e0 con 0x7f87fc137fe0 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88028d1700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 0x7f87fc06c7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009e80 tx=0x7f87f400b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2caf264d0b3b2a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 1 -- 172.21.15.47:0/402083422 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87f400e070 con 0x7f87fc137fe0 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: dump: 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:45.413 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:45.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: _finish_auth 0 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: _check_auth_tickets 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 5 monclient: authenticate success, global_id 4234 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 1 -- 172.21.15.47:0/402083422 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f400be30 con 0x7f87fc137fe0 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 4 set_mon_vals no callback set 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 1 -- 172.21.15.47:0/402083422 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87f4012860 con 0x7f87fc137fe0 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f88010ce700 10 monclient: dump: 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:45.415 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 10 monclient: shutdown 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:45.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 -- 172.21.15.47:0/402083422 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 msgr2=0x7f87fc06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 0x7f87fc06c7a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009e80 tx=0x7f87f400b330 comp rx=0 tx=0).stop 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 -- 172.21.15.47:0/402083422 shutdown_connections 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87fc137fe0 0x7f87fc06c7a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87fc071b50 0x7f87fc071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 --2- 172.21.15.47:0/402083422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87fc072a70 0x7f87fc072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.401+0000 7f8804334700 1 -- 172.21.15.47:0/402083422 >> 172.21.15.47:0/402083422 conn(0x7f87fc066610 msgr2=0x7f87fc066a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.402+0000 7f8804334700 1 -- 172.21.15.47:0/402083422 shutdown_connections 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.402+0000 7f8804334700 1 -- 172.21.15.47:0/402083422 wait complete. 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.402+0000 7f8804334700 10 monclient: build_initial_monmap 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.402+0000 7f8804334700 10 monclient: monmap: 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:45.417 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 librados: starting msgr at 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 librados: starting objecter 2024-04-20T02:15:45.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 Processor -- start 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 -- start start 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 librados: setting wanted keys 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 librados: calling monclient init 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient: init 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient: _add_conns ranks=[0,8,6,1,2,3,4,5,7] 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 0x7f87fc134c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): picked mon.noname-a con 0x7f87fc071b50 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f87fc072a70 0x7f87fc135190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): picked mon.noname-e con 0x7f87fc072a70 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 0x7f87fc11e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): picked mon.noname-d con 0x7f87fc137fe0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc121d70 con 0x7f87fc071b50 2024-04-20T02:15:45.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1cce80 con 0x7f87fc137fe0 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc06d8b0 con 0x7f87fc072a70 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f8804334700 10 monclient(hunting): authenticate will time out at 1624.804800s 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 0x7f87fc134c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88028d1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 0x7f87fc11e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88018cf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f87fc072a70 0x7f87fc135190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 0x7f87fc134c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56830/0 (socket says 172.21.15.47:56830) 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 1 -- 172.21.15.47:0/3479995226 learned_addr learned my addr 172.21.15.47:0/3479995226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88028d1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 0x7f87fc11e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52522/0 (socket says 172.21.15.47:52522) 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 10 monclient(hunting): get_auth_request con 0x7f87fc071b50 auth_method 0 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:45.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88020d0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88018cf700 10 monclient(hunting): get_auth_request con 0x7f87fc072a70 auth_method 0 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88018cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88018cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.403+0000 7f88018cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): get_auth_request con 0x7f87fc137fe0 auth_method 0 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88018cf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88018cf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88018cf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:45.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 -- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f87fc072a70 msgr2=0x7f87fc135190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f87fc072a70 0x7f87fc135190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 -- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 msgr2=0x7f87fc11e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 0x7f87fc11e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 10 monclient: _finish_hunting 0 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 monclient: found mon.noname-a 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 -- 172.21.15.47:0/3479995226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009b30 con 0x7f87fc071b50 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88018cf700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f87fc072a70 0x7f87fc135190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88028d1700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 0x7f87fc11e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:45.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88020d0700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 0x7f87fc134c50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f87f800e810 tx=0x7f87f800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 client.?.objecter ms_handle_connect 0x7f87fc071b50 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87f800c070 con 0x7f87fc071b50 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: dump: 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:45.423 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8013880 con 0x7f87fc071b50 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87f8013e60 con 0x7f87fc071b50 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 5 monclient: authenticate success, global_id 4257 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 10 monclient: _renew_subs 2024-04-20T02:15:45.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc122610 con 0x7f87fc071b50 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f88008cd700 4 set_mon_vals no callback set 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f87f2ffd700 10 monclient: dump: 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:45.425 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 10 client.4257.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 10 monclient: _renew_subs 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc122330 con 0x7f87fc071b50 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.404+0000 7f8804334700 1 librados: init done 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f87f80139f0 con 0x7f87fc071b50 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f8804334700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:45.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f8804334700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f8804334700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc067f90 con 0x7f87fc071b50 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f87f2ffd700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87e003a290 0x7f87e003c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4324+0+0 (secure 0 0 0) 0x7f87f8049b30 con 0x7f87fc071b50 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f87f2ffd700 10 client.4257.objecter ms_dispatch 0x7f87fc001550 osd_map(14..14 src has 1..14) v4 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f87f2ffd700 3 client.4257.objecter handle_osd_map got epochs [14,14] > 0 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.405+0000 7f87f2ffd700 3 client.4257.objecter handle_osd_map decoding full epoch 14 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.406+0000 7f87f2ffd700 20 client.4257.objecter dump_active .. 0 homeless 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.406+0000 7f88018cf700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87e003a290 0x7f87e003c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.406+0000 7f88018cf700 10 monclient: get_auth_request con 0x7f87e003a290 auth_method 0 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.406+0000 7f87f2ffd700 10 client.4257.objecter ms_handle_connect 0x7f87e003a290 2024-04-20T02:15:45.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.406+0000 7f88018cf700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87e003a290 0x7f87e003c750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0a2490 tx=0x7f87ec008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:45.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.411+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f87f8013ca0 con 0x7f87fc071b50 2024-04-20T02:15:45.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.411+0000 7f87f2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:45.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.411+0000 7f87f2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.630+0000 7f8804334700 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-04-20T02:15:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.630+0000 7f8804334700 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-04-20T02:15:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.630+0000 7f8804334700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:45.630+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f87fc06aac0 con 0x7f87fc071b50 2024-04-20T02:15:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.237+0000 7f87f2ffd700 1 -- 172.21.15.47:0/3479995226 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f87f8017070 con 0x7f87fc071b50 2024-04-20T02:15:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.237+0000 7f87f2ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-04-20T02:15:46.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.237+0000 7f87f2ffd700 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2024-04-20T02:15:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.242+0000 7f8804334700 10 librados: watch_flush enter 2024-04-20T02:15:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.242+0000 7f8804334700 10 librados: watch_flush exit 2024-04-20T02:15:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.242+0000 7f8804334700 20 client.4257.objecter shutdown clearing up homeless session... 2024-04-20T02:15:46.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.242+0000 7f8804334700 10 client.4257.objecter successfully canceled tick 2024-04-20T02:15:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87e003a290 msgr2=0x7f87e003c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87e003a290 0x7f87e003c750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0a2490 tx=0x7f87ec008040 comp rx=0 tx=0).stop 2024-04-20T02:15:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 10 monclient: shutdown 2024-04-20T02:15:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 msgr2=0x7f87fc134c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 0x7f87fc134c50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f87f800e810 tx=0x7f87f800eb20 comp rx=0 tx=0).stop 2024-04-20T02:15:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 shutdown_connections 2024-04-20T02:15:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87e003a290 0x7f87e003c750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87fc137fe0 0x7f87fc11e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87fc071b50 0x7f87fc134c50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 --2- 172.21.15.47:0/3479995226 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f87fc072a70 0x7f87fc135190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.244+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 >> 172.21.15.47:0/3479995226 conn(0x7f87fc066610 msgr2=0x7f87fc0a6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:46.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.246+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 shutdown_connections 2024-04-20T02:15:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.248+0000 7f8804334700 1 -- 172.21.15.47:0/3479995226 wait complete. 2024-04-20T02:15:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.248+0000 7f8804334700 1 librados: shutdown 2024-04-20T02:15:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-20T02:15:46.264 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T02:15:46.264 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T02:15:46.265 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T02:15:46.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T02:15:46.265 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T02:15:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.410+0000 7f8d99b25700 10 monclient: get_monmap_and_config 2024-04-20T02:15:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.410+0000 7f8d99b25700 10 monclient: build_initial_monmap 2024-04-20T02:15:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.410+0000 7f8d99b25700 10 monclient: monmap: 2024-04-20T02:15:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:46.411371+0000 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:46.411371+0000 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:46.411 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 Processor -- start 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 -- start start 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient: init 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient: _add_conns ranks=[5,7,10,9,8,1,0,4,3,2,6] 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d940a0950 0x7f8d940a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): picked mon.noname-h con 0x7f8d940a0950 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 0x7f8d940a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): picked mon.noname-i con 0x7f8d940a0160 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d940a2e90 0x7f8d940a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): picked mon.noname-f con 0x7f8d940a2e90 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d94072830 con 0x7f8d940a2e90 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9409e650 con 0x7f8d940a0950 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9409eee0 con 0x7f8d940a0160 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d99b25700 10 monclient(hunting): authenticate will time out at 1625.812665s 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d937fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d940a0950 0x7f8d940a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d93fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d940a2e90 0x7f8d940a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 0x7f8d940a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 0x7f8d940a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:54982/0 (socket says 172.21.15.47:54982) 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d937fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d940a0950 0x7f8d940a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:42350/0 (socket says 172.21.15.47:42350) 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 1 -- 172.21.15.47:0/2149369813 learned_addr learned my addr 172.21.15.47:0/2149369813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:46.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 10 monclient(hunting): get_auth_request con 0x7f8d940a0160 auth_method 0 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d92ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d93fff700 10 monclient(hunting): get_auth_request con 0x7f8d940a2e90 auth_method 0 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d93fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d93fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d93fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d937fe700 10 monclient(hunting): get_auth_request con 0x7f8d940a0950 auth_method 0 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d937fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d937fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.411+0000 7f8d937fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d93fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d93fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d93fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d937fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d937fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d937fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_done global_id 4237 payload 274 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 -- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d940a0950 msgr2=0x7f8d940a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 --2- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d940a0950 0x7f8d940a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 -- 172.21.15.47:0/2149369813 >> v2:172.21.15.179:3300/0 conn(0x7f8d940a2e90 msgr2=0x7f8d940a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 --2- 172.21.15.47:0/2149369813 >> v2:172.21.15.179:3300/0 conn(0x7f8d940a2e90 0x7f8d940a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 monclient: found mon.noname-i 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 -- 172.21.15.47:0/2149369813 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d940751e0 con 0x7f8d940a0160 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d93fff700 1 --2- 172.21.15.47:0/2149369813 >> v2:172.21.15.179:3300/0 conn(0x7f8d940a2e90 0x7f8d940a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d92ffd700 1 --2- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 0x7f8d940a2950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009920 tx=0x7f8d88009c30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=8739e645b79476d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 1 -- 172.21.15.47:0/2149369813 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d88011aa0 con 0x7f8d940a0160 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: dump: 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.426 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d99b25700 5 monclient: authenticate success, global_id 4237 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d99b25700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:46.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 1 -- 172.21.15.47:0/2149369813 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88011c00 con 0x7f8d940a0160 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d99b25700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d99b25700 4 set_mon_vals no callback set 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 1 -- 172.21.15.47:0/2149369813 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d88011d70 con 0x7f8d940a0160 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.412+0000 7f8d927fc700 10 monclient: dump: 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.428 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 10 monclient: shutdown 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 -- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 msgr2=0x7f8d940a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 --2- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 0x7f8d940a2950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009920 tx=0x7f8d88009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 -- 172.21.15.47:0/2149369813 shutdown_connections 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 --2- 172.21.15.47:0/2149369813 >> v2:172.21.15.179:3300/0 conn(0x7f8d940a2e90 0x7f8d940a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 --2- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d940a0950 0x7f8d940a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 --2- 172.21.15.47:0/2149369813 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8d940a0160 0x7f8d940a2950 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 -- 172.21.15.47:0/2149369813 >> 172.21.15.47:0/2149369813 conn(0x7f8d940a7570 msgr2=0x7f8d940a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 -- 172.21.15.47:0/2149369813 shutdown_connections 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.413+0000 7f8d99b25700 1 -- 172.21.15.47:0/2149369813 wait complete. 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 10 monclient: build_initial_monmap 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 10 monclient: monmap: 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 1 librados: starting msgr at 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 1 librados: starting objecter 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 1 Processor -- start 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 1 -- start start 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 1 librados: setting wanted keys 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 1 librados: calling monclient init 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.414+0000 7f8d99b25700 10 monclient: init 2024-04-20T02:15:46.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient: _add_conns ranks=[1,6,8,4,5,2,7,0,3] 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d940a0160 0x7f8d941bab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): picked mon.noname-f con 0x7f8d940a0160 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 0x7f8d941a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): picked mon.noname-d con 0x7f8d940a0950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8d940a2e90 0x7f8d941a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): picked mon.noname-e con 0x7f8d940a2e90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d941ccee0 con 0x7f8d940a0160 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8d941a9b00 con 0x7f8d940a0950 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9409e440 con 0x7f8d940a2e90 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:46.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d99b25700 10 monclient(hunting): authenticate will time out at 1625.816256s 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 0x7f8d941a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 0x7f8d941a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52536/0 (socket says 172.21.15.47:52536) 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 -- 172.21.15.47:0/3343487456 learned_addr learned my addr 172.21.15.47:0/3343487456 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): get_auth_request con 0x7f8d940a0950 auth_method 0 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8d940a2e90 0x7f8d941a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): get_auth_request con 0x7f8d940a2e90 auth_method 0 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d937fe700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d940a0160 0x7f8d941bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d937fe700 10 monclient(hunting): get_auth_request con 0x7f8d940a0160 auth_method 0 2024-04-20T02:15:46.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d937fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d937fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d937fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient(hunting): handle_auth_done global_id 4218 payload 824 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 -- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8d940a2e90 msgr2=0x7f8d941a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8d940a2e90 0x7f8d941a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 -- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d940a0160 msgr2=0x7f8d941bab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d940a0160 0x7f8d941bab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 monclient: found mon.noname-d 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 -- 172.21.15.47:0/3343487456 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c009a70 con 0x7f8d940a0950 2024-04-20T02:15:46.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d937fe700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d940a0160 0x7f8d941bab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d93fff700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8d940a2e90 0x7f8d941a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d92ffd700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 0x7f8d941a66b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d88013000 tx=0x7f8d88004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d7bfff700 10 client.?.objecter ms_handle_connect 0x7f8d940a0950 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.415+0000 7f8d7bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d88004480 con 0x7f8d940a0950 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: dump: 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.436 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: _finish_auth 0 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88003b10 con 0x7f8d940a0950 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 5 monclient: authenticate success, global_id 4218 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 10 monclient: _renew_subs 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:46.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d88009550 con 0x7f8d940a0950 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d88003c80 con 0x7f8d940a0950 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d7bfff700 10 monclient: dump: 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.438 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 10 client.4218.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 10 monclient: _renew_subs 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d91ffb700 4 set_mon_vals no callback set 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d941a9820 con 0x7f8d940a0950 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.416+0000 7f8d99b25700 1 librados: init done 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f8d88021070 con 0x7f8d940a0950 2024-04-20T02:15:46.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d99b25700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d99b25700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d99b25700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d941a9cb0 con 0x7f8d940a0950 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8003a260 0x7f8d8003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8d8804b260 con 0x7f8d940a0950 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 10 client.4218.objecter ms_dispatch 0x7f8d94001550 osd_map(15..15 src has 1..15) v4 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 3 client.4218.objecter handle_osd_map got epochs [15,15] > 0 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 3 client.4218.objecter handle_osd_map decoding full epoch 15 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d7bfff700 20 client.4218.objecter dump_active .. 0 homeless 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d937fe700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8003a260 0x7f8d8003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.417+0000 7f8d937fe700 10 monclient: get_auth_request con 0x7f8d8003a260 auth_method 0 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.418+0000 7f8d7bfff700 10 client.4218.objecter ms_handle_connect 0x7f8d8003a260 2024-04-20T02:15:46.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.418+0000 7f8d937fe700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8003a260 0x7f8d8003c720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009c30 tx=0x7f8d84008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:46.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.422+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f8d941a9cb0 con 0x7f8d940a0950 2024-04-20T02:15:46.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.422+0000 7f8d7bfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:46.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.422+0000 7f8d7bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.709+0000 7f8d99b25700 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-04-20T02:15:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.709+0000 7f8d99b25700 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-04-20T02:15:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.709+0000 7f8d99b25700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.709+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8d941a9cb0 con 0x7f8d940a0950 2024-04-20T02:15:46.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.711+0000 7f8d7bfff700 1 -- 172.21.15.47:0/3343487456 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+86038 (secure 0 0 0) 0x7f8d941a9cb0 con 0x7f8d940a0950 2024-04-20T02:15:46.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.711+0000 7f8d7bfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-04-20T02:15:46.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.711+0000 7f8d7bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:46.712 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 10 librados: watch_flush enter 2024-04-20T02:15:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 10 librados: watch_flush exit 2024-04-20T02:15:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 20 client.4218.objecter shutdown clearing up homeless session... 2024-04-20T02:15:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 10 client.4218.objecter successfully canceled tick 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8003a260 msgr2=0x7f8d8003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8003a260 0x7f8d8003c720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009c30 tx=0x7f8d84008040 comp rx=0 tx=0).stop 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 10 monclient: shutdown 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 msgr2=0x7f8d941a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 0x7f8d941a66b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8d88013000 tx=0x7f8d88004750 comp rx=0 tx=0).stop 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 shutdown_connections 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8d940a2e90 0x7f8d941a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8003a260 0x7f8d8003c720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d940a0160 0x7f8d941bab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 --2- 172.21.15.47:0/3343487456 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8d940a0950 0x7f8d941a66b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.713+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 >> 172.21.15.47:0/3343487456 conn(0x7f8d940a7570 msgr2=0x7f8d940a57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.714+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 shutdown_connections 2024-04-20T02:15:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.714+0000 7f8d99b25700 1 -- 172.21.15.47:0/3343487456 wait complete. 2024-04-20T02:15:46.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.714+0000 7f8d99b25700 1 librados: shutdown 2024-04-20T02:15:46.728 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":4,"flags":0,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":41706},{"type":"v1","addr":"172.21.15.179:6825","nonce":41706}]},"active_addr":"172.21.15.179:6825/41706","active_change":"2024-04-20T02:15:40.278169+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.179:0","nonce":1290484782}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.179:0","nonce":650233903}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.179:0","nonce":2232160297}]}]} 2024-04-20T02:15:46.732 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T02:15:46.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T02:15:46.732 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T02:15:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.873+0000 7f9ae299f700 10 monclient: get_monmap_and_config 2024-04-20T02:15:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.873+0000 7f9ae299f700 10 monclient: build_initial_monmap 2024-04-20T02:15:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.873+0000 7f9ae299f700 10 monclient: monmap: 2024-04-20T02:15:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.872 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:46.874296+0000 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:46.874296+0000 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:46.873 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.873+0000 7f9ae299f700 1 Processor -- start 2024-04-20T02:15:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 1 -- start start 2024-04-20T02:15:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient: init 2024-04-20T02:15:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient: _add_conns ranks=[5,9,0,2,7,1,3,6,8,10,4] 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9adc09df00 0x7f9adc099890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): picked mon.noname-h con 0x7f9adc09df00 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9adc0a1120 0x7f9adc0a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): picked mon.noname-a con 0x7f9adc0a1120 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9adc0a1a40 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0751e0 con 0x7f9adc0a1a40 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc09ea50 con 0x7f9adc0a1120 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc09ed30 con 0x7f9adc09df00 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae299f700 10 monclient(hunting): authenticate will time out at 1626.275625s 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9adc0a1120 0x7f9adc0a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae0f3c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9adc0a1a40 0x7f9adc13e6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54100/0 (socket says 172.21.15.47:54100) 2024-04-20T02:15:46.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9ae0f3c700 1 -- 172.21.15.47:0/130156624 learned_addr learned my addr 172.21.15.47:0/130156624 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): get_auth_request con 0x7f9adc0a1120 auth_method 0 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 1 --2- 172.21.15.47:0/130156624 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9adc09df00 0x7f9adc099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): get_auth_request con 0x7f9adc09df00 auth_method 0 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.874+0000 7f9adbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_done global_id 4266 payload 274 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 1 -- 172.21.15.47:0/130156624 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9adc09df00 msgr2=0x7f9adc099890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 1 --2- 172.21.15.47:0/130156624 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9adc09df00 0x7f9adc099890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 1 -- 172.21.15.47:0/130156624 >> v1:172.21.15.47:6789/0 conn(0x7f9adc0a1a40 legacy=0x7f9adc13e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 1 monclient: found mon.noname-a 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 1 -- 172.21.15.47:0/130156624 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc09f010 con 0x7f9adc0a1120 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 1 -- 172.21.15.47:0/130156624 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219741304 0 0) 0x7f9adc0751e0 con 0x7f9adc0a1a40 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adb7fe700 1 --2- 172.21.15.47:0/130156624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc0a1500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009840 tx=0x7f9ad0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fc38b575ee00aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 1 -- 172.21.15.47:0/130156624 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ad0011aa0 con 0x7f9adc0a1120 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: dump: 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.886 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 1 -- 172.21.15.47:0/130156624 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0011c00 con 0x7f9adc0a1120 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9ae299f700 5 monclient: authenticate success, global_id 4266 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9ae299f700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 1 -- 172.21.15.47:0/130156624 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ad0011d70 con 0x7f9adc0a1120 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9ae299f700 4 set_mon_vals no callback set 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.875+0000 7f9adaffd700 10 monclient: dump: 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.887 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 10 monclient: shutdown 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:46.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 -- 172.21.15.47:0/130156624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 msgr2=0x7f9adc0a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 --2- 172.21.15.47:0/130156624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc0a1500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009840 tx=0x7f9ad0009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 -- 172.21.15.47:0/130156624 shutdown_connections 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 --2- 172.21.15.47:0/130156624 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9adc09df00 0x7f9adc099890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 --2- 172.21.15.47:0/130156624 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc0a1500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 -- 172.21.15.47:0/130156624 >> 172.21.15.47:0/130156624 conn(0x7f9adc0a6720 msgr2=0x7f9adc0a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 -- 172.21.15.47:0/130156624 shutdown_connections 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 1 -- 172.21.15.47:0/130156624 wait complete. 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 10 monclient: build_initial_monmap 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.876+0000 7f9ae299f700 10 monclient: monmap: 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:46.889 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 librados: starting msgr at 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 librados: starting objecter 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 Processor -- start 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 -- start start 2024-04-20T02:15:46.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 librados: setting wanted keys 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 librados: calling monclient init 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient: init 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient: _add_conns ranks=[2,0,1,7,5,6,4,3,8] 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 0x7f9adc071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): picked mon.noname-b con 0x7f9adc09df00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): picked mon.noname-a con 0x7f9adc0a1120 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9adc0a1a40 0x7f9adc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): picked mon.noname-f con 0x7f9adc0a1a40 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc098310 con 0x7f9adc0a1120 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc091d80 con 0x7f9adc0a1a40 2024-04-20T02:15:46.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0a0410 con 0x7f9adc09df00 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.877+0000 7f9ae299f700 10 monclient(hunting): authenticate will time out at 1626.278921s 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 0x7f9adc071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56858/0 (socket says 172.21.15.47:56858) 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 0x7f9adc071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52978/0 (socket says 172.21.15.47:52978) 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 1 -- 172.21.15.47:0/552514423 learned_addr learned my addr 172.21.15.47:0/552514423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): get_auth_request con 0x7f9adc0a1120 auth_method 0 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): get_auth_request con 0x7f9adc09df00 auth_method 0 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9adc0a1a40 0x7f9adc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): get_auth_request con 0x7f9adc0a1a40 auth_method 0 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:46.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae0f3c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient(hunting): handle_auth_done global_id 4214 payload 824 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 -- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9adc0a1a40 msgr2=0x7f9adc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9adc0a1a40 0x7f9adc0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 -- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 msgr2=0x7f9adc071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 monclient: found mon.noname-b 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 -- 172.21.15.47:0/552514423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00094a0 con 0x7f9adc09df00 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adb7fe700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9adbfff700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 0x7f9adc071740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e930 tx=0x7f9acc00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 client.?.objecter ms_handle_connect 0x7f9adc09df00 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:46.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9acc013af0 con 0x7f9adc09df00 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: dump: 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.895 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 5 monclient: authenticate success, global_id 4214 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 10 monclient: _renew_subs 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc013c70 con 0x7f9adc09df00 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc0902d0 con 0x7f9adc09df00 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9acc013de0 con 0x7f9adc09df00 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 10 client.4214.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:46.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ada7fc700 4 set_mon_vals no callback set 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ad8ff9700 10 monclient: dump: 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:46.897 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 10 monclient: _renew_subs 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.878+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc092340 con 0x7f9adc09df00 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ae299f700 1 librados: init done 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f9acc013680 con 0x7f9adc09df00 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ad8ff9700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ac803a260 0x7f9ac803c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f9acc049c40 con 0x7f9adc09df00 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ad8ff9700 10 client.4214.objecter ms_dispatch 0x7f9adc001550 osd_map(15..15 src has 1..15) v4 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ad8ff9700 3 client.4214.objecter handle_osd_map got epochs [15,15] > 0 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.879+0000 7f9ad8ff9700 3 client.4214.objecter handle_osd_map decoding full epoch 15 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9ae299f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:46.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9ae299f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9ae299f700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc091fc0 con 0x7f9adc09df00 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9ad8ff9700 20 client.4214.objecter dump_active .. 0 homeless 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9adb7fe700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ac803a260 0x7f9ac803c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9adb7fe700 10 monclient: get_auth_request con 0x7f9ac803a260 auth_method 0 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9ad8ff9700 10 client.4214.objecter ms_handle_connect 0x7f9ac803a260 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.880+0000 7f9adb7fe700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ac803a260 0x7f9ac803c720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009840 tx=0x7f9ad0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.885+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f9acc017080 con 0x7f9adc09df00 2024-04-20T02:15:46.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.885+0000 7f9ad8ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:46.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:46.885+0000 7f9ad8ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.108+0000 7f9ae299f700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.109+0000 7f9ae299f700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.109+0000 7f9ae299f700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.109+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9adc09d110 con 0x7f9adc09df00 2024-04-20T02:15:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.109+0000 7f9ad8ff9700 1 -- 172.21.15.47:0/552514423 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11858 (secure 0 0 0) 0x7f9acc01ba00 con 0x7f9adc09df00 2024-04-20T02:15:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.109+0000 7f9ad8ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.109+0000 7f9ad8ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:47.109 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:47.109 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":15,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:15:46.233317+0000","last_up_change":"2024-04-20T02:15:43.202903+0000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T02:15:44.345718+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T02:15:44.739933+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":42309},{"type":"v1","addr":"172.21.15.47:6801","nonce":42309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":42309},{"type":"v1","addr":"172.21.15.47:6803","nonce":42309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":42309},{"type":"v1","addr":"172.21.15.47:6811","nonce":42309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":42309},{"type":"v1","addr":"172.21.15.47:6806","nonce":42309}]},"public_addr":"172.21.15.47:6801/42309","cluster_addr":"172.21.15.47:6803/42309","heartbeat_back_addr":"172.21.15.47:6811/42309","heartbeat_front_addr":"172.21.15.47:6806/42309","state":["exists","up"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":42310},{"type":"v1","addr":"172.21.15.47:6807","nonce":42310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":42310},{"type":"v1","addr":"172.21.15.47:6813","nonce":42310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6819","nonce":42310},{"type":"v1","addr":"172.21.15.47:6821","nonce":42310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":42310},{"type":"v1","addr":"172.21.15.47:6817","nonce":42310}]},"public_addr":"172.21.15.47:6807/42310","cluster_addr":"172.21.15.47:6813/42310","heartbeat_back_addr":"172.21.15.47:6821/42310","heartbeat_front_addr":"172.21.15.47:6817/42310","state":["exists","up"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":42311},{"type":"v1","addr":"172.21.15.47:6812","nonce":42311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":42311},{"type":"v1","addr":"172.21.15.47:6816","nonce":42311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":42311},{"type":"v1","addr":"172.21.15.47:6823","nonce":42311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":42311},{"type":"v1","addr":"172.21.15.47:6820","nonce":42311}]},"public_addr":"172.21.15.47:6812/42311","cluster_addr":"172.21.15.47:6816/42311","heartbeat_back_addr":"172.21.15.47:6823/42311","heartbeat_front_addr":"172.21.15.47:6820/42311","state":["exists","up"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":42294},{"type":"v1","addr":"172.21.15.179:6801","nonce":42294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":42294},{"type":"v1","addr":"172.21.15.179:6803","nonce":42294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":42294},{"type":"v1","addr":"172.21.15.179:6807","nonce":42294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":42294},{"type":"v1","addr":"172.21.15.179:6805","nonce":42294}]},"public_addr":"172.21.15.179:6801/42294","cluster_addr":"172.21.15.179:6803/42294","heartbeat_back_addr":"172.21.15.179:6807/42294","heartbeat_front_addr":"172.21.15.179:6805/42294","state":["exists","up"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":42295},{"type":"v1","addr":"172.21.15.179:6812","nonce":42295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":42295},{"type":"v1","addr":"172.21.15.179:6816","nonce":42295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":42295},{"type":"v1","addr":"172.21.15.179:6823","nonce":42295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":42295},{"type":"v1","addr":"172.21.15.179:6820","nonce":42295}]},"public_addr":"172.21.15.179:6812/42295","cluster_addr":"172.21.15.179:6816/42295","heartbeat_back_addr":"172.21.15.179:6823/42295","heartbeat_front_addr":"172.21.15.179:6820/42295","state":["exists","up"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":42296},{"type":"v1","addr":"172.21.15.179:6809","nonce":42296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":42296},{"type":"v1","addr":"172.21.15.179:6813","nonce":42296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":42296},{"type":"v1","addr":"172.21.15.179:6821","nonce":42296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":42296},{"type":"v1","addr":"172.21.15.179:6817","nonce":42296}]},"public_addr":"172.21.15.179:6809/42296","cluster_addr":"172.21.15.179:6813/42296","heartbeat_back_addr":"172.21.15.179:6821/42296","heartbeat_front_addr":"172.21.15.179:6817/42296","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.384773+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.348973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.447042+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.151528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 10 librados: watch_flush enter 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 10 librados: watch_flush exit 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 20 client.4214.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 10 client.4214.objecter successfully canceled tick 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ac803a260 msgr2=0x7f9ac803c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ac803a260 0x7f9ac803c720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009840 tx=0x7f9ad0008030 comp rx=0 tx=0).stop 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 10 monclient: shutdown 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 msgr2=0x7f9adc071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 0x7f9adc071740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e930 tx=0x7f9acc00ec40 comp rx=0 tx=0).stop 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 shutdown_connections 2024-04-20T02:15:47.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ac803a260 0x7f9ac803c720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9adc0a1a40 0x7f9adc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc09df00 0x7f9adc071740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 --2- 172.21.15.47:0/552514423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc0a1120 0x7f9adc071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 >> 172.21.15.47:0/552514423 conn(0x7f9adc0a6720 msgr2=0x7f9adc1bf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 shutdown_connections 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 -- 172.21.15.47:0/552514423 wait complete. 2024-04-20T02:15:47.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.112+0000 7f9ae299f700 1 librados: shutdown 2024-04-20T02:15:47.125 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T02:15:47.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T02:15:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.270+0000 7f7c13932700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.270+0000 7f7c13932700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.271+0000 7f7c13932700 10 monclient: monmap: 2024-04-20T02:15:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.272163+0000 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.272163+0000 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.271 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.271+0000 7f7c13932700 1 Processor -- start 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.271+0000 7f7c13932700 1 -- start start 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.271+0000 7f7c13932700 10 monclient: init 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient: _add_conns ranks=[0,10,5,1,7,8,3,9,6,4,2] 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7c0c0716f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7c0c06f090 0x7f7c0c1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): picked mon.noname-f con 0x7f7c0c06f090 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7c0c139ef0 0x7f7c0c13c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): picked mon.noname-h con 0x7f7c0c139ef0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0c0751e0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c06f4f0 con 0x7f7c0c06f090 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c09e910 con 0x7f7c0c139ef0 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c13932700 10 monclient(hunting): authenticate will time out at 1626.673544s 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c116ce700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7c0c0716f0 0x7f7c0c071ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54116/0 (socket says 172.21.15.47:54116) 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c116ce700 1 -- 172.21.15.47:0/936936337 learned_addr learned my addr 172.21.15.47:0/936936337 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 1 --2- 172.21.15.47:0/936936337 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7c0c139ef0 0x7f7c0c13c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 1 --2- 172.21.15.47:0/936936337 >> v2:172.21.15.179:3300/0 conn(0x7f7c0c06f090 0x7f7c0c1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): get_auth_request con 0x7f7c0c139ef0 auth_method 0 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): get_auth_request con 0x7f7c0c06f090 auth_method 0 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c03fff700 1 -- 172.21.15.47:0/936936337 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443822095 0 0) 0x7f7c0c0751e0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c03fff700 10 monclient(hunting): _negotiate 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c03fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c03fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c03fff700 10 monclient(hunting): my global_id is 4302 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c03fff700 1 -- 172.21.15.47:0/936936337 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf80035b0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.272+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_done global_id 4262 payload 274 2024-04-20T02:15:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 1 -- 172.21.15.47:0/936936337 >> v2:172.21.15.179:3300/0 conn(0x7f7c0c06f090 msgr2=0x7f7c0c1399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 1 --2- 172.21.15.47:0/936936337 >> v2:172.21.15.179:3300/0 conn(0x7f7c0c06f090 0x7f7c0c1399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 1 -- 172.21.15.47:0/936936337 >> v1:172.21.15.47:6789/0 conn(0x7f7c0c0716f0 legacy=0x7f7c0c071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 1 monclient: found mon.noname-h 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 1 -- 172.21.15.47:0/936936337 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c09ed20 con 0x7f7c0c139ef0 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c10ecd700 1 --2- 172.21.15.47:0/936936337 >> v2:172.21.15.179:3300/0 conn(0x7f7c0c06f090 0x7f7c0c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c11ecf700 1 --2- 172.21.15.47:0/936936337 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7c0c139ef0 0x7f7c0c13c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009d90 tx=0x7f7c0400b330 comp rx=0 tx=0).ready entity=mon.5 client_cookie=acfcf1fa70f3e08b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 1 -- 172.21.15.47:0/936936337 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c0400e070 con 0x7f7c0c139ef0 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: dump: 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.285 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 1 -- 172.21.15.47:0/936936337 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c040098f0 con 0x7f7c0c139ef0 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c13932700 5 monclient: authenticate success, global_id 4262 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c13932700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c13932700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 1 -- 172.21.15.47:0/936936337 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c040128a0 con 0x7f7c0c139ef0 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c13932700 4 set_mon_vals no callback set 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.273+0000 7f7c03fff700 10 monclient: dump: 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.287 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 10 monclient: shutdown 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 -- 172.21.15.47:0/936936337 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7c0c139ef0 msgr2=0x7f7c0c13c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 --2- 172.21.15.47:0/936936337 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7c0c139ef0 0x7f7c0c13c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009d90 tx=0x7f7c0400b330 comp rx=0 tx=0).stop 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 -- 172.21.15.47:0/936936337 shutdown_connections 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 --2- 172.21.15.47:0/936936337 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7c0c139ef0 0x7f7c0c13c2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 --2- 172.21.15.47:0/936936337 >> v2:172.21.15.179:3300/0 conn(0x7f7c0c06f090 0x7f7c0c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 -- 172.21.15.47:0/936936337 >> 172.21.15.47:0/936936337 conn(0x7f7c0c0a7570 msgr2=0x7f7c0c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 -- 172.21.15.47:0/936936337 shutdown_connections 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 1 -- 172.21.15.47:0/936936337 wait complete. 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.274+0000 7f7c13932700 10 monclient: monmap: 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.289 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 librados: starting msgr at 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 librados: starting objecter 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 Processor -- start 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 -- start start 2024-04-20T02:15:47.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 librados: setting wanted keys 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 librados: calling monclient init 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient: init 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient: _add_conns ranks=[8,1,2,0,5,4,7,3,6] 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 0x7f7c0c09d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): picked mon.noname-e con 0x7f7c0c06f090 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c0c0716f0 0x7f7c0c1cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): picked mon.noname-f con 0x7f7c0c0716f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c0c139ef0 0x7f7c0c1cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): picked mon.noname-b con 0x7f7c0c139ef0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c0a3610 con 0x7f7c0c0716f0 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c072480 con 0x7f7c0c139ef0 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c09ca70 con 0x7f7c0c06f090 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c13932700 10 monclient(hunting): authenticate will time out at 1626.676825s 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 0x7f7c0c09d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c11ecf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c0c139ef0 0x7f7c0c1cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 0x7f7c0c09d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43630/0 (socket says 172.21.15.47:43630) 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c11ecf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c0c139ef0 0x7f7c0c1cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52994/0 (socket says 172.21.15.47:52994) 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 1 -- 172.21.15.47:0/4283524077 learned_addr learned my addr 172.21.15.47:0/4283524077 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 10 monclient(hunting): get_auth_request con 0x7f7c0c06f090 auth_method 0 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c116ce700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c11ecf700 10 monclient(hunting): get_auth_request con 0x7f7c0c139ef0 auth_method 0 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c11ecf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c11ecf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.275+0000 7f7c11ecf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c0c0716f0 0x7f7c0c1cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): get_auth_request con 0x7f7c0c0716f0 auth_method 0 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c116ce700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c116ce700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c116ce700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c11ecf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient(hunting): handle_auth_done global_id 4312 payload 824 2024-04-20T02:15:47.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 -- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 msgr2=0x7f7c0c09d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 0x7f7c0c09d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 -- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c0c139ef0 msgr2=0x7f7c0c1cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c0c139ef0 0x7f7c0c1cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 monclient: found mon.noname-f 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 -- 172.21.15.47:0/4283524077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc0094a0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c116ce700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 0x7f7c0c09d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c10ecd700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c0c0716f0 0x7f7c0c1cc190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e9b0 tx=0x7f7bfc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 client.?.objecter ms_handle_connect 0x7f7c0c0716f0 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc00ee50 con 0x7f7c0c0716f0 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: dump: 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.295 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc00ccc0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c13932700 5 monclient: authenticate success, global_id 4312 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c13932700 10 monclient: _renew_subs 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c13932700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c04009a70 con 0x7f7c0c0716f0 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc00ce30 con 0x7f7c0c0716f0 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c01ffb700 10 monclient: dump: 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.296 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.276+0000 7f7c037fe700 4 set_mon_vals no callback set 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.277+0000 7f7c13932700 10 client.4312.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.277+0000 7f7c13932700 10 monclient: _renew_subs 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.277+0000 7f7c13932700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.277+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c132100 con 0x7f7c0c0716f0 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.277+0000 7f7c13932700 1 librados: init done 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.277+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f7bfc00c850 con 0x7f7c0c0716f0 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bf003a260 0x7f7bf003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f7bfc04a070 con 0x7f7c0c0716f0 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 10 client.4312.objecter ms_dispatch 0x7f7c0c001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 3 client.4312.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 3 client.4312.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 20 client.4312.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c13932700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c13932700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c13932700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c116ce700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bf003a260 0x7f7bf003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c0008d0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c116ce700 10 monclient: get_auth_request con 0x7f7bf003a260 auth_method 0 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c01ffb700 10 client.4312.objecter ms_handle_connect 0x7f7bf003a260 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.278+0000 7f7c116ce700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bf003a260 0x7f7bf003c720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005910 tx=0x7f7c08009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.285+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7bfc017080 con 0x7f7c0c0716f0 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.285+0000 7f7c01ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.285+0000 7f7c01ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.508+0000 7f7c13932700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.508+0000 7f7c13932700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.508+0000 7f7c13932700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.508+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c0c09bb80 con 0x7f7c0c0716f0 2024-04-20T02:15:47.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.509+0000 7f7c01ffb700 1 -- 172.21.15.47:0/4283524077 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11832 (secure 0 0 0) 0x7f7bfc01b3b0 con 0x7f7c0c0716f0 2024-04-20T02:15:47.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.509+0000 7f7c01ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:47.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.509+0000 7f7c01ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:47.509 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:47.509 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:15:47.245083+0000","last_up_change":"2024-04-20T02:15:43.202903+0000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T02:15:44.345718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T02:15:44.739933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":42309},{"type":"v1","addr":"172.21.15.47:6801","nonce":42309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":42309},{"type":"v1","addr":"172.21.15.47:6803","nonce":42309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":42309},{"type":"v1","addr":"172.21.15.47:6811","nonce":42309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":42309},{"type":"v1","addr":"172.21.15.47:6806","nonce":42309}]},"public_addr":"172.21.15.47:6801/42309","cluster_addr":"172.21.15.47:6803/42309","heartbeat_back_addr":"172.21.15.47:6811/42309","heartbeat_front_addr":"172.21.15.47:6806/42309","state":["exists","up"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":42310},{"type":"v1","addr":"172.21.15.47:6807","nonce":42310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":42310},{"type":"v1","addr":"172.21.15.47:6813","nonce":42310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6819","nonce":42310},{"type":"v1","addr":"172.21.15.47:6821","nonce":42310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":42310},{"type":"v1","addr":"172.21.15.47:6817","nonce":42310}]},"public_addr":"172.21.15.47:6807/42310","cluster_addr":"172.21.15.47:6813/42310","heartbeat_back_addr":"172.21.15.47:6821/42310","heartbeat_front_addr":"172.21.15.47:6817/42310","state":["exists","up"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":42311},{"type":"v1","addr":"172.21.15.47:6812","nonce":42311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":42311},{"type":"v1","addr":"172.21.15.47:6816","nonce":42311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":42311},{"type":"v1","addr":"172.21.15.47:6823","nonce":42311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":42311},{"type":"v1","addr":"172.21.15.47:6820","nonce":42311}]},"public_addr":"172.21.15.47:6812/42311","cluster_addr":"172.21.15.47:6816/42311","heartbeat_back_addr":"172.21.15.47:6823/42311","heartbeat_front_addr":"172.21.15.47:6820/42311","state":["exists","up"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":42294},{"type":"v1","addr":"172.21.15.179:6801","nonce":42294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":42294},{"type":"v1","addr":"172.21.15.179:6803","nonce":42294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":42294},{"type":"v1","addr":"172.21.15.179:6807","nonce":42294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":42294},{"type":"v1","addr":"172.21.15.179:6805","nonce":42294}]},"public_addr":"172.21.15.179:6801/42294","cluster_addr":"172.21.15.179:6803/42294","heartbeat_back_addr":"172.21.15.179:6807/42294","heartbeat_front_addr":"172.21.15.179:6805/42294","state":["exists","up"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":42295},{"type":"v1","addr":"172.21.15.179:6812","nonce":42295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":42295},{"type":"v1","addr":"172.21.15.179:6816","nonce":42295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":42295},{"type":"v1","addr":"172.21.15.179:6823","nonce":42295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":42295},{"type":"v1","addr":"172.21.15.179:6820","nonce":42295}]},"public_addr":"172.21.15.179:6812/42295","cluster_addr":"172.21.15.179:6816/42295","heartbeat_back_addr":"172.21.15.179:6823/42295","heartbeat_front_addr":"172.21.15.179:6820/42295","state":["exists","up"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":42296},{"type":"v1","addr":"172.21.15.179:6809","nonce":42296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":42296},{"type":"v1","addr":"172.21.15.179:6813","nonce":42296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":42296},{"type":"v1","addr":"172.21.15.179:6821","nonce":42296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":42296},{"type":"v1","addr":"172.21.15.179:6817","nonce":42296}]},"public_addr":"172.21.15.179:6809/42296","cluster_addr":"172.21.15.179:6813/42296","heartbeat_back_addr":"172.21.15.179:6821/42296","heartbeat_front_addr":"172.21.15.179:6817/42296","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.384773+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.348973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.447042+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.151528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 10 librados: watch_flush enter 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 10 librados: watch_flush exit 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 20 client.4312.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 10 client.4312.objecter successfully canceled tick 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bf003a260 msgr2=0x7f7bf003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bf003a260 0x7f7bf003c720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005910 tx=0x7f7c08009040 comp rx=0 tx=0).stop 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 10 monclient: shutdown 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c0c0716f0 msgr2=0x7f7c0c1cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c0c0716f0 0x7f7c0c1cc190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e9b0 tx=0x7f7bfc00ecc0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 shutdown_connections 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c0c139ef0 0x7f7c0c1cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7c0c06f090 0x7f7c0c09d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bf003a260 0x7f7bf003c720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 --2- 172.21.15.47:0/4283524077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c0c0716f0 0x7f7c0c1cc190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 >> 172.21.15.47:0/4283524077 conn(0x7f7c0c0a7570 msgr2=0x7f7c0c0a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 shutdown_connections 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 -- 172.21.15.47:0/4283524077 wait complete. 2024-04-20T02:15:47.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.512+0000 7f7c13932700 1 librados: shutdown 2024-04-20T02:15:47.525 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T02:15:47.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T02:15:47.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T02:15:47.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T02:15:47.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T02:15:47.528 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T02:15:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.727+0000 7f0f85b7d700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.727+0000 7f0f85b7d700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.727+0000 7f0f85b7d700 10 monclient: monmap: 2024-04-20T02:15:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.727 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.728465+0000 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.728465+0000 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.728 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.729 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.730+0000 7f0f85b7d700 1 Processor -- start 2024-04-20T02:15:47.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.730+0000 7f0f85b7d700 1 -- start start 2024-04-20T02:15:47.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.730+0000 7f0f85b7d700 10 monclient: init 2024-04-20T02:15:47.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient: _add_conns ranks=[10,8,0,4,6,1,2,9,7,5,3] 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0f8013c0f0 0x7f0f8009d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): picked mon.noname-f con 0x7f0f8013c0f0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f8013c4d0 0x7f0f8013c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): picked mon.noname-e con 0x7f0f8013c4d0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0f800a1060 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f800751e0 con 0x7f0f800a1060 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8009c680 con 0x7f0f8013c0f0 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8009cbb0 con 0x7f0f8013c4d0 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f85b7d700 10 monclient(hunting): authenticate will time out at 1627.132492s 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f7ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0f800a1060 0x7f0f800a14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54118/0 (socket says 172.21.15.47:54118) 2024-04-20T02:15:47.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f7ffff700 1 -- 172.21.15.47:0/2537520360 learned_addr learned my addr 172.21.15.47:0/2537520360 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.731+0000 7f0f7f7fe700 1 --2- 172.21.15.47:0/2537520360 >> v2:172.21.15.179:3300/0 conn(0x7f0f8013c0f0 0x7f0f8009d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 1 --2- 172.21.15.47:0/2537520360 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f8013c4d0 0x7f0f8013c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): get_auth_request con 0x7f0f8013c4d0 auth_method 0 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): get_auth_request con 0x7f0f8013c0f0 auth_method 0 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f7f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_done global_id 4247 payload 274 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 1 -- 172.21.15.47:0/2537520360 >> v2:172.21.15.179:3300/0 conn(0x7f0f8013c0f0 msgr2=0x7f0f8009d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 1 --2- 172.21.15.47:0/2537520360 >> v2:172.21.15.179:3300/0 conn(0x7f0f8013c0f0 0x7f0f8009d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 1 -- 172.21.15.47:0/2537520360 >> v1:172.21.15.47:6789/0 conn(0x7f0f800a1060 legacy=0x7f0f800a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 1 monclient: found mon.noname-e 2024-04-20T02:15:47.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:47.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.732+0000 7f0f76ffd700 1 -- 172.21.15.47:0/2537520360 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8009ac20 con 0x7f0f8013c4d0 2024-04-20T02:15:47.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.733+0000 7f0f76ffd700 1 --2- 172.21.15.47:0/2537520360 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f8013c4d0 0x7f0f8013c8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009840 tx=0x7f0f68009b50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a43b2f71c26580f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.737+0000 7f8da70cb700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.737+0000 7f8da70cb700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.737+0000 7f8da70cb700 10 monclient: monmap: 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.738499+0000 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.738499+0000 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.743 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.744 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.744 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.738+0000 7f8da70cb700 1 Processor -- start 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.738+0000 7f8da70cb700 1 -- start start 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.738+0000 7f8da70cb700 10 monclient: init 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient: _add_conns ranks=[10,9,7,0,3,2,8,1,5,4,6] 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8da0137d90 0x7f8da00a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): picked mon.noname-f con 0x7f8da0137d90 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8da0138170 0x7f8da00a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): picked mon.noname-a con 0x7f8da0138170 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 0x7f8da00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): picked mon.noname-i con 0x7f8da01385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:47.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da009fa40 con 0x7f8da0138170 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da009dfc0 con 0x7f8da0137d90 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00a7bc0 con 0x7f8da01385b0 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da70cb700 10 monclient(hunting): authenticate will time out at 1627.140594s 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 0x7f8da00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 0x7f8da00a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:54986/0 (socket says 172.21.15.47:54986) 2024-04-20T02:15:47.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 1 -- 172.21.15.47:0/3563247499 learned_addr learned my addr 172.21.15.47:0/3563247499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 10 monclient(hunting): get_auth_request con 0x7f8da01385b0 auth_method 0 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.739+0000 7f8da5668700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da4e67700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.179:3300/0 conn(0x7f8da0137d90 0x7f8da00a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8d9ffff700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.47:3300/0 conn(0x7f8da0138170 0x7f8da00a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 10 monclient(hunting): handle_auth_done global_id 4255 payload 274 2024-04-20T02:15:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 1 -- 172.21.15.47:0/3563247499 >> v2:172.21.15.179:3300/0 conn(0x7f8da0137d90 msgr2=0x7f8da00a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.179:3300/0 conn(0x7f8da0137d90 0x7f8da00a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 1 -- 172.21.15.47:0/3563247499 >> v2:172.21.15.47:3300/0 conn(0x7f8da0138170 msgr2=0x7f8da00a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.47:3300/0 conn(0x7f8da0138170 0x7f8da00a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 1 monclient: found mon.noname-i 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8da5668700 1 -- 172.21.15.47:0/3563247499 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00751e0 con 0x7f8da01385b0 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8d9ffff700 10 monclient: get_auth_request con 0x7f8da0138170 auth_method 0 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.740+0000 7f8d9ffff700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.47:3300/0 conn(0x7f8da0138170 0x7f8da00a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8da5668700 1 --2- 172.21.15.47:0/3563247499 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 0x7f8da00a61d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009e50 tx=0x7f8d9800b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=b20f49cd4b3241bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 1 -- 172.21.15.47:0/3563247499 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d9800e070 con 0x7f8da01385b0 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: dump: 2024-04-20T02:15:47.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 1 -- 172.21.15.47:0/3563247499 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d980099b0 con 0x7f8da01385b0 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 1 -- 172.21.15.47:0/3563247499 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d9800bdf0 con 0x7f8da01385b0 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:47.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8d9f7fe700 10 monclient: dump: 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.752 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8da70cb700 5 monclient: authenticate success, global_id 4255 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8da70cb700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.741+0000 7f8da70cb700 4 set_mon_vals no callback set 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 10 monclient: shutdown 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 -- 172.21.15.47:0/3563247499 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 msgr2=0x7f8da00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 --2- 172.21.15.47:0/3563247499 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 0x7f8da00a61d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009e50 tx=0x7f8d9800b330 comp rx=0 tx=0).stop 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 -- 172.21.15.47:0/3563247499 shutdown_connections 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 --2- 172.21.15.47:0/3563247499 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8da01385b0 0x7f8da00a61d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.179:3300/0 conn(0x7f8da0137d90 0x7f8da00a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 --2- 172.21.15.47:0/3563247499 >> v2:172.21.15.47:3300/0 conn(0x7f8da0138170 0x7f8da00a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 -- 172.21.15.47:0/3563247499 >> 172.21.15.47:0/3563247499 conn(0x7f8da00aa8b0 msgr2=0x7f8da00a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 -- 172.21.15.47:0/3563247499 shutdown_connections 2024-04-20T02:15:47.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.742+0000 7f8da70cb700 1 -- 172.21.15.47:0/3563247499 wait complete. 2024-04-20T02:15:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.743+0000 7f8da70cb700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.743+0000 7f8da70cb700 10 monclient: monmap: 2024-04-20T02:15:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.757 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 1 librados: starting msgr at 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 1 librados: starting objecter 2024-04-20T02:15:47.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 1 Processor -- start 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 1 -- start start 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 1 librados: setting wanted keys 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 1 librados: calling monclient init 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.744+0000 7f8da70cb700 10 monclient: init 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient: _add_conns ranks=[4,2,0,5,8,1,3,6,7] 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8da0137d90 0x7f8da01cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): picked mon.noname-c con 0x7f8da0137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8da0138170 0x7f8da0136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): picked mon.noname-b con 0x7f8da0138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:47.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8da01385b0 0x7f8da0137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): picked mon.noname-a con 0x7f8da01385b0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da009dc20 con 0x7f8da01385b0 2024-04-20T02:15:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da009e9e0 con 0x7f8da0138170 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8da01245b0 con 0x7f8da0137d90 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da70cb700 10 monclient(hunting): authenticate will time out at 1627.146444s 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8da01385b0 0x7f8da0137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da4e67700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8da0137d90 0x7f8da01cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8da01385b0 0x7f8da0137130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56876/0 (socket says 172.21.15.47:56876) 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 1 -- 172.21.15.47:0/495817023 learned_addr learned my addr 172.21.15.47:0/495817023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 10 monclient(hunting): get_auth_request con 0x7f8da01385b0 auth_method 0 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da5668700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da4e67700 10 monclient(hunting): get_auth_request con 0x7f8da0137d90 auth_method 0 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da4e67700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da4e67700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.745+0000 7f8da4e67700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da4e67700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da4e67700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da4e67700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 10 monclient(hunting): handle_auth_done global_id 4311 payload 824 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 -- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8da0137d90 msgr2=0x7f8da01cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8da0137d90 0x7f8da01cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 -- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8da0138170 msgr2=0x7f8da0136bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:15:47.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8da0138170 0x7f8da0136bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 monclient: found mon.noname-a 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 -- 172.21.15.47:0/495817023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98009b30 con 0x7f8da01385b0 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da4e67700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8da0137d90 0x7f8da01cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:47.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8da5668700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8da01385b0 0x7f8da0137130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800b370 tx=0x7f8d98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8d9d7fa700 10 client.?.objecter ms_handle_connect 0x7f8da01385b0 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.746+0000 7f8d9d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d98003e90 con 0x7f8da01385b0 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: dump: 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.766 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 5 monclient: authenticate success, global_id 4311 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d980046c0 con 0x7f8da01385b0 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d98004830 con 0x7f8da01385b0 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9d7fa700 10 monclient: dump: 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.769 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8d9effd700 4 set_mon_vals no callback set 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 10 monclient: _renew_subs 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da0122b30 con 0x7f8da01385b0 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 10 client.4311.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 10 monclient: _renew_subs 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da0122200 con 0x7f8da01385b0 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.747+0000 7f8da70cb700 1 librados: init done 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.748+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f8d98004250 con 0x7f8da01385b0 2024-04-20T02:15:47.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8d9d7fa700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8c03a260 0x7f8d8c03c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8d9804b3a0 con 0x7f8da01385b0 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8d9d7fa700 10 client.4311.objecter ms_dispatch 0x7f8da0001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8d9d7fa700 3 client.4311.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8d9d7fa700 3 client.4311.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8d9d7fa700 20 client.4311.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 10 client.4311.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 15 client.4311.objecter _session_command_op_assign -1 1 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f8da006aac0 0x7f8da0120de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 20 client.4311.objecter _get_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 20 client.4311.objecter _get_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 15 client.4311.objecter _session_command_op_remove -1 1 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 20 client.4311.objecter get_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 15 client.4311.objecter _session_command_op_assign 4 1 2024-04-20T02:15:47.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 4 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 10 client.4311.objecter _send_command 1 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 --> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8da0124230 con 0x7f8da006aac0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.749+0000 7f8da4e67700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8c03a260 0x7f8d8c03c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.750+0000 7f8da4e67700 10 monclient: get_auth_request con 0x7f8d8c03a260 auth_method 0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.750+0000 7f8d9d7fa700 10 client.4311.objecter ms_handle_connect 0x7f8d8c03a260 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.750+0000 7f8da4e67700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8c03a260 0x7f8d8c03c720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000af60 tx=0x7f8d90008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.751+0000 7f8d9ffff700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f8da006aac0 0x7f8da0120de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.751+0000 7f8d9ffff700 10 monclient: get_auth_request con 0x7f8da006aac0 auth_method 0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.751+0000 7f8d9d7fa700 10 client.4311.objecter ms_handle_connect 0x7f8da006aac0 2024-04-20T02:15:47.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.751+0000 7f8d9ffff700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f8da006aac0 0x7f8da0120de0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.752+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== osd.4 v2:172.21.15.179:6810/42295 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8d9400a510 con 0x7f8da006aac0 2024-04-20T02:15:47.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.752+0000 7f8d9d7fa700 10 client.4311.objecter ms_dispatch 0x7f8da0001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:15:47.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.752+0000 7f8d9d7fa700 10 client.4311.objecter _finish_command 1 = system:0 2024-04-20T02:15:47.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.752+0000 7f8d9d7fa700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 4 2024-04-20T02:15:47.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.752+0000 7f8d9d7fa700 15 client.4311.objecter _session_command_op_remove 4 1 2024-04-20T02:15:47.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.753+0000 7f8b71014700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.753+0000 7f8b71014700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.753+0000 7f8b71014700 10 monclient: monmap: 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.754513+0000 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.754513+0000 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.754+0000 7f8b71014700 1 Processor -- start 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.754+0000 7f8b71014700 1 -- start start 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.754+0000 7f8b71014700 10 monclient: init 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient: _add_conns ranks=[4,1,2,7,5,8,9,0,6,10,3] 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 0x7f8b6c0a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): picked mon.noname-c con 0x7f8b6c0a5c90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8b6c0a0eb0 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:47.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): picked mon.noname-b con 0x7f8b6c0a6780 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6c0751e0 con 0x7f8b6c0a0eb0 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c09fd20 con 0x7f8b6c0a6780 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0a7bc0 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b71014700 10 monclient(hunting): authenticate will time out at 1627.156272s 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 0x7f8b6c0a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 0x7f8b6c0a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44408/0 (socket says 172.21.15.47:44408) 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 1 -- 172.21.15.47:0/121927440 learned_addr learned my addr 172.21.15.47:0/121927440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a5c90 auth_method 0 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.755+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6b59e700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_done global_id 4261 payload 274 2024-04-20T02:15:47.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 1 -- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8b6c0a6780 msgr2=0x7f8b6c143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 1 -- 172.21.15.47:0/121927440 >> v1:172.21.15.179:6789/0 conn(0x7f8b6c0a0eb0 legacy=0x7f8b6c0a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 1 monclient: found mon.noname-c 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 1 -- 172.21.15.47:0/121927440 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c09f760 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f3be40e8700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f3be40e8700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f3be40e8700 10 monclient: monmap: 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.757437+0000 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.757437+0000 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.780 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.781 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.781 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b6ad9d700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 0x7f8b6c0a0970 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009920 tx=0x7f8b54009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4494e02281bbbe6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 1 -- 172.21.15.47:0/121927440 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b54009e70 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: dump: 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.782 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.783 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b71014700 5 monclient: authenticate success, global_id 4261 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b71014700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 1 -- 172.21.15.47:0/121927440 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b540044a0 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 1 -- 172.21.15.47:0/121927440 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b5400b820 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:47.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.756+0000 7f8b69d9b700 10 monclient: dump: 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.785 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b6b59e700 10 monclient: get_auth_request con 0x7f8b6c0a6780 auth_method 0 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b71014700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b71014700 4 set_mon_vals no callback set 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b6b59e700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7fea16f55700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7fea16f55700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7fea16f55700 10 monclient: monmap: 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.758519+0000 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.758519+0000 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.787 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.788 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.788 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.788 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b71014700 10 monclient: shutdown 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b71014700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b71014700 1 -- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 msgr2=0x7f8b6c0a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.757+0000 7f8b71014700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 0x7f8b6c0a0970 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009920 tx=0x7f8b54009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f8b71014700 1 -- 172.21.15.47:0/121927440 shutdown_connections 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f8b71014700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f8b71014700 1 --2- 172.21.15.47:0/121927440 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b6c0a5c90 0x7f8b6c0a0970 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f8b71014700 1 -- 172.21.15.47:0/121927440 >> 172.21.15.47:0/121927440 conn(0x7f8b6c0aa8b0 msgr2=0x7f8b6c0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f3be40e8700 1 Processor -- start 2024-04-20T02:15:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f8b71014700 1 -- 172.21.15.47:0/121927440 shutdown_connections 2024-04-20T02:15:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 1 Processor -- start 2024-04-20T02:15:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f3be40e8700 1 -- start start 2024-04-20T02:15:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.758+0000 7f3be40e8700 10 monclient: init 2024-04-20T02:15:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient: _add_conns ranks=[0,10,2,3,5,1,4,7,9,6,8] 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3bdc0a44f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3bdc072030 0x7f3bdc1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): picked mon.noname-f con 0x7f3bdc072030 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3bdc139ef0 0x7f3bdc13c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): picked mon.noname-b con 0x7f3bdc139ef0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0751e0 con 0x7f3bdc0a44f0 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc0a0390 con 0x7f3bdc072030 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0a48d0 con 0x7f3bdc139ef0 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be40e8700 10 monclient(hunting): authenticate will time out at 1627.160331s 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f27e39ab700 10 monclient: get_monmap_and_config 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f27e39ab700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f27e39ab700 10 monclient: monmap: 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:47.760427+0000 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:47.760427+0000 2024-04-20T02:15:47.792 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.793 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.794 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.794 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.794 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.794 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.794 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:47.794 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:47.795 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f3be1683700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3bdc072030 0x7f3bdc1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 1 -- start start 2024-04-20T02:15:47.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 10 monclient: init 2024-04-20T02:15:47.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 10 monclient: _add_conns ranks=[0,4,9,7,2,10,6,8,1,3,5] 2024-04-20T02:15:47.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fea10099450 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:47.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea100a0260 0x7fea10139020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7fea16f55700 10 monclient(hunting): picked mon.noname-c con 0x7fea100a0260 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:47.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea100a06a0 0x7fea1013b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 10 monclient(hunting): picked mon.noname-a con 0x7fea100a06a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea100751e0 con 0x7fea10099450 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1009da90 con 0x7fea100a06a0 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009f1f0 con 0x7fea100a0260 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea16f55700 10 monclient(hunting): authenticate will time out at 1627.161056s 2024-04-20T02:15:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1e84700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3bdc0a44f0 0x7f3bdc071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54132/0 (socket says 172.21.15.47:54132) 2024-04-20T02:15:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1e84700 1 -- 172.21.15.47:0/3110235469 learned_addr learned my addr 172.21.15.47:0/3110235469 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): get_auth_request con 0x7f3bdc072030 auth_method 0 2024-04-20T02:15:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.759+0000 7f8b71014700 1 -- 172.21.15.47:0/121927440 wait complete. 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f8b71014700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f8b71014700 10 monclient: monmap: 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.800 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.801 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f8b71014700 1 librados: starting msgr at 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f8b71014700 1 librados: starting objecter 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 Processor -- start 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea14cf1700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fea10099450 0x7fea10099830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54144/0 (socket says 172.21.15.47:54144) 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea14cf1700 1 -- 172.21.15.47:0/1747124105 learned_addr learned my addr 172.21.15.47:0/1747124105 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea0ffff700 1 --2- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea100a0260 0x7fea10139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 1 --2- 172.21.15.47:0/1747124105 >> v2:172.21.15.47:3300/0 conn(0x7fea100a06a0 0x7fea1013b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): get_auth_request con 0x7fea100a06a0 auth_method 0 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea0ffff700 10 monclient(hunting): get_auth_request con 0x7fea100a0260 auth_method 0 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea0ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea0ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea0ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7fea154f2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 10 monclient(hunting): handle_auth_done global_id 4320 payload 274 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 1 -- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea100a0260 msgr2=0x7fea10139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 1 --2- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea100a0260 0x7fea10139020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 1 -- 172.21.15.47:0/1747124105 >> v1:172.21.15.47:6789/0 conn(0x7fea10099450 legacy=0x7fea10099830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 1 monclient: found mon.noname-a 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 1 -- 172.21.15.47:0/1747124105 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1009f4d0 con 0x7fea100a06a0 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.760+0000 7f3be1683700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 10 monclient(hunting): handle_auth_done global_id 4330 payload 274 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 1 -- 172.21.15.47:0/3110235469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3bdc139ef0 msgr2=0x7f3bdc13c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 1 --2- 172.21.15.47:0/3110235469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3bdc139ef0 0x7f3bdc13c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 1 -- 172.21.15.47:0/3110235469 >> v1:172.21.15.47:6789/0 conn(0x7f3bdc0a44f0 legacy=0x7f3bdc071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 1 monclient: found mon.noname-f 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f3be1683700 1 -- 172.21.15.47:0/3110235469 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc09e930 con 0x7f3bdc072030 2024-04-20T02:15:47.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7fea154f2700 1 --2- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a06a0 0x7fea1013b570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea0800bfa0 tx=0x7fea0800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa8cdc7d61a99e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 -- start start 2024-04-20T02:15:47.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 librados: setting wanted keys 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 librados: calling monclient init 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient: init 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient: _add_conns ranks=[7,0,3,5,4,6,2,1,8] 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b6c0a0eb0 0x7f8b6c1364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): picked mon.noname-i con 0x7f8b6c0a0eb0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b6c0a5c90 0x7f8b6c071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): picked mon.noname-a con 0x7f8b6c0a5c90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): picked mon.noname-g con 0x7f8b6c0a6780 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c139920 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c09d9d0 con 0x7f8b6c0a6780 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c143790 con 0x7f8b6c0a0eb0 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.761+0000 7f8b71014700 10 monclient(hunting): authenticate will time out at 1627.162902s 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b6c0a5c90 0x7f8b6c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b6c0a5c90 0x7f8b6c071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56894/0 (socket says 172.21.15.47:56894) 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 -- 172.21.15.47:0/1686134764 learned_addr learned my addr 172.21.15.47:0/1686134764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a5c90 auth_method 0 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be1683700 1 --2- 172.21.15.47:0/3110235469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc072030 0x7f3bdc1399b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009840 tx=0x7f3bd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c959a1a311c3543 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b6c0a0eb0 0x7f8b6c1364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a6780 auth_method 0 2024-04-20T02:15:47.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a0eb0 auth_method 0 2024-04-20T02:15:47.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1747124105 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea08010070 con 0x7fea100a06a0 2024-04-20T02:15:47.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: dump: 2024-04-20T02:15:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.812 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1747124105 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0800bb00 con 0x7fea100a06a0 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1747124105 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea0800b2e0 con 0x7fea100a06a0 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea0f7fe700 10 monclient: dump: 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.814 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.815 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6b59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient(hunting): handle_auth_done global_id 4347 payload 824 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 1 -- 172.21.15.47:0/3110235469 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3bd8012df0 con 0x7f3bdc072030 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: dump: 2024-04-20T02:15:47.817 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.818 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 1 -- 172.21.15.47:0/3110235469 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd801b400 con 0x7f3bdc072030 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 1 -- 172.21.15.47:0/3110235469 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3bd801b9e0 con 0x7f3bdc072030 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be0e82700 10 monclient: dump: 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.819 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.820 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.821 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.821 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be40e8700 5 monclient: authenticate success, global_id 4330 2024-04-20T02:15:47.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be40e8700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f3be40e8700 4 set_mon_vals no callback set 2024-04-20T02:15:47.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f27e39ab700 1 Processor -- start 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 -- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b6c0a0eb0 msgr2=0x7f8b6c1364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b6c0a0eb0 0x7f8b6c1364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 -- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a6780 msgr2=0x7f8b6c135160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a6780 0x7f8b6c135160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 monclient: found mon.noname-a 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 -- 172.21.15.47:0/1686134764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54009550 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b6a59c700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b6c0a5c90 0x7f8b6c071880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c4a0 tx=0x7f8b5c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b637fe700 10 client.?.objecter ms_handle_connect 0x7f8b6c0a5c90 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7f8b637fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea16f55700 5 monclient: authenticate success, global_id 4320 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea16f55700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.762+0000 7fea16f55700 4 set_mon_vals no callback set 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7fea16f55700 10 monclient: shutdown 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7fea16f55700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7fea16f55700 1 -- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a06a0 msgr2=0x7fea1013b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7fea16f55700 1 --2- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a06a0 0x7fea1013b570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea0800bfa0 tx=0x7fea0800d330 comp rx=0 tx=0).stop 2024-04-20T02:15:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f3be40e8700 10 monclient: shutdown 2024-04-20T02:15:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f3be40e8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f3be40e8700 1 -- 172.21.15.47:0/3110235469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc072030 msgr2=0x7f3bdc1399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f3be40e8700 1 --2- 172.21.15.47:0/3110235469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc072030 0x7f3bdc1399b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009840 tx=0x7f3bd8009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b5c011a00 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: dump: 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c011b80 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b5c011cf0 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b637fe700 10 monclient: dump: 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.828 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f8b6959a700 4 set_mon_vals no callback set 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 5 monclient: authenticate success, global_id 4347 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 10 monclient: _renew_subs 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c139080 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 10 client.4347.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 10 monclient: _renew_subs 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c139640 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f3be40e8700 1 -- 172.21.15.47:0/3110235469 shutdown_connections 2024-04-20T02:15:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f3be40e8700 1 --2- 172.21.15.47:0/3110235469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3bdc139ef0 0x7f3bdc13c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f3be40e8700 1 --2- 172.21.15.47:0/3110235469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc072030 0x7f3bdc1399b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f8b71014700 1 librados: init done 2024-04-20T02:15:47.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f3be40e8700 1 -- 172.21.15.47:0/3110235469 >> 172.21.15.47:0/3110235469 conn(0x7f3bdc0a39d0 msgr2=0x7f3bdc0a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f3be40e8700 1 -- 172.21.15.47:0/3110235469 shutdown_connections 2024-04-20T02:15:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f27e39ab700 1 -- start start 2024-04-20T02:15:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.763+0000 7f27e39ab700 10 monclient: init 2024-04-20T02:15:47.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient: _add_conns ranks=[9,8,1,4,3,10,2,0,6,7,5] 2024-04-20T02:15:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27dc0721d0 0x7f27dc09ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): picked mon.noname-a con 0x7f27dc0721d0 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f27dc09f930 0x7f27dc133990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): picked mon.noname-e con 0x7f27dc09f930 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f27dc09fd30 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc0751e0 con 0x7f27dc09fd30 2024-04-20T02:15:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27dc09cb50 con 0x7f27dc0721d0 2024-04-20T02:15:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0725b0 con 0x7f27dc09f930 2024-04-20T02:15:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e39ab700 10 monclient(hunting): authenticate will time out at 1627.165211s 2024-04-20T02:15:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e1f48700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f27dc09fd30 0x7f27dc133ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:52586/0 (socket says 172.21.15.47:52586) 2024-04-20T02:15:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f27e1f48700 1 -- 172.21.15.47:0/3286223571 learned_addr learned my addr 172.21.15.47:0/3286223571 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 1 --2- 172.21.15.47:0/3286223571 >> v2:172.21.15.47:3300/0 conn(0x7f27dc0721d0 0x7f27dc09ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): get_auth_request con 0x7f27dc0721d0 auth_method 0 2024-04-20T02:15:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 1 -- 172.21.15.47:0/3286223571 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206667667 0 0) 0x7f27dc0751e0 con 0x7f27dc09fd30 2024-04-20T02:15:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 10 monclient(hunting): _negotiate 2024-04-20T02:15:47.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 10 monclient(hunting): my global_id is 4339 2024-04-20T02:15:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 1 -- 172.21.15.47:0/3286223571 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c80035b0 con 0x7f27dc09fd30 2024-04-20T02:15:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient(hunting): handle_auth_done global_id 4356 payload 274 2024-04-20T02:15:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 1 -- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f27dc09f930 msgr2=0x7f27dc133990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 1 --2- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f27dc09f930 0x7f27dc133990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 1 -- 172.21.15.47:0/3286223571 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1924054607 0 0) 0x7f27c80035b0 con 0x7f27dc09fd30 2024-04-20T02:15:47.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 1 -- 172.21.15.47:0/3286223571 >> v1:172.21.15.179:6789/0 conn(0x7f27dc09fd30 legacy=0x7f27dc133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 1 monclient: found mon.noname-a 2024-04-20T02:15:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27e1747700 1 -- 172.21.15.47:0/3286223571 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0a2030 con 0x7f27dc0721d0 2024-04-20T02:15:47.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f27d3fff700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b71014700 10 client.4347.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b71014700 15 client.4347.objecter _session_command_op_assign -1 1 2024-04-20T02:15:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b71014700 20 client.4347.objecter _get_session osd=-1 returning homeless 2024-04-20T02:15:47.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b71014700 10 client.4347.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b71014700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b6c0a7790 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f8b5c011590 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b4c03a260 0x7f8b4c03c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8b5c049c30 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 10 client.4347.objecter ms_dispatch 0x7f8b6c001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 3 client.4347.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 3 client.4347.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 10 client.4347.objecter checking command 1 2024-04-20T02:15:47.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7f8b4c040070 0x7f8b4c042490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.764+0000 7f3be40e8700 1 -- 172.21.15.47:0/3110235469 wait complete. 2024-04-20T02:15:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f3be40e8700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f3be40e8700 10 monclient: monmap: 2024-04-20T02:15:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.844 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.845 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.846 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.847 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f3be40e8700 1 librados: starting msgr at 2024-04-20T02:15:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.765+0000 7f3be40e8700 1 librados: starting objecter 2024-04-20T02:15:47.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f3be40e8700 1 Processor -- start 2024-04-20T02:15:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27e1747700 1 --2- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27dc0721d0 0x7f27dc09ae40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00a9e0 tx=0x7f27cc00acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f01527f03ae6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 20 client.4347.objecter _get_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 15 client.4347.objecter _session_command_op_remove -1 1 2024-04-20T02:15:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 20 client.4347.objecter _get_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 20 client.4347.objecter get_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 15 client.4347.objecter _session_command_op_assign 0 1 2024-04-20T02:15:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 4 2024-04-20T02:15:47.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 10 client.4347.objecter _send_command 1 2024-04-20T02:15:47.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 --> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b4c042b40 con 0x7f8b4c040070 2024-04-20T02:15:47.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b637fe700 20 client.4347.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 1 -- 172.21.15.47:0/3286223571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27d4001400 con 0x7f27dc0721d0 2024-04-20T02:15:47.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: dump: 2024-04-20T02:15:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.852 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.853 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.854 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.854 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.854 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.854 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.854 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.854 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.855 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 1 -- 172.21.15.47:0/3286223571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc00ae20 con 0x7f27dc0721d0 2024-04-20T02:15:47.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 1 -- 172.21.15.47:0/3286223571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27cc0044a0 con 0x7f27dc0721d0 2024-04-20T02:15:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b6b59e700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7f8b4c040070 0x7f8b4c042490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f8b6b59e700 10 monclient: get_auth_request con 0x7f8b4c040070 auth_method 0 2024-04-20T02:15:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:47.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27d3fff700 10 monclient: dump: 2024-04-20T02:15:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.858 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.859 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.860 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27e39ab700 5 monclient: authenticate success, global_id 4356 2024-04-20T02:15:47.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27e39ab700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f27e39ab700 4 set_mon_vals no callback set 2024-04-20T02:15:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b5c011840 con 0x7f8b6c0a5c90 2024-04-20T02:15:47.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f8b637fe700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-04-20T02:15:47.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7fea16f55700 1 -- 172.21.15.47:0/1747124105 shutdown_connections 2024-04-20T02:15:47.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7fea16f55700 1 --2- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a06a0 0x7fea1013b570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7fea16f55700 1 --2- 172.21.15.47:0/1747124105 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea100a0260 0x7fea10139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7fea16f55700 1 -- 172.21.15.47:0/1747124105 >> 172.21.15.47:0/1747124105 conn(0x7fea100a2f00 msgr2=0x7fea100a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f8b6b59e700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7f8b4c040070 0x7f8b4c042490 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f8b637fe700 10 client.4347.objecter ms_handle_connect 0x7f8b4c040070 2024-04-20T02:15:47.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f8b6ad9d700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b4c03a260 0x7f8b4c03c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f8b6ad9d700 10 monclient: get_auth_request con 0x7f8b4c03a260 auth_method 0 2024-04-20T02:15:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 10 monclient: shutdown 2024-04-20T02:15:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 1 -- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27dc0721d0 msgr2=0x7f27dc09ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 1 --2- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27dc0721d0 0x7f27dc09ae40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00a9e0 tx=0x7f27cc00acf0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f3be40e8700 1 -- start start 2024-04-20T02:15:47.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f3be40e8700 1 librados: setting wanted keys 2024-04-20T02:15:47.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f3be40e8700 1 librados: calling monclient init 2024-04-20T02:15:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.766+0000 7f3be40e8700 10 monclient: init 2024-04-20T02:15:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient: _add_conns ranks=[6,1,3,2,4,7,8,0,5] 2024-04-20T02:15:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3bdc072030 0x7f3bdc1d9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): picked mon.noname-d con 0x7f3bdc072030 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:47.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc0a44f0 0x7f3bdc1d9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): picked mon.noname-f con 0x7f3bdc0a44f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:47.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bdc139ef0 0x7f3bdc131000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): picked mon.noname-g con 0x7f3bdc139ef0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:47.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc134cd0 con 0x7f3bdc0a44f0 2024-04-20T02:15:47.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc134f40 con 0x7f3bdc139ef0 2024-04-20T02:15:47.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc09f3f0 con 0x7f3bdc072030 2024-04-20T02:15:47.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be40e8700 10 monclient(hunting): authenticate will time out at 1627.168621s 2024-04-20T02:15:47.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 1 -- 172.21.15.47:0/3286223571 shutdown_connections 2024-04-20T02:15:47.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 1 --2- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27dc0721d0 0x7f27dc09ae40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 1 --2- 172.21.15.47:0/3286223571 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f27dc09f930 0x7f27dc133990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f27e39ab700 1 -- 172.21.15.47:0/3286223571 >> 172.21.15.47:0/3286223571 conn(0x7f27dc0a6210 msgr2=0x7f27dc0a6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7fea16f55700 1 -- 172.21.15.47:0/1747124105 shutdown_connections 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7fea16f55700 1 -- 172.21.15.47:0/1747124105 wait complete. 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.767+0000 7f3be1e84700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3bdc072030 0x7f3bdc1d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3bdc072030 0x7f3bdc1d9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52546/0 (socket says 172.21.15.47:52546) 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 -- 172.21.15.47:0/2887306240 learned_addr learned my addr 172.21.15.47:0/2887306240 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): get_auth_request con 0x7f3bdc072030 auth_method 0 2024-04-20T02:15:47.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc0a44f0 0x7f3bdc1d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be2685700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bdc139ef0 0x7f3bdc131000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): get_auth_request con 0x7f3bdc0a44f0 auth_method 0 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be2685700 10 monclient(hunting): get_auth_request con 0x7f3bdc139ef0 auth_method 0 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be2685700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be2685700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be2685700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1683700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient(hunting): handle_auth_done global_id 4236 payload 824 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 -- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bdc139ef0 msgr2=0x7f3bdc131000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3bdc139ef0 0x7f3bdc131000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 -- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc0a44f0 msgr2=0x7f3bdc1d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bdc0a44f0 0x7f3bdc1d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 monclient: found mon.noname-d 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:47.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f3be1e84700 1 -- 172.21.15.47:0/2887306240 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80094a0 con 0x7f3bdc072030 2024-04-20T02:15:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f27e39ab700 1 -- 172.21.15.47:0/3286223571 shutdown_connections 2024-04-20T02:15:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7f3be1e84700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3bdc072030 0x7f3bdc1d9420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400cba0 tx=0x7f3bd400ceb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7f3bd27fc700 10 client.?.objecter ms_handle_connect 0x7f3bdc072030 2024-04-20T02:15:47.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7f3bd27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7fea16f55700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7fea16f55700 10 monclient: monmap: 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7fea16f55700 1 librados: starting msgr at 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7fea16f55700 1 librados: starting objecter 2024-04-20T02:15:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.769+0000 7fea16f55700 1 Processor -- start 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== osd.0 v2:172.21.15.47:6800/42309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8b4c042b40 con 0x7f8b4c040070 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b637fe700 10 client.4347.objecter ms_dispatch 0x7f8b6c001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b637fe700 10 client.4347.objecter _finish_command 1 = system:0 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b637fe700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 4 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b637fe700 15 client.4347.objecter _session_command_op_remove 0 1 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b637fe700 10 client.4347.objecter ms_handle_connect 0x7f8b4c03a260 2024-04-20T02:15:47.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.768+0000 7f8b6ad9d700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b4c03a260 0x7f8b4c03c720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009920 tx=0x7f8b54008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3bd400ad40 con 0x7f3bdc072030 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 10 monclient: dump: 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3bd27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 5 monclient: authenticate success, global_id 4236 2024-04-20T02:15:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 monclient: _renew_subs 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc133710 con 0x7f3bdc072030 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 client.4236.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 monclient: _renew_subs 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc132b40 con 0x7f3bdc072030 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 1 librados: init done 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 client.4236.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 15 client.4236.objecter _session_command_op_assign -1 1 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 20 client.4236.objecter _get_session osd=-1 returning homeless 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 client.4236.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3bdc135080 con 0x7f3bdc072030 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd400a8d0 con 0x7f3bdc072030 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3bd400aea0 con 0x7f3bdc072030 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd27fc700 10 monclient: dump: 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.882 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f3bd3fff700 4 set_mon_vals no callback set 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f3bd400aa40 con 0x7f3bdc072030 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3bc003a260 0x7f3bc003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f3bd404a050 con 0x7f3bdc072030 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 10 client.4236.objecter ms_dispatch 0x7f3bdc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 3 client.4236.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 3 client.4236.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 10 client.4236.objecter checking command 1 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f3bc0040070 0x7f3bc0042490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 20 client.4236.objecter _get_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 15 client.4236.objecter _session_command_op_remove -1 1 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 20 client.4236.objecter _get_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 20 client.4236.objecter get_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 15 client.4236.objecter _session_command_op_assign 3 1 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 4 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 10 client.4236.objecter _send_command 1 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 --> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bc0042b40 con 0x7f3bc0040070 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 20 client.4236.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd404a330 con 0x7f3bdc072030 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f3bd27fc700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7f3be1683700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3bc003a260 0x7f3bc003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7f3be1683700 10 monclient: get_auth_request con 0x7f3bc003a260 auth_method 0 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7f3be1683700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3bc003a260 0x7f3bc003c720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80090b0 tx=0x7f3bd8012040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7f3bd27fc700 10 client.4236.objecter ms_handle_connect 0x7f3bc003a260 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7fea16f55700 1 -- start start 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7fea16f55700 1 librados: setting wanted keys 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7fea16f55700 1 librados: calling monclient init 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7fea16f55700 10 monclient: init 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient: _add_conns ranks=[4,8,2,0,6,5,3,7,1] 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea10099450 0x7fea101d9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): picked mon.noname-c con 0x7fea10099450 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea100a0260 0x7fea101d9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): picked mon.noname-e con 0x7fea100a0260 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea100a06a0 0x7fea1012eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): picked mon.noname-b con 0x7fea100a06a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009e960 con 0x7fea100a06a0 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea1006fe00 con 0x7fea10099450 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fea1012e4f0 con 0x7fea100a0260 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7fea16f55700 10 monclient(hunting): authenticate will time out at 1627.174764s 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea100a0260 0x7fea101d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea10099450 0x7fea101d9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea100a0260 0x7fea101d9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43652/0 (socket says 172.21.15.47:43652) 2024-04-20T02:15:47.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 1 -- 172.21.15.47:0/3114634860 learned_addr learned my addr 172.21.15.47:0/3114634860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): get_auth_request con 0x7fea100a0260 auth_method 0 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): get_auth_request con 0x7fea10099450 auth_method 0 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea100a06a0 0x7fea1012eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): get_auth_request con 0x7fea100a06a0 auth_method 0 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea14cf1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient(hunting): handle_auth_done global_id 4250 payload 824 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 -- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea100a0260 msgr2=0x7fea101d9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea100a0260 0x7fea101d9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 -- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea10099450 msgr2=0x7fea101d9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fea10099450 0x7fea101d9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 monclient: found mon.noname-b 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.774+0000 7fea154f2700 1 -- 172.21.15.47:0/3114634860 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0800bc80 con 0x7fea100a06a0 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea154f2700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea100a06a0 0x7fea1012eb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea08017000 tx=0x7fea08004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 client.?.objecter ms_handle_connect 0x7fea100a06a0 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea08003e10 con 0x7fea100a06a0 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 monclient: dump: 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea080046c0 con 0x7fea100a06a0 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea08004830 con 0x7fea100a06a0 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 5 monclient: authenticate success, global_id 4250 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 10 monclient: _renew_subs 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1006fb20 con 0x7fea100a06a0 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea0effd700 4 set_mon_vals no callback set 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 10 client.4250.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 10 monclient: _renew_subs 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea101d9db0 con 0x7fea100a06a0 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.775+0000 7fea16f55700 1 librados: init done 2024-04-20T02:15:47.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea0d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea0d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea16f55700 10 client.4250.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea0d7fa700 10 monclient: dump: 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea16f55700 15 client.4250.objecter _session_command_op_assign -1 1 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea16f55700 20 client.4250.objecter _get_session osd=-1 returning homeless 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea16f55700 10 client.4250.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea16f55700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.776+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea1009fa90 con 0x7fea100a06a0 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7fea08009020 con 0x7fea100a06a0 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f403a290 0x7fe9f403c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fea0806e030 con 0x7fea100a06a0 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 10 client.4250.objecter ms_dispatch 0x7fea10001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 3 client.4250.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 3 client.4250.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 10 client.4250.objecter checking command 1 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7fe9f4040120 0x7fe9f4042540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 20 client.4250.objecter _get_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 15 client.4250.objecter _session_command_op_remove -1 1 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 20 client.4250.objecter _get_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 20 client.4250.objecter get_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 15 client.4250.objecter _session_command_op_assign 1 1 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 4 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 10 client.4250.objecter _send_command 1 2024-04-20T02:15:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9f4042bf0 con 0x7fe9f4040120 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 20 client.4250.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea0806e310 con 0x7fea100a06a0 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0d7fa700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0ffff700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7fe9f4040120 0x7fe9f4042540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea14cf1700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f403a290 0x7fe9f403c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f3be2685700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f3bc0040070 0x7f3bc0042490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f3be2685700 10 monclient: get_auth_request con 0x7f3bc0040070 auth_method 0 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f3be2685700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f3bc0040070 0x7f3bc0042490 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f3bd27fc700 10 client.4236.objecter ms_handle_connect 0x7f3bc0040070 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 10 client.4311.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 15 client.4311.objecter _session_command_op_assign -1 2 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 20 client.4311.objecter _get_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 20 client.4311.objecter _get_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 15 client.4311.objecter _session_command_op_remove -1 2 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 20 client.4311.objecter get_session s=0x7f8da009f760 osd=4 3 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 15 client.4311.objecter _session_command_op_assign 4 2 2024-04-20T02:15:47.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 4 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 10 client.4311.objecter _send_command 2 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.779+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 --> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8da009e430 con 0x7f8da006aac0 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8d9d7fa700 1 -- 172.21.15.47:0/495817023 <== osd.4 v2:172.21.15.179:6810/42295 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8da009e430 con 0x7f8da006aac0 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8d9d7fa700 10 client.4311.objecter ms_dispatch 0x7f8da0001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8d9d7fa700 10 client.4311.objecter _finish_command 2 = system:0 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8d9d7fa700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 4 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8d9d7fa700 15 client.4311.objecter _session_command_op_remove 4 2 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 10 librados: watch_flush enter 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 10 librados: watch_flush exit 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 10 client.4311.objecter close_session for osd.4 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f8da006aac0 msgr2=0x7f8da0120de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f8da006aac0 0x7f8da0120de0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 20 client.4311.objecter put_session s=0x7f8da009f760 osd=4 1 2024-04-20T02:15:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 20 client.4311.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f8da70cb700 10 client.4311.objecter successfully canceled tick 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8c03a260 msgr2=0x7f8d8c03c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d8c03a260 0x7f8d8c03c720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000af60 tx=0x7f8d90008040 comp rx=0 tx=0).stop 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f27e39ab700 1 -- 172.21.15.47:0/3286223571 wait complete. 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f27e39ab700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.770+0000 7f27e39ab700 10 monclient: monmap: 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 1 librados: starting msgr at 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 1 librados: starting objecter 2024-04-20T02:15:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 1 Processor -- start 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 1 -- start start 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 1 librados: setting wanted keys 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 1 librados: calling monclient init 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.771+0000 7f27e39ab700 10 monclient: init 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient: _add_conns ranks=[1,2,3,7,4,5,0,6,8] 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27dc0721d0 0x7f27dc1e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): picked mon.noname-f con 0x7f27dc0721d0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27dc09f930 0x7f27dc1ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): picked mon.noname-b con 0x7f27dc09f930 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27dc09fd30 0x7f27dc130300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): picked mon.noname-g con 0x7f27dc09fd30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc12cc00 con 0x7f27dc0721d0 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc09ba60 con 0x7f27dc09f930 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc09bf20 con 0x7f27dc09fd30 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e39ab700 10 monclient(hunting): authenticate will time out at 1627.173469s 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27dc09fd30 0x7f27dc130300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1747700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27dc0721d0 0x7f27dc1e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27dc09fd30 0x7f27dc130300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:48764/0 (socket says 172.21.15.47:48764) 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 1 -- 172.21.15.47:0/269563054 learned_addr learned my addr 172.21.15.47:0/269563054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 10 monclient(hunting): get_auth_request con 0x7f27dc09fd30 auth_method 0 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1f48700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1747700 10 monclient(hunting): get_auth_request con 0x7f27dc0721d0 auth_method 0 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1747700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1747700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e1747700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e0f46700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27dc09f930 0x7f27dc1ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e0f46700 10 monclient(hunting): get_auth_request con 0x7f27dc09f930 auth_method 0 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e0f46700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e0f46700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.772+0000 7f27e0f46700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1f48700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1f48700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1f48700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e0f46700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e0f46700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e0f46700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 10 monclient(hunting): handle_auth_done global_id 4357 payload 824 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 1 -- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27dc09fd30 msgr2=0x7f27dc130300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f27dc09fd30 0x7f27dc130300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 1 -- 172.21.15.47:0/269563054 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27dc09f930 msgr2=0x7f27dc1ea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f27dc09f930 0x7f27dc1ea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 1 monclient: found mon.noname-f 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.773+0000 7f27e1747700 1 -- 172.21.15.47:0/269563054 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc00a640 con 0x7f27dc0721d0 2024-04-20T02:15:47.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.780+0000 7f27e1747700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27dc0721d0 0x7f27dc1e9b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f27cc019000 tx=0x7f27cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea0ffff700 10 monclient: get_auth_request con 0x7fe9f4040120 auth_method 0 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.777+0000 7fea14cf1700 10 monclient: get_auth_request con 0x7fe9f403a290 auth_method 0 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 10 client.4250.objecter ms_handle_connect 0x7fe9f403a290 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea14cf1700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f403a290 0x7fe9f403c750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea04005910 tx=0x7fea04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 10 client.4250.objecter ms_handle_connect 0x7fe9f4040120 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0ffff700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7fe9f4040120 0x7fe9f4042540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== osd.1 v2:172.21.15.47:6805/42310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe9f4042bf0 con 0x7fe9f4040120 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 10 client.4250.objecter ms_dispatch 0x7fea10001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 10 client.4250.objecter _finish_command 1 = system:0 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 4 2024-04-20T02:15:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.778+0000 7fea0d7fa700 15 client.4250.objecter _session_command_op_remove 1 1 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 10 monclient: shutdown 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8da01385b0 msgr2=0x7f8da0137130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 --2- 172.21.15.47:0/495817023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8da01385b0 0x7f8da0137130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800b370 tx=0x7f8d98004ad0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da5668700 1 -- 172.21.15.47:0/495817023 reap_dead start 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 shutdown_connections 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 >> 172.21.15.47:0/495817023 conn(0x7f8da00aa8b0 msgr2=0x7f8da01bf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 shutdown_connections 2024-04-20T02:15:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 -- 172.21.15.47:0/495817023 wait complete. 2024-04-20T02:15:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f8da70cb700 1 librados: shutdown 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 client.?.objecter ms_handle_connect 0x7f27dc0721d0 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27cc006e60 con 0x7f27dc0721d0 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: dump: 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc0036a0 con 0x7f27dc0721d0 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 5 monclient: authenticate success, global_id 4357 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 10 monclient: _renew_subs 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc132ac0 con 0x7f27dc0721d0 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f27cc003c80 con 0x7f27dc0721d0 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d1ffb700 10 monclient: dump: 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.909 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.910 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== osd.3 v2:172.21.15.179:6800/42294 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3bc0042b40 con 0x7f3bc0040070 2024-04-20T02:15:47.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f3bd27fc700 10 client.4236.objecter ms_dispatch 0x7f3bdc001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:15:47.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f3bd27fc700 10 client.4236.objecter _finish_command 1 = system:0 2024-04-20T02:15:47.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f3bd27fc700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 4 2024-04-20T02:15:47.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f3bd27fc700 15 client.4236.objecter _session_command_op_remove 3 1 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 10 client.4357.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 10 monclient: _renew_subs 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc131ef0 con 0x7f27dc0721d0 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27e39ab700 1 librados: init done 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.781+0000 7f27d37fe700 4 set_mon_vals no callback set 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.782+0000 7f27e39ab700 10 client.4357.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.782+0000 7f27e39ab700 15 client.4357.objecter _session_command_op_assign -1 1 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.782+0000 7f27e39ab700 20 client.4357.objecter _get_session osd=-1 returning homeless 2024-04-20T02:15:47.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.782+0000 7f27e39ab700 10 client.4357.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.782+0000 7f27e39ab700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.782+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27dc09c7e0 con 0x7f27dc0721d0 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f27cc003810 con 0x7f27dc0721d0 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f27c003a260 0x7f27c003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f27cc04aca0 con 0x7f27dc0721d0 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 10 client.4357.objecter ms_dispatch 0x7f27dc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 3 client.4357.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 3 client.4357.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 10 client.4357.objecter checking command 1 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f27c00400f0 0x7f27c0042510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 20 client.4357.objecter _get_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 15 client.4357.objecter _session_command_op_remove -1 1 2024-04-20T02:15:47.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 20 client.4357.objecter _get_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.792+0000 7f27d1ffb700 20 client.4357.objecter get_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 15 client.4357.objecter _session_command_op_assign 5 1 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 4 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 10 client.4357.objecter _send_command 1 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 --> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27c0042bc0 con 0x7f27c00400f0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 20 client.4357.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27cc04d1f0 con 0x7f27dc0721d0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27e0f46700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f27c003a260 0x7f27c003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27e0f46700 10 monclient: get_auth_request con 0x7f27c003a260 auth_method 0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27e1f48700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f27c00400f0 0x7f27c0042510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27e1f48700 10 monclient: get_auth_request con 0x7f27c00400f0 auth_method 0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27e0f46700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f27c003a260 0x7f27c003c720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0a2610 tx=0x7f27d8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 10 client.4357.objecter ms_handle_connect 0x7f27c003a260 2024-04-20T02:15:47.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27d1ffb700 10 client.4357.objecter ms_handle_connect 0x7f27c00400f0 2024-04-20T02:15:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.793+0000 7f27e1f48700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f27c00400f0 0x7f27c0042510 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.794+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== osd.5 v2:172.21.15.179:6808/42296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f27c0042bc0 con 0x7f27c00400f0 2024-04-20T02:15:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.794+0000 7f27d1ffb700 10 client.4357.objecter ms_dispatch 0x7f27dc001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:15:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.794+0000 7f27d1ffb700 10 client.4357.objecter _finish_command 1 = system:0 2024-04-20T02:15:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.794+0000 7f27d1ffb700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 4 2024-04-20T02:15:47.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.794+0000 7f27d1ffb700 15 client.4357.objecter _session_command_op_remove 5 1 2024-04-20T02:15:47.917 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:47.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 10 client.4347.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 15 client.4347.objecter _session_command_op_assign -1 2 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 20 client.4347.objecter _get_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 20 client.4347.objecter _get_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 15 client.4347.objecter _session_command_op_remove -1 2 2024-04-20T02:15:47.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 20 client.4347.objecter get_session s=0x7f8b4c03f9a0 osd=0 3 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 15 client.4347.objecter _session_command_op_assign 0 2 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 4 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 10 client.4347.objecter _send_command 2 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 --> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b6c143980 con 0x7f8b4c040070 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b637fe700 1 -- 172.21.15.47:0/1686134764 <== osd.0 v2:172.21.15.47:6800/42309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8b6c143980 con 0x7f8b4c040070 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b637fe700 10 client.4347.objecter ms_dispatch 0x7f8b6c001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b637fe700 10 client.4347.objecter _finish_command 2 = system:0 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b637fe700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 4 2024-04-20T02:15:47.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.805+0000 7f8b637fe700 15 client.4347.objecter _session_command_op_remove 0 2 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 10 librados: watch_flush enter 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 10 librados: watch_flush exit 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 10 client.4347.objecter close_session for osd.0 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7f8b4c040070 msgr2=0x7f8b4c042490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7f8b4c040070 0x7f8b4c042490 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 20 client.4347.objecter put_session s=0x7f8b4c03f9a0 osd=0 1 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 20 client.4347.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 10 client.4347.objecter successfully canceled tick 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b4c03a260 msgr2=0x7f8b4c03c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b4c03a260 0x7f8b4c03c720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009920 tx=0x7f8b54008030 comp rx=0 tx=0).stop 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 10 monclient: shutdown 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b6c0a5c90 msgr2=0x7f8b6c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 --2- 172.21.15.47:0/1686134764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b6c0a5c90 0x7f8b6c071880 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c4a0 tx=0x7f8b5c00c7b0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b6b59e700 1 -- 172.21.15.47:0/1686134764 reap_dead start 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 shutdown_connections 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 >> 172.21.15.47:0/1686134764 conn(0x7f8b6c0aa8b0 msgr2=0x7f8b6c0a3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 shutdown_connections 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.806+0000 7f8b71014700 1 -- 172.21.15.47:0/1686134764 wait complete. 2024-04-20T02:15:47.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.807+0000 7f8b71014700 1 librados: shutdown 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 10 client.4250.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 15 client.4250.objecter _session_command_op_assign -1 2 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 20 client.4250.objecter _get_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 20 client.4250.objecter _get_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 15 client.4250.objecter _session_command_op_remove -1 2 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 20 client.4250.objecter get_session s=0x7fe9f403fa50 osd=1 3 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 15 client.4250.objecter _session_command_op_assign 1 2 2024-04-20T02:15:47.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 4 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 10 client.4250.objecter _send_command 2 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.811+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea10068180 con 0x7fe9f4040120 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3114634860 <== osd.1 v2:172.21.15.47:6805/42310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea10068180 con 0x7fe9f4040120 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea0d7fa700 10 client.4250.objecter ms_dispatch 0x7fea10001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea0d7fa700 10 client.4250.objecter _finish_command 2 = system:0 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea0d7fa700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 4 2024-04-20T02:15:47.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea0d7fa700 15 client.4250.objecter _session_command_op_remove 1 2 2024-04-20T02:15:47.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 10 librados: watch_flush enter 2024-04-20T02:15:47.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 10 librados: watch_flush exit 2024-04-20T02:15:47.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 10 client.4250.objecter close_session for osd.1 2024-04-20T02:15:47.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7fe9f4040120 msgr2=0x7fe9f4042540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7fe9f4040120 0x7fe9f4042540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 20 client.4250.objecter put_session s=0x7fe9f403fa50 osd=1 1 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 20 client.4250.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.812+0000 7fea16f55700 10 client.4250.objecter successfully canceled tick 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f403a290 msgr2=0x7fe9f403c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f403a290 0x7fe9f403c750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea04005910 tx=0x7fea04009040 comp rx=0 tx=0).stop 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 10 monclient: shutdown 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea100a06a0 msgr2=0x7fea1012eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 --2- 172.21.15.47:0/3114634860 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea100a06a0 0x7fea1012eb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fea08017000 tx=0x7fea08004ad0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea154f2700 1 -- 172.21.15.47:0/3114634860 reap_dead start 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 shutdown_connections 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 >> 172.21.15.47:0/3114634860 conn(0x7fea100a2f00 msgr2=0x7fea100a5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 shutdown_connections 2024-04-20T02:15:47.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 -- 172.21.15.47:0/3114634860 wait complete. 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.813+0000 7fea16f55700 1 librados: shutdown 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 10 client.4236.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 15 client.4236.objecter _session_command_op_assign -1 2 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 20 client.4236.objecter _get_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 20 client.4236.objecter _get_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 15 client.4236.objecter _session_command_op_remove -1 2 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 20 client.4236.objecter get_session s=0x7f3bc003f9a0 osd=3 3 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 15 client.4236.objecter _session_command_op_assign 3 2 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 4 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 10 client.4236.objecter _send_command 2 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 --> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3bdc09ed70 con 0x7f3bc0040070 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3bd27fc700 1 -- 172.21.15.47:0/2887306240 <== osd.3 v2:172.21.15.179:6800/42294 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3bcc0094a0 con 0x7f3bc0040070 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3bd27fc700 10 client.4236.objecter ms_dispatch 0x7f3bdc001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3bd27fc700 10 client.4236.objecter _finish_command 2 = system:0 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3bd27fc700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 4 2024-04-20T02:15:47.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.815+0000 7f3bd27fc700 15 client.4236.objecter _session_command_op_remove 3 2 2024-04-20T02:15:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 10 librados: watch_flush enter 2024-04-20T02:15:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 10 librados: watch_flush exit 2024-04-20T02:15:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 10 client.4236.objecter close_session for osd.3 2024-04-20T02:15:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f3bc0040070 msgr2=0x7f3bc0042490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f3bc0040070 0x7f3bc0042490 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 20 client.4236.objecter put_session s=0x7f3bc003f9a0 osd=3 1 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 20 client.4236.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 10 client.4236.objecter successfully canceled tick 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3bc003a260 msgr2=0x7f3bc003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3bc003a260 0x7f3bc003c720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80090b0 tx=0x7f3bd8012040 comp rx=0 tx=0).stop 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 10 monclient: shutdown 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3bdc072030 msgr2=0x7f3bdc1d9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 --2- 172.21.15.47:0/2887306240 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3bdc072030 0x7f3bdc1d9420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400cba0 tx=0x7f3bd400ceb0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be2685700 1 -- 172.21.15.47:0/2887306240 reap_dead start 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 shutdown_connections 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 >> 172.21.15.47:0/2887306240 conn(0x7f3bdc0a39d0 msgr2=0x7f3bdc0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 shutdown_connections 2024-04-20T02:15:47.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.818+0000 7f3be40e8700 1 -- 172.21.15.47:0/2887306240 wait complete. 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f3be40e8700 1 librados: shutdown 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 10 client.4357.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 15 client.4357.objecter _session_command_op_assign -1 2 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 20 client.4357.objecter _get_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 20 client.4357.objecter _get_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 15 client.4357.objecter _session_command_op_remove -1 2 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 20 client.4357.objecter get_session s=0x7f27c003fa20 osd=5 3 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 15 client.4357.objecter _session_command_op_assign 5 2 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 4 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 10 client.4357.objecter _send_command 2 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.819+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 --> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f27dc06aac0 con 0x7f27c00400f0 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27d1ffb700 1 -- 172.21.15.47:0/269563054 <== osd.5 v2:172.21.15.179:6808/42296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f27d4008040 con 0x7f27c00400f0 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27d1ffb700 10 client.4357.objecter ms_dispatch 0x7f27dc001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27d1ffb700 10 client.4357.objecter _finish_command 2 = system:0 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27d1ffb700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 4 2024-04-20T02:15:47.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27d1ffb700 15 client.4357.objecter _session_command_op_remove 5 2 2024-04-20T02:15:47.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 10 librados: watch_flush enter 2024-04-20T02:15:47.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 10 librados: watch_flush exit 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 10 client.4357.objecter close_session for osd.5 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f27c00400f0 msgr2=0x7f27c0042510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f27c00400f0 0x7f27c0042510 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 20 client.4357.objecter put_session s=0x7f27c003fa20 osd=5 1 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 20 client.4357.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 10 client.4357.objecter successfully canceled tick 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f27c003a260 msgr2=0x7f27c003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f27c003a260 0x7f27c003c720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0a2610 tx=0x7f27d8009190 comp rx=0 tx=0).stop 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 10 monclient: shutdown 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27dc0721d0 msgr2=0x7f27dc1e9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e39ab700 1 --2- 172.21.15.47:0/269563054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27dc0721d0 0x7f27dc1e9b70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f27cc019000 tx=0x7f27cc004750 comp rx=0 tx=0).stop 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.820+0000 7f27e1f48700 1 -- 172.21.15.47:0/269563054 reap_dead start 2024-04-20T02:15:47.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.822+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 shutdown_connections 2024-04-20T02:15:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.822+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 >> 172.21.15.47:0/269563054 conn(0x7f27dc0a6210 msgr2=0x7f27dc09deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.822+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 shutdown_connections 2024-04-20T02:15:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.822+0000 7f27e39ab700 1 -- 172.21.15.47:0/269563054 wait complete. 2024-04-20T02:15:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.822+0000 7f27e39ab700 1 librados: shutdown 2024-04-20T02:15:47.935 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:47.936 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T02:15:47.936 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:47.937 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.842+0000 7f0f7effd700 1 -- 172.21.15.47:0/2537520360 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f68011aa0 con 0x7f0f8013c4d0 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.842+0000 7f0f7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.842+0000 7f0f7effd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.842+0000 7f0f7effd700 10 monclient: dump: 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.938 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 1 -- 172.21.15.47:0/2537520360 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68011c00 con 0x7f0f8013c4d0 2024-04-20T02:15:47.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f85b7d700 5 monclient: authenticate success, global_id 4247 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f85b7d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f85b7d700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 1 -- 172.21.15.47:0/2537520360 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f68011d70 con 0x7f0f8013c4d0 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f85b7d700 4 set_mon_vals no callback set 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.843+0000 7f0f7effd700 10 monclient: dump: 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.940 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.941 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.942 INFO:teuthology.orchestra.run.smithi047.stdout:51539607555 2024-04-20T02:15:47.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 10 monclient: shutdown 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 -- 172.21.15.47:0/2537520360 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f8013c4d0 msgr2=0x7f0f8013c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 --2- 172.21.15.47:0/2537520360 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f8013c4d0 0x7f0f8013c8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009840 tx=0x7f0f68009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 -- 172.21.15.47:0/2537520360 shutdown_connections 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 --2- 172.21.15.47:0/2537520360 >> v2:172.21.15.179:3300/0 conn(0x7f0f8013c0f0 0x7f0f8009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 --2- 172.21.15.47:0/2537520360 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f8013c4d0 0x7f0f8013c8d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 -- 172.21.15.47:0/2537520360 >> 172.21.15.47:0/2537520360 conn(0x7f0f800a6210 msgr2=0x7f0f800a6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 -- 172.21.15.47:0/2537520360 shutdown_connections 2024-04-20T02:15:47.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.844+0000 7f0f85b7d700 1 -- 172.21.15.47:0/2537520360 wait complete. 2024-04-20T02:15:47.944 INFO:teuthology.orchestra.run.smithi047.stdout:47244640260 2024-04-20T02:15:47.944 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 10 monclient: build_initial_monmap 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 10 monclient: monmap: 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:47.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 1 librados: starting msgr at 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 1 librados: starting objecter 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 1 Processor -- start 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 1 -- start start 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 1 librados: setting wanted keys 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 1 librados: calling monclient init 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.845+0000 7f0f85b7d700 10 monclient: init 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:47.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient: _add_conns ranks=[0,1,4,5,8,6,2,7,3] 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f800a1060 0x7f0f801444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): picked mon.noname-a con 0x7f0f800a1060 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f8013c0f0 0x7f0f8012da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): picked mon.noname-f con 0x7f0f8013c0f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0f8013c4d0 0x7f0f8012dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): picked mon.noname-c con 0x7f0f8013c4d0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80143b10 con 0x7f0f800a1060 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80143d60 con 0x7f0f8013c0f0 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8009b590 con 0x7f0f8013c4d0 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f85b7d700 10 monclient(hunting): authenticate will time out at 1627.247258s 2024-04-20T02:15:47.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0f8013c4d0 0x7f0f8012dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f800a1060 0x7f0f801444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f8013c0f0 0x7f0f8012da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f8013c0f0 0x7f0f8012da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:48150/0 (socket says 172.21.15.47:48150) 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0f8013c4d0 0x7f0f8012dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44430/0 (socket says 172.21.15.47:44430) 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 -- 172.21.15.47:0/1322382198 learned_addr learned my addr 172.21.15.47:0/1322382198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): get_auth_request con 0x7f0f8013c0f0 auth_method 0 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f800a1060 0x7f0f801444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56918/0 (socket says 172.21.15.47:56918) 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): get_auth_request con 0x7f0f800a1060 auth_method 0 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): get_auth_request con 0x7f0f8013c4d0 auth_method 0 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f7f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient(hunting): handle_auth_done global_id 4366 payload 824 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 -- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0f8013c4d0 msgr2=0x7f0f8012dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0f8013c4d0 0x7f0f8012dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 -- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f800a1060 msgr2=0x7f0f801444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f800a1060 0x7f0f801444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 monclient: found mon.noname-f 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.846+0000 7f0f76ffd700 1 -- 172.21.15.47:0/1322382198 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680094a0 con 0x7f0f8013c0f0 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f76ffd700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f8013c0f0 0x7f0f8012da80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f68004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 client.?.objecter ms_handle_connect 0x7f0f8013c0f0 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f68011a00 con 0x7f0f8013c0f0 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: dump: 2024-04-20T02:15:47.955 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.956 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6801f070 con 0x7f0f8013c0f0 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f68004370 con 0x7f0f8013c0f0 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 5 monclient: authenticate success, global_id 4366 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 10 monclient: _renew_subs 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7e7fc700 4 set_mon_vals no callback set 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f80072260 con 0x7f0f8013c0f0 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f7cff9700 10 monclient: dump: 2024-04-20T02:15:47.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:47.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 10 client.4366.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 10 monclient: _renew_subs 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f80141990 con 0x7f0f8013c0f0 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.847+0000 7f0f85b7d700 1 librados: init done 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f85b7d700 10 client.4366.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f85b7d700 15 client.4366.objecter _session_command_op_assign -1 1 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f85b7d700 20 client.4366.objecter _get_session osd=-1 returning homeless 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f85b7d700 10 client.4366.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f85b7d700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f8009c310 con 0x7f0f8013c0f0 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f0f68021030 con 0x7f0f8013c0f0 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f6003a260 0x7f0f6003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f0f6804b130 con 0x7f0f8013c0f0 2024-04-20T02:15:47.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 10 client.4366.objecter ms_dispatch 0x7f0f80001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 3 client.4366.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 3 client.4366.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7f7fe700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f6003a260 0x7f0f6003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7f7fe700 10 monclient: get_auth_request con 0x7f0f6003a260 auth_method 0 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.848+0000 7f0f7cff9700 10 client.4366.objecter checking command 1 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f0f60040070 0x7f0f60042490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 20 client.4366.objecter _get_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 15 client.4366.objecter _session_command_op_remove -1 1 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 20 client.4366.objecter _get_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 20 client.4366.objecter get_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 15 client.4366.objecter _session_command_op_assign 2 1 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 4 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 10 client.4366.objecter _send_command 1 2024-04-20T02:15:47.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 --> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f60042b40 con 0x7f0f60040070 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 20 client.4366.objecter dump_active .. 0 homeless 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f6804d5e0 con 0x7f0f8013c0f0 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7ffff700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f0f60040070 0x7f0f60042490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7ffff700 10 monclient: get_auth_request con 0x7f0f60040070 auth_method 0 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7f7fe700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f6003a260 0x7f0f6003c720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0f700113e0 tx=0x7f0f7000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 10 client.4366.objecter ms_handle_connect 0x7f0f6003a260 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7cff9700 10 client.4366.objecter ms_handle_connect 0x7f0f60040070 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.849+0000 7f0f7ffff700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f0f60040070 0x7f0f60042490 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.850+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== osd.2 v2:172.21.15.47:6808/42311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0f60042b40 con 0x7f0f60040070 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.850+0000 7f0f7cff9700 10 client.4366.objecter ms_dispatch 0x7f0f80001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.850+0000 7f0f7cff9700 10 client.4366.objecter _finish_command 1 = system:0 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.850+0000 7f0f7cff9700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 4 2024-04-20T02:15:47.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.850+0000 7f0f7cff9700 15 client.4366.objecter _session_command_op_remove 2 1 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.868+0000 7f0f85b7d700 10 client.4366.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 15 client.4366.objecter _session_command_op_assign -1 2 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter _get_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter _get_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 15 client.4366.objecter _session_command_op_remove -1 2 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter get_session s=0x7f0f6003f9a0 osd=2 3 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 15 client.4366.objecter _session_command_op_assign 2 2 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 4 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 10 client.4366.objecter _send_command 2 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 --> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f8009afd0 con 0x7f0f60040070 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f7cff9700 1 -- 172.21.15.47:0/1322382198 <== osd.2 v2:172.21.15.47:6808/42311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0f78004b70 con 0x7f0f60040070 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f7cff9700 10 client.4366.objecter ms_dispatch 0x7f0f80001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f7cff9700 10 client.4366.objecter _finish_command 2 = system:0 2024-04-20T02:15:47.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f7cff9700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 4 2024-04-20T02:15:47.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f7cff9700 15 client.4366.objecter _session_command_op_remove 2 2 2024-04-20T02:15:47.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 10 librados: watch_flush enter 2024-04-20T02:15:47.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 10 librados: watch_flush exit 2024-04-20T02:15:47.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 10 client.4366.objecter close_session for osd.2 2024-04-20T02:15:47.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f0f60040070 msgr2=0x7f0f60042490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f0f60040070 0x7f0f60042490 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter put_session s=0x7f0f6003f9a0 osd=2 1 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 client.4366.objecter shutdown clearing up homeless session... 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 10 client.4366.objecter successfully canceled tick 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f6003a260 msgr2=0x7f0f6003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f6003a260 0x7f0f6003c720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0f700113e0 tx=0x7f0f7000b040 comp rx=0 tx=0).stop 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 10 monclient: shutdown 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f8013c0f0 msgr2=0x7f0f8012da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 --2- 172.21.15.47:0/1322382198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f8013c0f0 0x7f0f8012da80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f68000c00 tx=0x7f0f68004750 comp rx=0 tx=0).stop 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f7ffff700 1 -- 172.21.15.47:0/1322382198 reap_dead start 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 shutdown_connections 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 >> 172.21.15.47:0/1322382198 conn(0x7f0f800a6210 msgr2=0x7f0f8009e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 shutdown_connections 2024-04-20T02:15:47.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.869+0000 7f0f85b7d700 1 -- 172.21.15.47:0/1322382198 wait complete. 2024-04-20T02:15:47.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:47.870+0000 7f0f85b7d700 1 librados: shutdown 2024-04-20T02:15:47.970 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:47.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f8f03b52700 10 monclient: get_monmap_and_config 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f8f03b52700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f8f03b52700 10 monclient: monmap: 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:48.151214+0000 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:48.151214+0000 2024-04-20T02:15:48.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:48.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f2658e0e700 10 monclient: get_monmap_and_config 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f2658e0e700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f2658e0e700 10 monclient: monmap: 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:48.151271+0000 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:48.151271+0000 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.152 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f2658e0e700 1 Processor -- start 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f2658e0e700 1 -- start start 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.150+0000 7f2658e0e700 10 monclient: init 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.151+0000 7f2658e0e700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.151+0000 7f2658e0e700 10 monclient: _add_conns ranks=[3,8,2,9,7,10,6,5,4,0,1] 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 0x7f265409ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): picked mon.noname-g con 0x7f26540721d0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f265409f930 0x7f2654133990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): picked mon.noname-e con 0x7f265409f930 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f265409fd30 0x7f2654133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): picked mon.noname-b con 0x7f265409fd30 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265409cb50 con 0x7f265409fd30 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26540725b0 con 0x7f26540721d0 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f26540a2030 con 0x7f265409f930 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f2658e0e700 10 monclient(hunting): authenticate will time out at 1627.553115s 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 0x7f265409ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 0x7f265409ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:48776/0 (socket says 172.21.15.47:48776) 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 -- 172.21.15.47:0/244027553 learned_addr learned my addr 172.21.15.47:0/244027553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): get_auth_request con 0x7f26540721d0 auth_method 0 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient(hunting): handle_auth_done global_id 4278 payload 274 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 -- 172.21.15.47:0/244027553 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f265409f930 msgr2=0x7f2654133990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f265409f930 0x7f2654133990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 -- 172.21.15.47:0/244027553 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f265409fd30 msgr2=0x7f2654133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f265409fd30 0x7f2654133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 monclient: found mon.noname-g 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 -- 172.21.15.47:0/244027553 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540751e0 con 0x7f26540721d0 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.152+0000 7f265259c700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 0x7f265409ae40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2644008950 tx=0x7f2644008c60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=1b030766dbaa3622 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f8f03b52700 1 Processor -- start 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 1 -- 172.21.15.47:0/244027553 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f26440124d0 con 0x7f26540721d0 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: dump: 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.157 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.158 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.158 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 1 -- 172.21.15.47:0/244027553 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2644012630 con 0x7f26540721d0 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 1 -- 172.21.15.47:0/244027553 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2644012c10 con 0x7f26540721d0 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f265159a700 10 monclient: dump: 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.159 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f2658e0e700 5 monclient: authenticate success, global_id 4278 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f2658e0e700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:48.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f2658e0e700 4 set_mon_vals no callback set 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f8f03b52700 1 -- start start 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.153+0000 7f8f03b52700 10 monclient: init 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient: _add_conns ranks=[2,10,7,9,4,6,0,3,8,1,5] 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc0a3960 0x7f8efc06ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): picked mon.noname-b con 0x7f8efc0a3960 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:48.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8efc06d370 0x7f8efc06d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): picked mon.noname-f con 0x7f8efc06d370 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8efc137ab0 0x7f8efc137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): picked mon.noname-i con 0x7f8efc137ab0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc0a4900 con 0x7f8efc06d370 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc06e7a0 con 0x7f8efc0a3960 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0a3d40 con 0x7f8efc137ab0 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f03b52700 10 monclient(hunting): authenticate will time out at 1627.555262s 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 10 monclient: shutdown 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 -- 172.21.15.47:0/244027553 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 msgr2=0x7f265409ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 0x7f265409ae40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2644008950 tx=0x7f2644008c60 comp rx=0 tx=0).stop 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 -- 172.21.15.47:0/244027553 shutdown_connections 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f265409fd30 0x7f2654133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f26540721d0 0x7f265409ae40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 --2- 172.21.15.47:0/244027553 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f265409f930 0x7f2654133990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f2658e0e700 1 -- 172.21.15.47:0/244027553 >> 172.21.15.47:0/244027553 conn(0x7f26540a6210 msgr2=0x7f26540a6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc0a3960 0x7f8efc06ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc0a3960 0x7f8efc06ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53048/0 (socket says 172.21.15.47:53048) 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 1 -- 172.21.15.47:0/140152448 learned_addr learned my addr 172.21.15.47:0/140152448 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): get_auth_request con 0x7f8efc0a3960 auth_method 0 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 --2- 172.21.15.47:0/140152448 >> v2:172.21.15.179:3300/0 conn(0x7f8efc06d370 0x7f8efc06d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): get_auth_request con 0x7f8efc06d370 auth_method 0 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f020ef700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8efc137ab0 0x7f8efc137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f020ef700 10 monclient(hunting): get_auth_request con 0x7f8efc137ab0 auth_method 0 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f020ef700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f020ef700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f020ef700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f018ee700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient(hunting): handle_auth_done global_id 4375 payload 274 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 -- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8efc137ab0 msgr2=0x7f8efc137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8efc137ab0 0x7f8efc137f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 -- 172.21.15.47:0/140152448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc0a3960 msgr2=0x7f8efc06ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc0a3960 0x7f8efc06ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 monclient: found mon.noname-f 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.154+0000 7f8f010ed700 1 -- 172.21.15.47:0/140152448 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0751e0 con 0x7f8efc06d370 2024-04-20T02:15:48.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f2658e0e700 1 -- 172.21.15.47:0/244027553 shutdown_connections 2024-04-20T02:15:48.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f010ed700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8efc06d370 0x7f8efc06d770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009840 tx=0x7f8eec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cb8ae9aaebfafa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 1 -- 172.21.15.47:0/140152448 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8eec011aa0 con 0x7f8efc06d370 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: dump: 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.170 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f03b52700 5 monclient: authenticate success, global_id 4375 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 1 -- 172.21.15.47:0/140152448 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec011c00 con 0x7f8efc06d370 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f03b52700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f03b52700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f03b52700 4 set_mon_vals no callback set 2024-04-20T02:15:48.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 1 -- 172.21.15.47:0/140152448 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8eec011d70 con 0x7f8efc06d370 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f8f008ec700 10 monclient: dump: 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.172 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 10 monclient: shutdown 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 -- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8efc06d370 msgr2=0x7f8efc06d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8efc06d370 0x7f8efc06d770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009840 tx=0x7f8eec009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 -- 172.21.15.47:0/140152448 shutdown_connections 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8efc137ab0 0x7f8efc137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc0a3960 0x7f8efc06ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 --2- 172.21.15.47:0/140152448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8efc06d370 0x7f8efc06d770 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009840 tx=0x7f8eec009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 -- 172.21.15.47:0/140152448 >> 172.21.15.47:0/140152448 conn(0x7f8efc0a4a50 msgr2=0x7f8efc0a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 -- 172.21.15.47:0/140152448 shutdown_connections 2024-04-20T02:15:48.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f8f03b52700 1 -- 172.21.15.47:0/140152448 wait complete. 2024-04-20T02:15:48.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.155+0000 7f2658e0e700 1 -- 172.21.15.47:0/244027553 wait complete. 2024-04-20T02:15:48.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f2658e0e700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f2658e0e700 10 monclient: monmap: 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.176 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f2658e0e700 1 librados: starting msgr at 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.156+0000 7f2658e0e700 1 librados: starting objecter 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 1 Processor -- start 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 1 -- start start 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 1 librados: setting wanted keys 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 1 librados: calling monclient init 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 10 monclient: init 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 10 monclient: _add_conns ranks=[1,3,4,0,7,8,6,2,5] 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 0x7f26541d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f2658e0e700 10 monclient(hunting): picked mon.noname-f con 0x7f26540721d0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:48.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f265409f930 0x7f26541d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): picked mon.noname-g con 0x7f265409f930 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f265409fd30 0x7f2654124cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): picked mon.noname-c con 0x7f265409fd30 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654130850 con 0x7f26540721d0 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265409ba60 con 0x7f265409f930 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26541285b0 con 0x7f265409fd30 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2658e0e700 10 monclient(hunting): authenticate will time out at 1627.559098s 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 0x7f26541d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 0x7f26541d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:48168/0 (socket says 172.21.15.47:48168) 2024-04-20T02:15:48.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 -- 172.21.15.47:0/2738897678 learned_addr learned my addr 172.21.15.47:0/2738897678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): get_auth_request con 0x7f26540721d0 auth_method 0 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2651d9b700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f265409f930 0x7f26541d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2651d9b700 10 monclient(hunting): get_auth_request con 0x7f265409f930 auth_method 0 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2651d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2651d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f2651d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient(hunting): handle_auth_done global_id 4384 payload 824 2024-04-20T02:15:48.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 -- 172.21.15.47:0/2738897678 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f265409fd30 msgr2=0x7f2654124cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f265409fd30 0x7f2654124cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 -- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f265409f930 msgr2=0x7f26541d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f265409f930 0x7f26541d12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 monclient: found mon.noname-f 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f265259c700 1 -- 172.21.15.47:0/2738897678 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440085d0 con 0x7f26540721d0 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f265259c700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 0x7f26541d0db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26540726f0 tx=0x7f2644004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 client.?.objecter ms_handle_connect 0x7f26540721d0 2024-04-20T02:15:48.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f8f03b52700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f8f03b52700 10 monclient: monmap: 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:48.181 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f8f03b52700 1 librados: starting msgr at 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.157+0000 7f8f03b52700 1 librados: starting objecter 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 Processor -- start 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 -- start start 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 librados: setting wanted keys 2024-04-20T02:15:48.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 librados: calling monclient init 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient: init 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient: _add_conns ranks=[4,0,2,8,3,6,7,5,1] 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8efc0a3960 0x7f8efc12c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): picked mon.noname-c con 0x7f8efc0a3960 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8efc137ab0 0x7f8efc08b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): picked mon.noname-a con 0x7f8efc137ab0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc07c930 0x7f8efc12ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): picked mon.noname-b con 0x7f8efc07c930 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc12bdc0 con 0x7f8efc137ab0 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc07c380 con 0x7f8efc07c930 2024-04-20T02:15:48.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc134660 con 0x7f8efc0a3960 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.158+0000 7f8f03b52700 10 monclient(hunting): authenticate will time out at 1627.559819s 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc07c930 0x7f8efc12ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc07c930 0x7f8efc12ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53056/0 (socket says 172.21.15.47:53056) 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2644012c90 con 0x7f26540721d0 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: dump: 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.184 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2644012df0 con 0x7f26540721d0 2024-04-20T02:15:48.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2644012250 con 0x7f26540721d0 2024-04-20T02:15:48.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 1 -- 172.21.15.47:0/4019782992 learned_addr learned my addr 172.21.15.47:0/4019782992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): get_auth_request con 0x7f8efc07c930 auth_method 0 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8efc0a3960 0x7f8efc12c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): get_auth_request con 0x7f8efc0a3960 auth_method 0 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f020ef700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient(hunting): handle_auth_done global_id 4297 payload 824 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 -- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc07c930 msgr2=0x7f8efc12ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc07c930 0x7f8efc12ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 -- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8efc137ab0 msgr2=0x7f8efc08b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8efc137ab0 0x7f8efc08b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 monclient: found mon.noname-c 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:48.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f8f018ee700 1 -- 172.21.15.47:0/4019782992 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec0094a0 con 0x7f8efc0a3960 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2642ffd700 10 monclient: dump: 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.159+0000 7f2650d99700 4 set_mon_vals no callback set 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 5 monclient: authenticate success, global_id 4384 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 10 monclient: _renew_subs 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2654128f90 con 0x7f26540721d0 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 10 client.4384.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:48.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 10 monclient: _renew_subs 2024-04-20T02:15:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2654128b70 con 0x7f26540721d0 2024-04-20T02:15:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.160+0000 7f2658e0e700 1 librados: init done 2024-04-20T02:15:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f8f018ee700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8efc0a3960 0x7f8efc12c130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800e910 tx=0x7f8ef800ec20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f8ef27fc700 10 client.?.objecter ms_handle_connect 0x7f8efc0a3960 2024-04-20T02:15:48.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f8ef27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f2644012820 con 0x7f26540721d0 2024-04-20T02:15:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f263403a260 0x7f263403c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f264404b5f0 con 0x7f26540721d0 2024-04-20T02:15:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 10 client.4384.objecter ms_dispatch 0x7f2654001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:48.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 3 client.4384.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:48.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 3 client.4384.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:48.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2642ffd700 20 client.4384.objecter dump_active .. 0 homeless 2024-04-20T02:15:48.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2651d9b700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f263403a260 0x7f263403c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.161+0000 7f2651d9b700 10 monclient: get_auth_request con 0x7f263403a260 auth_method 0 2024-04-20T02:15:48.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.162+0000 7f2642ffd700 10 client.4384.objecter ms_handle_connect 0x7f263403a260 2024-04-20T02:15:48.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.162+0000 7f2651d9b700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f263403a260 0x7f263403c720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2648006fd0 tx=0x7f2648008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ef800ee60 con 0x7f8efc0a3960 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 10 monclient: dump: 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.195 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef800cc60 con 0x7f8efc0a3960 2024-04-20T02:15:48.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 5 monclient: authenticate success, global_id 4297 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 10 monclient: _renew_subs 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc0720f0 con 0x7f8efc0a3960 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 10 client.4297.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 10 monclient: _renew_subs 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc07d950 con 0x7f8efc0a3960 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.163+0000 7f8f03b52700 1 librados: init done 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8ef27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ef800cdd0 con 0x7f8efc0a3960 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8ef27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8ef27fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:48.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8f03b52700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8ef27fc700 10 monclient: dump: 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.198 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8f03b52700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8f03b52700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc067f90 con 0x7f8efc0a3960 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.164+0000 7f8ef3fff700 4 set_mon_vals no callback set 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f8ef800c7f0 con 0x7f8efc0a3960 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8ee003a260 0x7f8ee003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8ef8013490 con 0x7f8efc0a3960 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 10 client.4297.objecter ms_dispatch 0x7f8efc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 3 client.4297.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 3 client.4297.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.165+0000 7f8ef27fc700 20 client.4297.objecter dump_active .. 0 homeless 2024-04-20T02:15:48.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.166+0000 7f8f010ed700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8ee003a260 0x7f8ee003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.166+0000 7f8f010ed700 10 monclient: get_auth_request con 0x7f8ee003a260 auth_method 0 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.166+0000 7f8ef27fc700 10 client.4297.objecter ms_handle_connect 0x7f8ee003a260 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.166+0000 7f8f010ed700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8ee003a260 0x7f8ee003c720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009840 tx=0x7f8eec008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.170+0000 7f2658e0e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.170+0000 7f2658e0e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.170+0000 7f2658e0e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.170+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265406a620 con 0x7f26540721d0 2024-04-20T02:15:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.171+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f8ef801b9a0 con 0x7f8efc0a3960 2024-04-20T02:15:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.171+0000 7f8ef27fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.171+0000 7f8ef27fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.175+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2644016020 con 0x7f26540721d0 2024-04-20T02:15:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.175+0000 7f2642ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.175+0000 7f2642ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.193+0000 7f5856c41700 10 monclient: get_monmap_and_config 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.193+0000 7f5856c41700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.193+0000 7f5856c41700 10 monclient: monmap: 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:48.194773+0000 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:48.194773+0000 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:48.204 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.194+0000 7f5856c41700 1 Processor -- start 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.194+0000 7f5856c41700 1 -- start start 2024-04-20T02:15:48.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.194+0000 7f5856c41700 10 monclient: init 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient: _add_conns ranks=[2,3,5,10,4,1,9,0,6,7,8] 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850137d90 0x7f58500a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): picked mon.noname-b con 0x7f5850137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5850138170 0x7f58500a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): picked mon.noname-g con 0x7f5850138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58501385b0 0x7f58500a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): picked mon.noname-h con 0x7f58501385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585009fa40 con 0x7f5850137d90 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585009dfc0 con 0x7f5850138170 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58500a7bc0 con 0x7f58501385b0 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f5856c41700 10 monclient(hunting): authenticate will time out at 1627.596702s 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.195+0000 7f58549dd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850137d90 0x7f58500a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58549dd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850137d90 0x7f58500a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53070/0 (socket says 172.21.15.47:53070) 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58549dd700 1 -- 172.21.15.47:0/1319699645 learned_addr learned my addr 172.21.15.47:0/1319699645 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58549dd700 10 monclient(hunting): get_auth_request con 0x7f5850137d90 auth_method 0 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58549dd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58549dd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58549dd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58501385b0 0x7f58500a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5850138170 0x7f58500a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): get_auth_request con 0x7f58501385b0 auth_method 0 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): get_auth_request con 0x7f5850138170 auth_method 0 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f58551de700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient(hunting): handle_auth_done global_id 4296 payload 274 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 -- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58501385b0 msgr2=0x7f58500a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58501385b0 0x7f58500a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 -- 172.21.15.47:0/1319699645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850137d90 msgr2=0x7f58500a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850137d90 0x7f58500a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 monclient: found mon.noname-g 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:15:48.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 -- 172.21.15.47:0/1319699645 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500751e0 con 0x7f5850138170 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584ffff700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5850138170 0x7f58500a5c90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f583c0098a0 tx=0x7f583c009bb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3c4c3d2c71583583 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.196+0000 7f584f7fe700 1 -- 172.21.15.47:0/1319699645 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583c011aa0 con 0x7f5850138170 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: dump: 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.210 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 1 -- 172.21.15.47:0/1319699645 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c011c00 con 0x7f5850138170 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 1 -- 172.21.15.47:0/1319699645 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583c011d70 con 0x7f5850138170 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f584f7fe700 10 monclient: dump: 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.212 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f5856c41700 5 monclient: authenticate success, global_id 4296 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f5856c41700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.197+0000 7f5856c41700 4 set_mon_vals no callback set 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 10 monclient: shutdown 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 -- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5850138170 msgr2=0x7f58500a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5850138170 0x7f58500a5c90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f583c0098a0 tx=0x7f583c009bb0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 -- 172.21.15.47:0/1319699645 shutdown_connections 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58501385b0 0x7f58500a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850137d90 0x7f58500a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 --2- 172.21.15.47:0/1319699645 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5850138170 0x7f58500a5c90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 -- 172.21.15.47:0/1319699645 >> 172.21.15.47:0/1319699645 conn(0x7f58500aa8b0 msgr2=0x7f58500a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 -- 172.21.15.47:0/1319699645 shutdown_connections 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.198+0000 7f5856c41700 1 -- 172.21.15.47:0/1319699645 wait complete. 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.199+0000 7f5856c41700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.199+0000 7f5856c41700 10 monclient: monmap: 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.214 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.199+0000 7f5856c41700 1 librados: starting msgr at 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.199+0000 7f5856c41700 1 librados: starting objecter 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.199+0000 7f5856c41700 1 Processor -- start 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 -- start start 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 librados: setting wanted keys 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 librados: calling monclient init 2024-04-20T02:15:48.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient: init 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f628998e700 10 monclient: get_monmap_and_config 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f628998e700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f628998e700 10 monclient: monmap: 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:48.201912+0000 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:48.201912+0000 2024-04-20T02:15:48.216 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.217 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:48.218 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:48.218 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient: _add_conns ranks=[6,8,1,4,0,5,3,2,7] 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5850137d90 0x7f5850136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): picked mon.noname-d con 0x7f5850137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5850138170 0x7f5850137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): picked mon.noname-e con 0x7f5850138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58501385b0 0x7f58501318e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): picked mon.noname-f con 0x7f58501385b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5850132cc0 con 0x7f58501385b0 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f585009e9e0 con 0x7f5850137d90 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f585009dc20 con 0x7f5850138170 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.200+0000 7f5856c41700 10 monclient(hunting): authenticate will time out at 1627.601902s 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58501385b0 0x7f58501318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58501385b0 0x7f58501318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:48180/0 (socket says 172.21.15.47:48180) 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 1 -- 172.21.15.47:0/2467968587 learned_addr learned my addr 172.21.15.47:0/2467968587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): get_auth_request con 0x7f58501385b0 auth_method 0 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5850138170 0x7f5850137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): get_auth_request con 0x7f5850138170 auth_method 0 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f58551de700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 1 -- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5850137d90 msgr2=0x7f5850136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5850137d90 0x7f5850136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 1 -- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58501385b0 msgr2=0x7f58501318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58501385b0 0x7f58501318e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.201+0000 7f584ffff700 1 monclient: found mon.noname-e 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584ffff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584ffff700 1 -- 172.21.15.47:0/2467968587 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0094a0 con 0x7f5850138170 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584ffff700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5850138170 0x7f5850137110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f583c016000 tx=0x7f583c004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 client.?.objecter ms_handle_connect 0x7f5850138170 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583c011a60 con 0x7f5850138170 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:48.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: dump: 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c01f070 con 0x7f5850138170 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583c004520 con 0x7f5850138170 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f584d7fa700 10 monclient: dump: 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.226 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.227 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 5 monclient: authenticate success, global_id 4265 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 10 monclient: _renew_subs 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5850134690 con 0x7f5850138170 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 10 client.4265.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 10 monclient: _renew_subs 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58500723b0 con 0x7f5850138170 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.202+0000 7f5856c41700 1 librados: init done 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.203+0000 7f5856c41700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.203+0000 7f5856c41700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.203+0000 7f5856c41700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:48.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.203+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585009e430 con 0x7f5850138170 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.203+0000 7f584effd700 4 set_mon_vals no callback set 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f583c021030 con 0x7f5850138170 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a290 0x7f583403c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f583c04b260 con 0x7f5850138170 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 10 client.4265.objecter ms_dispatch 0x7f5850001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 3 client.4265.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 3 client.4265.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f584d7fa700 20 client.4265.objecter dump_active .. 0 homeless 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f58549dd700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a290 0x7f583403c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f58549dd700 10 monclient: get_auth_request con 0x7f583403a290 auth_method 0 2024-04-20T02:15:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f628998e700 1 Processor -- start 2024-04-20T02:15:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f628998e700 1 -- start start 2024-04-20T02:15:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.204+0000 7f628998e700 10 monclient: init 2024-04-20T02:15:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient: _add_conns ranks=[10,1,4,0,3,2,9,6,7,8,5] 2024-04-20T02:15:48.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6284137d90 0x7f62840a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): picked mon.noname-f con 0x7f6284137d90 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6284138170 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 0x7f62840a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): picked mon.noname-c con 0x7f62841385b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840751e0 con 0x7f6284138170 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628409fa40 con 0x7f6284137d90 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f628409dfc0 con 0x7f62841385b0 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.205+0000 7f628998e700 10 monclient(hunting): authenticate will time out at 1627.606878s 2024-04-20T02:15:48.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 0x7f62840a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f58549dd700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a290 0x7f583403c750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58440098f0 tx=0x7f5844008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f584d7fa700 10 client.4265.objecter ms_handle_connect 0x7f583403a290 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 0x7f62840a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44462/0 (socket says 172.21.15.47:44462) 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 -- 172.21.15.47:0/623928157 learned_addr learned my addr 172.21.15.47:0/623928157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): get_auth_request con 0x7f62841385b0 auth_method 0 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient(hunting): handle_auth_done global_id 4306 payload 274 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 -- 172.21.15.47:0/623928157 >> v2:172.21.15.179:3300/0 conn(0x7f6284137d90 msgr2=0x7f62840a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 --2- 172.21.15.47:0/623928157 >> v2:172.21.15.179:3300/0 conn(0x7f6284137d90 0x7f62840a09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 -- 172.21.15.47:0/623928157 >> v1:172.21.15.179:6789/0 conn(0x7f6284138170 legacy=0x7f62840a5c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 monclient: found mon.noname-c 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 -- 172.21.15.47:0/623928157 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840a7bc0 con 0x7f62841385b0 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.206+0000 7f62837fe700 1 --2- 172.21.15.47:0/623928157 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 0x7f62840a63a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62780098f0 tx=0x7f6278009c00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=fcc4b57139b95cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 1 -- 172.21.15.47:0/623928157 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f62780044a0 con 0x7f62841385b0 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:48.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: dump: 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.235 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 5 monclient: authenticate success, global_id 4306 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 1 -- 172.21.15.47:0/623928157 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62780036a0 con 0x7f62841385b0 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 4 set_mon_vals no callback set 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 1 -- 172.21.15.47:0/623928157 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6278003c80 con 0x7f62841385b0 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f6281ffb700 10 monclient: dump: 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.237 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 10 monclient: shutdown 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 1 -- 172.21.15.47:0/623928157 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 msgr2=0x7f62840a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.207+0000 7f628998e700 1 --2- 172.21.15.47:0/623928157 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 0x7f62840a63a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62780098f0 tx=0x7f6278009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7f628998e700 1 -- 172.21.15.47:0/623928157 shutdown_connections 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7f628998e700 1 --2- 172.21.15.47:0/623928157 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f62841385b0 0x7f62840a63a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7f628998e700 1 --2- 172.21.15.47:0/623928157 >> v2:172.21.15.179:3300/0 conn(0x7f6284137d90 0x7f62840a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7f628998e700 1 -- 172.21.15.47:0/623928157 >> 172.21.15.47:0/623928157 conn(0x7f62840aa8b0 msgr2=0x7f62840a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7f628998e700 1 -- 172.21.15.47:0/623928157 shutdown_connections 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7fcb1cc32700 10 monclient: get_monmap_and_config 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7fcb1cc32700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7fcb1cc32700 10 monclient: monmap: 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:48.209512+0000 2024-04-20T02:15:48.239 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:48.209512+0000 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:48.240 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.208+0000 7f628998e700 1 -- 172.21.15.47:0/623928157 wait complete. 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7f628998e700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7f628998e700 10 monclient: monmap: 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.241 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7f628998e700 1 librados: starting msgr at 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7f628998e700 1 librados: starting objecter 2024-04-20T02:15:48.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7f628998e700 1 Processor -- start 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7eff33df8700 10 monclient: get_monmap_and_config 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7eff33df8700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.209+0000 7eff33df8700 10 monclient: monmap: 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:48.211032+0000 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:48.211032+0000 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.243 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:48.244 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7fcb1cc32700 1 Processor -- start 2024-04-20T02:15:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7fcb1cc32700 1 -- start start 2024-04-20T02:15:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7fcb1cc32700 10 monclient: init 2024-04-20T02:15:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient: _add_conns ranks=[10,1,2,4,8,7,0,3,5,9,6] 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 -- start start 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 librados: setting wanted keys 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 librados: calling monclient init 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient: init 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient: _add_conns ranks=[7,8,6,4,0,2,1,5,3] 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 0x7f62841c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): picked mon.noname-i con 0x7f6284137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6284138170 0x7f6284135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): picked mon.noname-e con 0x7f6284138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f62841385b0 0x7f62841363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): picked mon.noname-d con 0x7f62841385b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6284122280 con 0x7f62841385b0 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f628409e9e0 con 0x7f6284137d90 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f628409dc20 con 0x7f6284138170 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.210+0000 7f628998e700 10 monclient(hunting): authenticate will time out at 1627.611971s 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb18066610 0x7fcb1813c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): picked mon.noname-f con 0x7fcb18066610 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:48.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcb1809cd50 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb1809d1b0 0x7fcb180a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): picked mon.noname-b con 0x7fcb1809d1b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb180751e0 con 0x7fcb1809cd50 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1809a5a0 con 0x7fcb18066610 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180669f0 con 0x7fcb1809d1b0 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1cc32700 10 monclient(hunting): authenticate will time out at 1627.612273s 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb18066610 0x7fcb1813c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb18066610 0x7fcb1813c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:48196/0 (socket says 172.21.15.47:48196) 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 -- 172.21.15.47:0/4041477404 learned_addr learned my addr 172.21.15.47:0/4041477404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): get_auth_request con 0x7fcb18066610 auth_method 0 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb15d9b700 1 --1- 172.21.15.47:0/4041477404 >> v1:172.21.15.179:6789/0 conn(0x7fcb1809cd50 0x7fcb180a1f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:52600/0 (socket says 172.21.15.47:52600) 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 0x7f62841c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 0x7f62841c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:55002/0 (socket says 172.21.15.47:55002) 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 -- 172.21.15.47:0/1282879714 learned_addr learned my addr 172.21.15.47:0/1282879714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): get_auth_request con 0x7f6284137d90 auth_method 0 2024-04-20T02:15:48.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 1 --2- 172.21.15.47:0/4041477404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb1809d1b0 0x7fcb180a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): get_auth_request con 0x7fcb1809d1b0 auth_method 0 2024-04-20T02:15:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f62837fe700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f62841385b0 0x7f62841363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f62837fe700 10 monclient(hunting): get_auth_request con 0x7f62841385b0 auth_method 0 2024-04-20T02:15:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f62837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f62837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f62837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient(hunting): handle_auth_done global_id 4402 payload 274 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 -- 172.21.15.47:0/4041477404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb1809d1b0 msgr2=0x7fcb180a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 --2- 172.21.15.47:0/4041477404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb1809d1b0 0x7fcb180a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient(hunting): handle_auth_done global_id 4282 payload 824 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 -- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6284138170 msgr2=0x7f6284135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6284138170 0x7f6284135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 -- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f62841385b0 msgr2=0x7f62841363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f62841385b0 0x7f62841363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 monclient: found mon.noname-i 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7f6282ffd700 1 -- 172.21.15.47:0/1282879714 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278009550 con 0x7f6284137d90 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 -- 172.21.15.47:0/4041477404 >> v1:172.21.15.179:6789/0 conn(0x7fcb1809cd50 legacy=0x7fcb180a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 monclient: found mon.noname-f 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.211+0000 7fcb1659c700 1 -- 172.21.15.47:0/4041477404 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb18099090 con 0x7fcb18066610 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f6282ffd700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 0x7f62841c7c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f627400b550 tx=0x7f627400b910 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 client.?.objecter ms_handle_connect 0x7f6284137d90 2024-04-20T02:15:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:48.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1659c700 1 --2- 172.21.15.47:0/4041477404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb18066610 0x7fcb1813c260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0098f0 tx=0x7fcb0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d76916870b04aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6274011aa0 con 0x7f6284137d90 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: dump: 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.257 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6274011c20 con 0x7f6284137d90 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6274011d90 con 0x7f6284137d90 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f626bfff700 10 monclient: dump: 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.259 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.260 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.260 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.260 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.260 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 5 monclient: authenticate success, global_id 4282 2024-04-20T02:15:48.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 10 monclient: _renew_subs 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 1 -- 172.21.15.47:0/4041477404 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb0c011aa0 con 0x7fcb18066610 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: dump: 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 1 -- 172.21.15.47:0/4041477404 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c011c00 con 0x7fcb18066610 2024-04-20T02:15:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 1 -- 172.21.15.47:0/4041477404 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb0c011d70 con 0x7fcb18066610 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f583c0239c0 con 0x7f5850138170 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f584d7fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f584d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7eff33df8700 1 Processor -- start 2024-04-20T02:15:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1559a700 10 monclient: dump: 2024-04-20T02:15:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.264 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.265 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1cc32700 5 monclient: authenticate success, global_id 4402 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1cc32700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7fcb1cc32700 4 set_mon_vals no callback set 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7fcb1cc32700 10 monclient: shutdown 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7fcb1cc32700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7fcb1cc32700 1 -- 172.21.15.47:0/4041477404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb18066610 msgr2=0x7fcb1813c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/4041477404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb18066610 0x7fcb1813c260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0098f0 tx=0x7fcb0c009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7fcb1cc32700 1 -- 172.21.15.47:0/4041477404 shutdown_connections 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/4041477404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb1809d1b0 0x7fcb180a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/4041477404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb18066610 0x7fcb1813c260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7fcb1cc32700 1 -- 172.21.15.47:0/4041477404 >> 172.21.15.47:0/4041477404 conn(0x7fcb180a3000 msgr2=0x7fcb180a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6284121fa0 con 0x7f6284137d90 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 10 client.4282.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 10 monclient: _renew_subs 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6284121260 con 0x7f6284137d90 2024-04-20T02:15:48.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f628998e700 1 librados: init done 2024-04-20T02:15:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7f628998e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.212+0000 7f6280ff9700 4 set_mon_vals no callback set 2024-04-20T02:15:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7eff33df8700 1 -- start start 2024-04-20T02:15:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7eff33df8700 10 monclient: init 2024-04-20T02:15:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.213+0000 7eff33df8700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient: _add_conns ranks=[7,8,5,10,3,1,0,2,4,6,9] 2024-04-20T02:15:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c06cf40 0x7eff2c0a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): picked mon.noname-i con 0x7eff2c06cf40 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7eff2c0a2db0 0x7eff2c0a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): picked mon.noname-e con 0x7eff2c0a2db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7eff2c134690 0x7eff2c134ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): picked mon.noname-h con 0x7eff2c134690 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0a36f0 con 0x7eff2c134690 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0a0240 con 0x7eff2c06cf40 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c06d320 con 0x7eff2c0a2db0 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff33df8700 10 monclient(hunting): authenticate will time out at 1627.615120s 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7eff2c0a2db0 0x7eff2c0a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7eff2c0a2db0 0x7eff2c0a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43688/0 (socket says 172.21.15.47:43688) 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 1 -- 172.21.15.47:0/3757160605 learned_addr learned my addr 172.21.15.47:0/3757160605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): get_auth_request con 0x7eff2c0a2db0 auth_method 0 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7eff2c134690 0x7eff2c134ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): get_auth_request con 0x7eff2c134690 auth_method 0 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff32395700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.214+0000 7eff31393700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 10 monclient(hunting): handle_auth_done global_id 4289 payload 274 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 -- 172.21.15.47:0/3757160605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7eff2c0a2db0 msgr2=0x7eff2c0a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7eff2c0a2db0 0x7eff2c0a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 -- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c06cf40 msgr2=0x7eff2c0a2870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c06cf40 0x7eff2c0a2870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 monclient: found mon.noname-h 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:48.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 -- 172.21.15.47:0/3757160605 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0751e0 con 0x7eff2c134690 2024-04-20T02:15:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f628998e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f628998e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628409e430 con 0x7f6284137d90 2024-04-20T02:15:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff32395700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7eff2c134690 0x7eff2c134ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7eff2400f3a0 tx=0x7eff2400f6b0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=440d683688350e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f6274011630 con 0x7f6284137d90 2024-04-20T02:15:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 1 -- 172.21.15.47:0/3757160605 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff24012070 con 0x7eff2c134690 2024-04-20T02:15:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: dump: 2024-04-20T02:15:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.274 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 1 -- 172.21.15.47:0/3757160605 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2400d540 con 0x7eff2c134690 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 1 -- 172.21.15.47:0/3757160605 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff2400fea0 con 0x7eff2c134690 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff30b92700 10 monclient: dump: 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.276 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.277 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f626bfff700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f627003a260 0x7f627003c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f6274049eb0 con 0x7f6284137d90 2024-04-20T02:15:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f626bfff700 10 client.4282.objecter ms_dispatch 0x7f6284001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f626bfff700 3 client.4282.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:48.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7f626bfff700 3 client.4282.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7f626bfff700 20 client.4282.objecter dump_active .. 0 homeless 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7f62827fc700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f627003a260 0x7f627003c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7f62827fc700 10 monclient: get_auth_request con 0x7f627003a260 auth_method 0 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff33df8700 5 monclient: authenticate success, global_id 4289 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff33df8700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.215+0000 7eff33df8700 4 set_mon_vals no callback set 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 10 monclient: shutdown 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 -- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7eff2c134690 msgr2=0x7eff2c134ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7eff2c134690 0x7eff2c134ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7eff2400f3a0 tx=0x7eff2400f6b0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7f62827fc700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f627003a260 0x7f627003c720 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62840721d0 tx=0x7f627c009250 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7f626bfff700 10 client.4282.objecter ms_handle_connect 0x7f627003a260 2024-04-20T02:15:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 -- 172.21.15.47:0/3757160605 shutdown_connections 2024-04-20T02:15:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7eff2c134690 0x7eff2c134ad0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c06cf40 0x7eff2c0a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 --2- 172.21.15.47:0/3757160605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7eff2c0a2db0 0x7eff2c0a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.216+0000 7eff33df8700 1 -- 172.21.15.47:0/3757160605 >> 172.21.15.47:0/3757160605 conn(0x7eff2c0a4010 msgr2=0x7eff2c06c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.217+0000 7eff33df8700 1 -- 172.21.15.47:0/3757160605 shutdown_connections 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.217+0000 7fcb1cc32700 1 -- 172.21.15.47:0/4041477404 shutdown_connections 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.217+0000 7fcb1cc32700 1 -- 172.21.15.47:0/4041477404 wait complete. 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.217+0000 7fcb1cc32700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient: monmap: 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:48.284 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 librados: starting msgr at 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 librados: starting objecter 2024-04-20T02:15:48.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 Processor -- start 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 -- start start 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 librados: setting wanted keys 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 librados: calling monclient init 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient: init 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient: _add_conns ranks=[4,1,5,3,7,8,0,2,6] 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb18066610 0x7fcb18138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): picked mon.noname-c con 0x7fcb18066610 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb1809cd50 0x7fcb18072590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): picked mon.noname-f con 0x7fcb1809cd50 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:48.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcb1809d1b0 0x7fcb1809d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): picked mon.noname-h con 0x7fcb1809d1b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1809ee40 con 0x7fcb1809cd50 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180992b0 con 0x7fcb18066610 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180a0770 con 0x7fcb1809d1b0 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.218+0000 7fcb1cc32700 10 monclient(hunting): authenticate will time out at 1627.619923s 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcb1809d1b0 0x7fcb1809d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb1809cd50 0x7fcb18072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb18066610 0x7fcb18138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb18066610 0x7fcb18138a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44470/0 (socket says 172.21.15.47:44470) 2024-04-20T02:15:48.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 1 -- 172.21.15.47:0/939123047 learned_addr learned my addr 172.21.15.47:0/939123047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): get_auth_request con 0x7fcb18066610 auth_method 0 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): get_auth_request con 0x7fcb1809cd50 auth_method 0 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): get_auth_request con 0x7fcb1809d1b0 auth_method 0 2024-04-20T02:15:48.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb15d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7fcb1659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7eff33df8700 1 -- 172.21.15.47:0/3757160605 wait complete. 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7eff33df8700 10 monclient: build_initial_monmap 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.219+0000 7eff33df8700 10 monclient: monmap: 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:48.291 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:48.292 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 10 monclient(hunting): handle_auth_done global_id 4298 payload 824 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 -- 172.21.15.47:0/939123047 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb18066610 msgr2=0x7fcb18138a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb18066610 0x7fcb18138a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 -- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb1809cd50 msgr2=0x7fcb18072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb1809cd50 0x7fcb18072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 monclient: found mon.noname-h 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:15:48.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 -- 172.21.15.47:0/939123047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c009550 con 0x7fcb1809d1b0 2024-04-20T02:15:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 1 librados: starting msgr at 2024-04-20T02:15:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 1 librados: starting objecter 2024-04-20T02:15:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb16d9d700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcb1809d1b0 0x7fcb1809d900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000efc0 tx=0x7fcb1000c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 client.?.objecter ms_handle_connect 0x7fcb1809d1b0 2024-04-20T02:15:48.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 1 Processor -- start 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 1 -- start start 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 1 librados: setting wanted keys 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 1 librados: calling monclient init 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7eff33df8700 10 monclient: init 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb1000cea0 con 0x7fcb1809d1b0 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: dump: 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.296 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient: _add_conns ranks=[6,7,1,4,8,5,0,2,3] 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7eff2c06cf40 0x7eff2c1d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): picked mon.noname-d con 0x7eff2c06cf40 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c0a2db0 0x7eff2c1ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): picked mon.noname-i con 0x7eff2c0a2db0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c134690 0x7eff2c1cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): picked mon.noname-f con 0x7eff2c134690 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ba7f0 con 0x7eff2c134690 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c09f370 con 0x7eff2c06cf40 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1d66e0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:48.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7eff33df8700 10 monclient(hunting): authenticate will time out at 1627.622339s 2024-04-20T02:15:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f6274021d40 con 0x7f6284137d90 2024-04-20T02:15:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7f626bfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7f626bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb1cc32700 5 monclient: authenticate success, global_id 4298 2024-04-20T02:15:48.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1000f040 con 0x7fcb1809d1b0 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb10009e60 con 0x7fcb1809d1b0 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb14d99700 4 set_mon_vals no callback set 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.220+0000 7fcb06ffd700 10 monclient: dump: 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.300 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7fcb1cc32700 10 monclient: _renew_subs 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7fcb1cc32700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:48.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.221+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb181396b0 con 0x7fcb1809d1b0 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c134690 0x7eff2c1cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c134690 0x7eff2c1cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:48206/0 (socket says 172.21.15.47:48206) 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 1 -- 172.21.15.47:0/3129076260 learned_addr learned my addr 172.21.15.47:0/3129076260 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 10 monclient(hunting): get_auth_request con 0x7eff2c134690 auth_method 0 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff32395700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff31393700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c0a2db0 0x7eff2c1ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff31393700 10 monclient(hunting): get_auth_request con 0x7eff2c0a2db0 auth_method 0 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff31393700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:48.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff31393700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7eff31393700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7fcb1cc32700 10 client.4298.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7fcb1cc32700 10 monclient: _renew_subs 2024-04-20T02:15:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7fcb1cc32700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:48.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb181d0fe0 con 0x7fcb1809d1b0 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.222+0000 7fcb1cc32700 1 librados: init done 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.223+0000 7fcb1cc32700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.223+0000 7fcb1cc32700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.223+0000 7fcb1cc32700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.223+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1809cb30 con 0x7fcb1809d1b0 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.224+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7fcb1000ca30 con 0x7fcb1809d1b0 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.224+0000 7fcb06ffd700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcaf803a260 0x7fcaf803c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 10 monclient(hunting): handle_auth_done global_id 4291 payload 824 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7fcb1659c700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcaf803a260 0x7fcaf803c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 -- 172.21.15.47:0/3129076260 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7eff2c06cf40 msgr2=0x7eff2c1d5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7eff2c06cf40 0x7eff2c1d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 -- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c134690 msgr2=0x7eff2c1cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c134690 0x7eff2c1cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 10 monclient: _finish_hunting 0 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 monclient: found mon.noname-i 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 -- 172.21.15.47:0/3129076260 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2400f050 con 0x7eff2c0a2db0 2024-04-20T02:15:48.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7fcb1659c700 10 monclient: get_auth_request con 0x7fcaf803a260 auth_method 0 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff31393700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c0a2db0 0x7eff2c1ccdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7eff2800fbb0 tx=0x7eff2800bf70 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 client.?.objecter ms_handle_connect 0x7eff2c0a2db0 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff28010440 con 0x7eff2c0a2db0 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: dump: 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.307 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff280105c0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff28010ba0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff227fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.225+0000 7eff23fff700 4 set_mon_vals no callback set 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff227fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff227fc700 10 monclient: dump: 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:48.309 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:48.310 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fcb1000cce0 con 0x7fcb1809d1b0 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb06ffd700 10 client.4298.objecter ms_dispatch 0x7fcb18001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb06ffd700 3 client.4298.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb06ffd700 3 client.4298.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb06ffd700 20 client.4298.objecter dump_active .. 0 homeless 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb1659c700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcaf803a260 0x7fcaf803c720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009c40 tx=0x7fcb0c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7fcb06ffd700 10 client.4298.objecter ms_handle_connect 0x7fcaf803a260 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 5 monclient: authenticate success, global_id 4291 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 10 monclient: _renew_subs 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c0725a0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 10 client.4291.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 10 monclient: _renew_subs 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1b9ee0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.226+0000 7eff33df8700 1 librados: init done 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7eff28010d00 con 0x7eff2c0a2db0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff33df8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff33df8700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff33df8700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c1329b0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7eff1003a290 0x7eff1003c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7eff2804a140 con 0x7eff2c0a2db0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 10 client.4291.objecter ms_dispatch 0x7eff2c001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 3 client.4291.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 3 client.4291.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.228+0000 7eff227fc700 20 client.4291.objecter dump_active .. 0 homeless 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.229+0000 7eff31b94700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7eff1003a290 0x7eff1003c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.229+0000 7eff31b94700 10 monclient: get_auth_request con 0x7eff1003a290 auth_method 0 2024-04-20T02:15:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.230+0000 7eff31b94700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7eff1003a290 0x7eff1003c750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009770 tx=0x7eff1c006cd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.230+0000 7eff227fc700 10 client.4291.objecter ms_handle_connect 0x7eff1003a290 2024-04-20T02:15:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.232+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fcb1004f060 con 0x7fcb1809d1b0 2024-04-20T02:15:48.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.232+0000 7fcb06ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.232+0000 7fcb06ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.244+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7eff2800c8d0 con 0x7eff2c0a2db0 2024-04-20T02:15:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.244+0000 7eff227fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:48.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.244+0000 7eff227fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.424+0000 7f8f03b52700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-04-20T02:15:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.424+0000 7f8f03b52700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-04-20T02:15:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.424+0000 7f8f03b52700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.424+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8efc0a4570 con 0x7f8efc0a3960 2024-04-20T02:15:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.425+0000 7f8ef27fc700 1 -- 172.21.15.47:0/4019782992 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ef8017020 con 0x7f8efc0a3960 2024-04-20T02:15:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.425+0000 7f8ef27fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-04-20T02:15:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.425+0000 7f8ef27fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:48.425 INFO:teuthology.orchestra.run.smithi047.stdout:51539607555 2024-04-20T02:15:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.427+0000 7f8f03b52700 10 librados: watch_flush enter 2024-04-20T02:15:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 10 librados: watch_flush exit 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 20 client.4297.objecter shutdown clearing up homeless session... 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 10 client.4297.objecter successfully canceled tick 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8ee003a260 msgr2=0x7f8ee003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8ee003a260 0x7f8ee003c720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009840 tx=0x7f8eec008030 comp rx=0 tx=0).stop 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 10 monclient: shutdown 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8efc0a3960 msgr2=0x7f8efc12c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.428+0000 7f8f03b52700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8efc0a3960 0x7f8efc12c130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800e910 tx=0x7f8ef800ec20 comp rx=0 tx=0).stop 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 shutdown_connections 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8ee003a260 0x7f8ee003c720 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8efc07c930 0x7f8efc12ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8efc0a3960 0x7f8efc12c130 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 --2- 172.21.15.47:0/4019782992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8efc137ab0 0x7f8efc08b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 >> 172.21.15.47:0/4019782992 conn(0x7f8efc0a4a50 msgr2=0x7f8efc0a1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 shutdown_connections 2024-04-20T02:15:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 -- 172.21.15.47:0/4019782992 wait complete. 2024-04-20T02:15:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.429+0000 7f8f03b52700 1 librados: shutdown 2024-04-20T02:15:48.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.4 2024-04-20T02:15:48.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f5856c41700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f5856c41700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f5856c41700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5850134030 con 0x7f5850138170 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f584d7fa700 1 -- 172.21.15.47:0/2467968587 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f583c015070 con 0x7f5850138170 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f584d7fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.454+0000 7f584d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:48.454 INFO:teuthology.orchestra.run.smithi047.stdout:51539607554 2024-04-20T02:15:48.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.456+0000 7f5856c41700 10 librados: watch_flush enter 2024-04-20T02:15:48.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 10 librados: watch_flush exit 2024-04-20T02:15:48.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 20 client.4265.objecter shutdown clearing up homeless session... 2024-04-20T02:15:48.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 10 client.4265.objecter successfully canceled tick 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a290 msgr2=0x7f583403c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a290 0x7f583403c750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58440098f0 tx=0x7f5844008040 comp rx=0 tx=0).stop 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 10 monclient: shutdown 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5850138170 msgr2=0x7f5850137110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5850138170 0x7f5850137110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f583c016000 tx=0x7f583c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 shutdown_connections 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a290 0x7f583403c750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58501385b0 0x7f58501318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5850137d90 0x7f5850136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 --2- 172.21.15.47:0/2467968587 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5850138170 0x7f5850137110 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 >> 172.21.15.47:0/2467968587 conn(0x7f58500aa8b0 msgr2=0x7f58500a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 shutdown_connections 2024-04-20T02:15:48.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 -- 172.21.15.47:0/2467968587 wait complete. 2024-04-20T02:15:48.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.457+0000 7f5856c41700 1 librados: shutdown 2024-04-20T02:15:48.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607555 got 51539607554 for osd.3 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2658e0e700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2658e0e700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2658e0e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f26541287f0 con 0x7f26540721d0 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2642ffd700 1 -- 172.21.15.47:0/2738897678 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2644012ad0 con 0x7f26540721d0 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2642ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.605+0000 7f2642ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:48.605 INFO:teuthology.orchestra.run.smithi047.stdout:47244640258 2024-04-20T02:15:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 10 librados: watch_flush enter 2024-04-20T02:15:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 10 librados: watch_flush exit 2024-04-20T02:15:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 20 client.4384.objecter shutdown clearing up homeless session... 2024-04-20T02:15:48.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 10 client.4384.objecter successfully canceled tick 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f263403a260 msgr2=0x7f263403c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f263403a260 0x7f263403c720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2648006fd0 tx=0x7f2648008040 comp rx=0 tx=0).stop 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 10 monclient: shutdown 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 msgr2=0x7f26541d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 0x7f26541d0db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26540726f0 tx=0x7f2644004750 comp rx=0 tx=0).stop 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 shutdown_connections 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f265409fd30 0x7f2654124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f263403a260 0x7f263403c720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26540721d0 0x7f26541d0db0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 --2- 172.21.15.47:0/2738897678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f265409f930 0x7f26541d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 >> 172.21.15.47:0/2738897678 conn(0x7f26540a6210 msgr2=0x7f26541b8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.609+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 shutdown_connections 2024-04-20T02:15:48.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.610+0000 7f2658e0e700 1 -- 172.21.15.47:0/2738897678 wait complete. 2024-04-20T02:15:48.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.610+0000 7f2658e0e700 1 librados: shutdown 2024-04-20T02:15:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.615+0000 7f628998e700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-04-20T02:15:48.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.615+0000 7f628998e700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-04-20T02:15:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.615+0000 7f628998e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.615+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f62841224a0 con 0x7f6284137d90 2024-04-20T02:15:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.616+0000 7f626bfff700 1 -- 172.21.15.47:0/1282879714 <== mon.7 v2:172.21.15.179:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6274015070 con 0x7f6284137d90 2024-04-20T02:15:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.616+0000 7f626bfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-04-20T02:15:48.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.616+0000 7f626bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:48.615 INFO:teuthology.orchestra.run.smithi047.stdout:51539607555 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 10 librados: watch_flush enter 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 10 librados: watch_flush exit 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 20 client.4282.objecter shutdown clearing up homeless session... 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 10 client.4282.objecter successfully canceled tick 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f627003a260 msgr2=0x7f627003c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f627003a260 0x7f627003c720 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62840721d0 tx=0x7f627c009250 comp rx=0 tx=0).stop 2024-04-20T02:15:48.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 10 monclient: shutdown 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 msgr2=0x7f62841c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 0x7f62841c7c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f627400b550 tx=0x7f627400b910 comp rx=0 tx=0).stop 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 shutdown_connections 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f627003a260 0x7f627003c720 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f62841385b0 0x7f62841363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6284137d90 0x7f62841c7c40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 --2- 172.21.15.47:0/1282879714 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6284138170 0x7f6284135e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.618+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 >> 172.21.15.47:0/1282879714 conn(0x7f62840aa8b0 msgr2=0x7f628409ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.619+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 shutdown_connections 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.619+0000 7f628998e700 1 -- 172.21.15.47:0/1282879714 wait complete. 2024-04-20T02:15:48.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.619+0000 7f628998e700 1 librados: shutdown 2024-04-20T02:15:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb1cc32700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-04-20T02:15:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb1cc32700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-04-20T02:15:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb1cc32700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:15:48.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb1806c7a0 con 0x7fcb1809d1b0 2024-04-20T02:15:48.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb06ffd700 1 -- 172.21.15.47:0/939123047 <== mon.5 v2:172.21.15.179:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb1004f070 con 0x7fcb1809d1b0 2024-04-20T02:15:48.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb06ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-04-20T02:15:48.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.622+0000 7fcb06ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:48.622 INFO:teuthology.orchestra.run.smithi047.stdout:51539607555 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 10 librados: watch_flush enter 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 10 librados: watch_flush exit 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 20 client.4298.objecter shutdown clearing up homeless session... 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 10 client.4298.objecter successfully canceled tick 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcaf803a260 msgr2=0x7fcaf803c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcaf803a260 0x7fcaf803c720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009c40 tx=0x7fcb0c008030 comp rx=0 tx=0).stop 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 10 monclient: shutdown 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcb1809d1b0 msgr2=0x7fcb1809d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.625+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcb1809d1b0 0x7fcb1809d900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000efc0 tx=0x7fcb1000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.5 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 shutdown_connections 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcb1809d1b0 0x7fcb1809d900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb18066610 0x7fcb18138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcaf803a260 0x7fcaf803c720 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 --2- 172.21.15.47:0/939123047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb1809cd50 0x7fcb18072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 >> 172.21.15.47:0/939123047 conn(0x7fcb180a3000 msgr2=0x7fcb181c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 shutdown_connections 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 -- 172.21.15.47:0/939123047 wait complete. 2024-04-20T02:15:48.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.626+0000 7fcb1cc32700 1 librados: shutdown 2024-04-20T02:15:48.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.1 2024-04-20T02:15:48.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.0 2024-04-20T02:15:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff33df8700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff33df8700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff33df8700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7eff2c072a10 con 0x7eff2c0a2db0 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff227fc700 1 -- 172.21.15.47:0/3129076260 <== mon.7 v2:172.21.15.179:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff2c072a10 con 0x7eff2c0a2db0 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff227fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.654+0000 7eff227fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:48.655 INFO:teuthology.orchestra.run.smithi047.stdout:51539607555 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 10 librados: watch_flush enter 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 10 librados: watch_flush exit 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 20 client.4291.objecter shutdown clearing up homeless session... 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 10 client.4291.objecter successfully canceled tick 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7eff1003a290 msgr2=0x7eff1003c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7eff1003a290 0x7eff1003c750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009770 tx=0x7eff1c006cd0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 10 monclient: shutdown 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c0a2db0 msgr2=0x7eff2c1ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c0a2db0 0x7eff2c1ccdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7eff2800fbb0 tx=0x7eff2800bf70 comp rx=0 tx=0).stop 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 shutdown_connections 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7eff1003a290 0x7eff1003c750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c134690 0x7eff2c1cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7eff2c06cf40 0x7eff2c1d5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 --2- 172.21.15.47:0/3129076260 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7eff2c0a2db0 0x7eff2c1ccdc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 >> 172.21.15.47:0/3129076260 conn(0x7eff2c0a4010 msgr2=0x7eff2c133ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 shutdown_connections 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 -- 172.21.15.47:0/3129076260 wait complete. 2024-04-20T02:15:48.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:48.657+0000 7eff33df8700 1 librados: shutdown 2024-04-20T02:15:48.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.2 2024-04-20T02:15:49.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T02:15:49.472 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T02:15:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.594+0000 7fd5371f3700 10 monclient: get_monmap_and_config 2024-04-20T02:15:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.594+0000 7fd5371f3700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.594+0000 7fd5371f3700 10 monclient: monmap: 2024-04-20T02:15:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.593 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:49.595459+0000 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:49.595459+0000 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.594 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:49.595 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:49.595 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 Processor -- start 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 -- start start 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient: init 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient: _add_conns ranks=[8,5,7,9,1,10,3,0,4,6,2] 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 0x7fd5300a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): picked mon.noname-e con 0x7fd530137d90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530138170 0x7fd5300a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): picked mon.noname-h con 0x7fd530138170 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd5301385b0 0x7fd5300a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): picked mon.noname-i con 0x7fd5301385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd53009fa40 con 0x7fd530138170 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd53009dfc0 con 0x7fd5301385b0 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300a7bc0 con 0x7fd530137d90 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd5371f3700 10 monclient(hunting): authenticate will time out at 1628.996824s 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.595+0000 7fd534f8f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 0x7fd5300a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 0x7fd5300a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43698/0 (socket says 172.21.15.47:43698) 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 -- 172.21.15.47:0/3331086982 learned_addr learned my addr 172.21.15.47:0/3331086982 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): get_auth_request con 0x7fd530137d90 auth_method 0 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd535790700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd5301385b0 0x7fd5300a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52ffff700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530138170 0x7fd5300a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52ffff700 10 monclient(hunting): get_auth_request con 0x7fd530138170 auth_method 0 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd535790700 10 monclient(hunting): get_auth_request con 0x7fd5301385b0 auth_method 0 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd535790700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd535790700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd535790700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient(hunting): handle_auth_done global_id 4283 payload 274 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 -- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd5301385b0 msgr2=0x7fd5300a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd5301385b0 0x7fd5300a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 -- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530138170 msgr2=0x7fd5300a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530138170 0x7fd5300a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 monclient: found mon.noname-e 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 -- 172.21.15.47:0/3331086982 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300751e0 con 0x7fd530137d90 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52ffff700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530138170 0x7fd5300a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:49.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd534f8f700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 0x7fd5300a09e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd520009950 tx=0x7fd520009c60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=142f06c96611691a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 1 -- 172.21.15.47:0/3331086982 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd520009ea0 con 0x7fd530137d90 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: dump: 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.613 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd5371f3700 5 monclient: authenticate success, global_id 4283 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd5371f3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 1 -- 172.21.15.47:0/3331086982 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52000bc90 con 0x7fd530137d90 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd5371f3700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd5371f3700 4 set_mon_vals no callback set 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 1 -- 172.21.15.47:0/3331086982 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd52000be00 con 0x7fd530137d90 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.596+0000 7fd52f7fe700 10 monclient: dump: 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.615 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.616 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 10 monclient: shutdown 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 -- 172.21.15.47:0/3331086982 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 msgr2=0x7fd5300a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 0x7fd5300a09e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd520009950 tx=0x7fd520009c60 comp rx=0 tx=0).stop 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 -- 172.21.15.47:0/3331086982 shutdown_connections 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd5301385b0 0x7fd5300a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd530137d90 0x7fd5300a09e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 --2- 172.21.15.47:0/3331086982 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530138170 0x7fd5300a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 -- 172.21.15.47:0/3331086982 >> 172.21.15.47:0/3331086982 conn(0x7fd5300aa8b0 msgr2=0x7fd5300a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 -- 172.21.15.47:0/3331086982 shutdown_connections 2024-04-20T02:15:49.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.597+0000 7fd5371f3700 1 -- 172.21.15.47:0/3331086982 wait complete. 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 10 monclient: monmap: 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.618 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 1 librados: starting msgr at 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 1 librados: starting objecter 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 1 Processor -- start 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 1 -- start start 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 1 librados: setting wanted keys 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 1 librados: calling monclient init 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.598+0000 7fd5371f3700 10 monclient: init 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient: _add_conns ranks=[5,4,8,6,1,0,7,3,2] 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530137d90 0x7fd5301c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): picked mon.noname-h con 0x7fd530137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 0x7fd530136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): picked mon.noname-c con 0x7fd530138170 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd5301385b0 0x7fd530137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): picked mon.noname-e con 0x7fd5301385b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd530124a30 con 0x7fd530138170 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301c9700 con 0x7fd530137d90 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301233c0 con 0x7fd5301385b0 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd5371f3700 10 monclient(hunting): authenticate will time out at 1629.000347s 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 0x7fd530136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 0x7fd530136fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44474/0 (socket says 172.21.15.47:44474) 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 1 -- 172.21.15.47:0/438313651 learned_addr learned my addr 172.21.15.47:0/438313651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): get_auth_request con 0x7fd530138170 auth_method 0 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd5301385b0 0x7fd530137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530137d90 0x7fd5301c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): get_auth_request con 0x7fd5301385b0 auth_method 0 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): get_auth_request con 0x7fd530137d90 auth_method 0 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd534f8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient(hunting): handle_auth_done global_id 4292 payload 824 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 -- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530137d90 msgr2=0x7fd5301c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530137d90 0x7fd5301c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 -- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 msgr2=0x7fd530136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 0x7fd530136fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 monclient: found mon.noname-e 2024-04-20T02:15:49.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd535790700 1 -- 172.21.15.47:0/438313651 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520009550 con 0x7fd5301385b0 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.599+0000 7fd52ffff700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 0x7fd530136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd535790700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd5301385b0 0x7fd530137500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd528007cf0 tx=0x7fd52800e0e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 client.?.objecter ms_handle_connect 0x7fd5301385b0 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd528018070 con 0x7fd5301385b0 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: dump: 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.623 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.624 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52800ede0 con 0x7fd5301385b0 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 5 monclient: authenticate success, global_id 4292 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 10 monclient: _renew_subs 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53009e9e0 con 0x7fd5301385b0 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd528014890 con 0x7fd5301385b0 2024-04-20T02:15:49.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52effd700 4 set_mon_vals no callback set 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd52d7fa700 10 monclient: dump: 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.626 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 10 client.4292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 10 monclient: _renew_subs 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd530124100 con 0x7fd5301385b0 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.600+0000 7fd5371f3700 1 librados: init done 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd528014420 con 0x7fd5301385b0 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd5371f3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd5371f3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd5371f3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53009f710 con 0x7fd5301385b0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd51c03a320 0x7fd51c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fd528049c70 con 0x7fd5301385b0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 10 client.4292.objecter ms_dispatch 0x7fd530001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 3 client.4292.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 3 client.4292.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd52d7fa700 20 client.4292.objecter dump_active .. 0 homeless 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd534f8f700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd51c03a320 0x7fd51c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.601+0000 7fd534f8f700 10 monclient: get_auth_request con 0x7fd51c03a320 auth_method 0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.602+0000 7fd52d7fa700 10 client.4292.objecter ms_handle_connect 0x7fd51c03a320 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.602+0000 7fd534f8f700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd51c03a320 0x7fd51c03c7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd520000c00 tx=0x7fd520008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.606+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd5280146d0 con 0x7fd5301385b0 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.606+0000 7fd52d7fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.606+0000 7fd52d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:49.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T02:15:49.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.632+0000 7fc50a619700 10 monclient: get_monmap_and_config 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.632+0000 7fc50a619700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.632+0000 7fc50a619700 10 monclient: monmap: 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:49.634035+0000 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:49.634035+0000 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.632 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:49.633 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:49.633 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.639 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.633+0000 7fc50a619700 1 Processor -- start 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.633+0000 7fc50a619700 1 -- start start 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.633+0000 7fc50a619700 10 monclient: init 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient: _add_conns ranks=[7,6,4,10,1,0,2,8,5,9,3] 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc5040a2950 0x7fc5040a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): picked mon.noname-i con 0x7fc5040a2950 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5040a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): picked mon.noname-d con 0x7fc5040a0d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc50409b6e0 0x7fc504135710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): picked mon.noname-c con 0x7fc50409b6e0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc50409aa40 con 0x7fc50409b6e0 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc50409acb0 con 0x7fc5040a0d90 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc5040a37b0 con 0x7fc5040a2950 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc50a619700 10 monclient(hunting): authenticate will time out at 1629.035466s 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5040a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc50409b6e0 0x7fc504135710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5040a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52574/0 (socket says 172.21.15.47:52574) 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 1 -- 172.21.15.47:0/3624189366 learned_addr learned my addr 172.21.15.47:0/3624189366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): get_auth_request con 0x7fc5040a0d90 auth_method 0 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): get_auth_request con 0x7fc50409b6e0 auth_method 0 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc503fff700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc5040a2950 0x7fc5040a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc503fff700 10 monclient(hunting): get_auth_request con 0x7fc5040a2950 auth_method 0 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc503fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc503fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc503fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc508bb6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.634+0000 7fc5037fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 10 monclient(hunting): handle_auth_done global_id 4333 payload 274 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 -- 172.21.15.47:0/3624189366 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc5040a2950 msgr2=0x7fc5040a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc5040a2950 0x7fc5040a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 -- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 msgr2=0x7fc5040a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5040a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 monclient: found mon.noname-c 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 -- 172.21.15.47:0/3624189366 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5040751e0 con 0x7fc50409b6e0 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc5037fe700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5040a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc508bb6700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc50409b6e0 0x7fc504135710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009df0 tx=0x7fc4fc00b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4a4716d0593a9c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 1 -- 172.21.15.47:0/3624189366 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4fc00e070 con 0x7fc50409b6e0 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: dump: 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.648 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc50a619700 5 monclient: authenticate success, global_id 4333 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc50a619700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 1 -- 172.21.15.47:0/3624189366 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc0098f0 con 0x7fc50409b6e0 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc50a619700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc50a619700 4 set_mon_vals no callback set 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 1 -- 172.21.15.47:0/3624189366 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4fc0128a0 con 0x7fc50409b6e0 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.635+0000 7fc502ffd700 10 monclient: dump: 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.649 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 10 monclient: shutdown 2024-04-20T02:15:49.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 -- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc50409b6e0 msgr2=0x7fc504135710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc50409b6e0 0x7fc504135710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009df0 tx=0x7fc4fc00b330 comp rx=0 tx=0).stop 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 -- 172.21.15.47:0/3624189366 shutdown_connections 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc50409b6e0 0x7fc504135710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc5040a2950 0x7fc5040a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 --2- 172.21.15.47:0/3624189366 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5040a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 -- 172.21.15.47:0/3624189366 >> 172.21.15.47:0/3624189366 conn(0x7fc5040a7570 msgr2=0x7fc5040a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 -- 172.21.15.47:0/3624189366 shutdown_connections 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.636+0000 7fc50a619700 1 -- 172.21.15.47:0/3624189366 wait complete. 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 10 monclient: monmap: 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:49.651 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 1 librados: starting msgr at 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 1 librados: starting objecter 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 1 Processor -- start 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 1 -- start start 2024-04-20T02:15:49.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 1 librados: setting wanted keys 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 1 librados: calling monclient init 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.637+0000 7fc50a619700 10 monclient: init 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient: _add_conns ranks=[1,6,3,4,7,8,5,2,0] 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50409b6e0 0x7fc5041c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): picked mon.noname-f con 0x7fc50409b6e0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5041ba970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): picked mon.noname-d con 0x7fc5040a0d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc5040a2950 0x7fc5041cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): picked mon.noname-g con 0x7fc5040a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50409f920 con 0x7fc50409b6e0 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5041bb2e0 con 0x7fc5040a2950 2024-04-20T02:15:49.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc504066ea0 con 0x7fc5040a0d90 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc50a619700 10 monclient(hunting): authenticate will time out at 1629.039386s 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5041ba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5041ba970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52578/0 (socket says 172.21.15.47:52578) 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 -- 172.21.15.47:0/3323017544 learned_addr learned my addr 172.21.15.47:0/3323017544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): get_auth_request con 0x7fc5040a0d90 auth_method 0 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50409b6e0 0x7fc5041c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): get_auth_request con 0x7fc50409b6e0 auth_method 0 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc508bb6700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc5040a2950 0x7fc5041cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc508bb6700 10 monclient(hunting): get_auth_request con 0x7fc5040a2950 auth_method 0 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc508bb6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc508bb6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc508bb6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc503fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient(hunting): handle_auth_done global_id 4263 payload 824 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 -- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc5040a2950 msgr2=0x7fc5041cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc5040a2950 0x7fc5041cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 -- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50409b6e0 msgr2=0x7fc5041c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50409b6e0 0x7fc5041c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 monclient: found mon.noname-d 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc5037fe700 1 -- 172.21.15.47:0/3323017544 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8009550 con 0x7fc5040a0d90 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.638+0000 7fc508bb6700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc5040a2950 0x7fc5041cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc5037fe700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5041ba970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800ebe0 tx=0x7fc4f800eef0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 client.?.objecter ms_handle_connect 0x7fc5040a0d90 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f800ccc0 con 0x7fc5040a0d90 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: dump: 2024-04-20T02:15:49.656 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800ce20 con 0x7fc5040a0d90 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f80138e0 con 0x7fc5040a0d90 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc5027fc700 4 set_mon_vals no callback set 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc500ff9700 10 monclient: dump: 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.658 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 5 monclient: authenticate success, global_id 4263 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 10 monclient: _renew_subs 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc009a70 con 0x7fc5040a0d90 2024-04-20T02:15:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 10 client.4263.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 10 monclient: _renew_subs 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5041a9ab0 con 0x7fc5040a0d90 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.639+0000 7fc50a619700 1 librados: init done 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc4f8013470 con 0x7fc5040a0d90 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4f003a2f0 0x7fc4f003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fc4f804a250 con 0x7fc5040a0d90 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 10 client.4263.objecter ms_dispatch 0x7fc504001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 3 client.4263.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 3 client.4263.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.640+0000 7fc500ff9700 20 client.4263.objecter dump_active .. 0 homeless 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc503fff700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4f003a2f0 0x7fc4f003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc503fff700 10 monclient: get_auth_request con 0x7fc4f003a2f0 auth_method 0 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc50a619700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc50a619700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc50a619700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50406b890 con 0x7fc5040a0d90 2024-04-20T02:15:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc503fff700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4f003a2f0 0x7fc4f003c7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4006b20 tx=0x7fc4f4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.641+0000 7fc500ff9700 10 client.4263.objecter ms_handle_connect 0x7fc4f003a2f0 2024-04-20T02:15:49.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T02:15:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.651+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc4f8017080 con 0x7fc5040a0d90 2024-04-20T02:15:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.651+0000 7fc500ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.651+0000 7fc500ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:49.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T02:15:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.808+0000 7fa7af251700 10 monclient: get_monmap_and_config 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.808+0000 7fa7af251700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.808+0000 7fa7af251700 10 monclient: monmap: 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:49.809701+0000 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:49.809701+0000 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.809+0000 7fa7af251700 1 Processor -- start 2024-04-20T02:15:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 -- start start 2024-04-20T02:15:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient: init 2024-04-20T02:15:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient: _add_conns ranks=[5,2,9,4,10,3,1,8,0,7,6] 2024-04-20T02:15:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa7a8137d90 0x7fa7a80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): picked mon.noname-h con 0x7fa7a8137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 0x7fa7a80a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): picked mon.noname-b con 0x7fa7a8138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7a81385b0 0x7fa7a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): picked mon.noname-a con 0x7fa7a81385b0 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a809fa40 con 0x7fa7a81385b0 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a809dfc0 con 0x7fa7a8138170 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a80a7bc0 con 0x7fa7a8137d90 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.810+0000 7fa7af251700 10 monclient(hunting): authenticate will time out at 1629.211996s 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 0x7fa7a80a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7a81385b0 0x7fa7a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 0x7fa7a80a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53082/0 (socket says 172.21.15.47:53082) 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7a81385b0 0x7fa7a80a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56932/0 (socket says 172.21.15.47:56932) 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 -- 172.21.15.47:0/1377481507 learned_addr learned my addr 172.21.15.47:0/1377481507 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): get_auth_request con 0x7fa7a8138170 auth_method 0 2024-04-20T02:15:49.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): get_auth_request con 0x7fa7a81385b0 auth_method 0 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7ad7ee700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_done global_id 4295 payload 274 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 -- 172.21.15.47:0/1377481507 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa7a8137d90 msgr2=0x7fa7a80a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 --2- 172.21.15.47:0/1377481507 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa7a8137d90 0x7fa7a80a09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 -- 172.21.15.47:0/1377481507 >> v2:172.21.15.47:3300/0 conn(0x7fa7a81385b0 msgr2=0x7fa7a80a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 --2- 172.21.15.47:0/1377481507 >> v2:172.21.15.47:3300/0 conn(0x7fa7a81385b0 0x7fa7a80a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 monclient: found mon.noname-b 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.811+0000 7fa7a7fff700 1 -- 172.21.15.47:0/1377481507 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80751e0 con 0x7fa7a8138170 2024-04-20T02:15:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.812+0000 7fa7a7fff700 1 --2- 172.21.15.47:0/1377481507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 0x7fa7a80a5c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098f0 tx=0x7fa79c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac1225410685928 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 1 -- 172.21.15.47:0/1377481507 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa79c011aa0 con 0x7fa7a8138170 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: dump: 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.815 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 1 -- 172.21.15.47:0/1377481507 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c011c00 con 0x7fa7a8138170 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 1 -- 172.21.15.47:0/1377481507 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa79c011d70 con 0x7fa7a8138170 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7a77fe700 10 monclient: dump: 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.817 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7af251700 5 monclient: authenticate success, global_id 4295 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7af251700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.813+0000 7fa7af251700 4 set_mon_vals no callback set 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 10 monclient: shutdown 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 -- 172.21.15.47:0/1377481507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 msgr2=0x7fa7a80a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 --2- 172.21.15.47:0/1377481507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 0x7fa7a80a5c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098f0 tx=0x7fa79c009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 -- 172.21.15.47:0/1377481507 shutdown_connections 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 --2- 172.21.15.47:0/1377481507 >> v2:172.21.15.47:3300/0 conn(0x7fa7a81385b0 0x7fa7a80a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 --2- 172.21.15.47:0/1377481507 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa7a8137d90 0x7fa7a80a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 --2- 172.21.15.47:0/1377481507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a8138170 0x7fa7a80a5c90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.814+0000 7fa7af251700 1 -- 172.21.15.47:0/1377481507 >> 172.21.15.47:0/1377481507 conn(0x7fa7a80aa8b0 msgr2=0x7fa7a80a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.816+0000 7fa7af251700 1 -- 172.21.15.47:0/1377481507 shutdown_connections 2024-04-20T02:15:49.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.819+0000 7fa7af251700 1 -- 172.21.15.47:0/1377481507 wait complete. 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.819+0000 7fa7af251700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 10 monclient: monmap: 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.820 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 1 librados: starting msgr at 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 1 librados: starting objecter 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 1 Processor -- start 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 1 -- start start 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 1 librados: setting wanted keys 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 1 librados: calling monclient init 2024-04-20T02:15:49.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.820+0000 7fa7af251700 10 monclient: init 2024-04-20T02:15:49.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient: _add_conns ranks=[3,4,2,5,6,8,0,1,7] 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a8137d90 0x7fa7a81363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): picked mon.noname-g con 0x7fa7a8137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 0x7fa7a8136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): picked mon.noname-c con 0x7fa7a8138170 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a81385b0 0x7fa7a81290e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): picked mon.noname-b con 0x7fa7a81385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a812bbd0 con 0x7fa7a81385b0 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a812be20 con 0x7fa7a8137d90 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a809e9e0 con 0x7fa7a8138170 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.821+0000 7fa7af251700 10 monclient(hunting): authenticate will time out at 1629.222761s 2024-04-20T02:15:49.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 0x7fa7a8136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 0x7fa7a8136920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44498/0 (socket says 172.21.15.47:44498) 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 -- 172.21.15.47:0/2344119567 learned_addr learned my addr 172.21.15.47:0/2344119567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): get_auth_request con 0x7fa7a8138170 auth_method 0 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a81385b0 0x7fa7a81290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): get_auth_request con 0x7fa7a81385b0 auth_method 0 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7ad7ee700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient(hunting): handle_auth_done global_id 4342 payload 824 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 -- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a8137d90 msgr2=0x7fa7a81363e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a8137d90 0x7fa7a81363e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 -- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a81385b0 msgr2=0x7fa7a81290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a81385b0 0x7fa7a81290e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 monclient: found mon.noname-c 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.822+0000 7fa7a7fff700 1 -- 172.21.15.47:0/2344119567 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c009550 con 0x7fa7a8138170 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a7fff700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 0x7fa7a8136920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009f50 tx=0x7fa79c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 client.?.objecter ms_handle_connect 0x7fa7a8138170 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:49.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa79c004310 con 0x7fa7a8138170 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: dump: 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.826 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c004490 con 0x7fa7a8138170 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa79c01a420 con 0x7fa7a8138170 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a57fa700 10 monclient: dump: 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.827 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7a6ffd700 4 set_mon_vals no callback set 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 5 monclient: authenticate success, global_id 4342 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 10 monclient: _renew_subs 2024-04-20T02:15:49.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:49.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a809dc20 con 0x7fa7a8138170 2024-04-20T02:15:49.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 10 client.4342.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:49.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 10 monclient: _renew_subs 2024-04-20T02:15:49.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:49.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a812b580 con 0x7fa7a8138170 2024-04-20T02:15:49.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.823+0000 7fa7af251700 1 librados: init done 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.825+0000 7f53fc688700 10 monclient: get_monmap_and_config 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.825+0000 7f53fc688700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.825+0000 7f53fc688700 10 monclient: monmap: 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:49.826348+0000 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:49.826348+0000 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.830 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7f53fc688700 1 Processor -- start 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7f53fc688700 1 -- start start 2024-04-20T02:15:49.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7f53fc688700 10 monclient: init 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa79c0036a0 con 0x7fa7a8138170 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa78c03a2f0 0x7fa78c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fa79c04b2b0 con 0x7fa7a8138170 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 10 client.4342.objecter ms_dispatch 0x7fa7a8001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 3 client.4342.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 3 client.4342.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:49.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.826+0000 7fa7a57fa700 20 client.4342.objecter dump_active .. 0 homeless 2024-04-20T02:15:49.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient: _add_conns ranks=[2,5,7,6,10,0,4,9,3,8,1] 2024-04-20T02:15:49.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53f4137d90 0x7f53f40a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): picked mon.noname-b con 0x7f53f4137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53f4138170 0x7f53f40a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): picked mon.noname-h con 0x7f53f4138170 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53f41385b0 0x7f53f40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): picked mon.noname-i con 0x7f53f41385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f409fa40 con 0x7f53f4137d90 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f409dfc0 con 0x7f53f4138170 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40a7bc0 con 0x7f53f41385b0 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fc688700 10 monclient(hunting): authenticate will time out at 1629.228441s 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53f4138170 0x7f53f40a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53f4138170 0x7f53f40a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:42432/0 (socket says 172.21.15.47:42432) 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 1 -- 172.21.15.47:0/3888643425 learned_addr learned my addr 172.21.15.47:0/3888643425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 10 monclient(hunting): get_auth_request con 0x7f53f4138170 auth_method 0 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53f9c23700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fa424700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53f4137d90 0x7f53f40a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fa424700 10 monclient(hunting): get_auth_request con 0x7f53f4137d90 auth_method 0 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fa424700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fa424700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fa424700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fac25700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53f41385b0 0x7f53f40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fac25700 10 monclient(hunting): get_auth_request con 0x7f53f41385b0 auth_method 0 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fac25700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fac25700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.827+0000 7f53fac25700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fa424700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fa424700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fa424700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9c23700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9c23700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9c23700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 10 monclient(hunting): handle_auth_done global_id 4318 payload 274 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 -- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53f4138170 msgr2=0x7f53f40a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53f4138170 0x7f53f40a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 -- 172.21.15.47:0/3888643425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53f4137d90 msgr2=0x7f53f40a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53f4137d90 0x7f53f40a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 monclient: found mon.noname-i 2024-04-20T02:15:49.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 -- 172.21.15.47:0/3888643425 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40751e0 con 0x7f53f41385b0 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fa424700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53f4137d90 0x7f53f40a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fac25700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53f41385b0 0x7f53f40a61d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009df0 tx=0x7f53ec00b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=560de659fce602e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 1 -- 172.21.15.47:0/3888643425 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53ec00e070 con 0x7f53f41385b0 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 10 monclient: dump: 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.837 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.838 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53f9422700 1 -- 172.21.15.47:0/3888643425 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec012430 con 0x7f53f41385b0 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fc688700 5 monclient: authenticate success, global_id 4318 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.828+0000 7f53fc688700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53f9422700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53f9422700 1 -- 172.21.15.47:0/3888643425 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53ec012a10 con 0x7f53f41385b0 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53f9422700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53f9422700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53f9422700 10 monclient: dump: 2024-04-20T02:15:49.839 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.840 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7fa7af251700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7fa7af251700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7fa7af251700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a80a6710 con 0x7fa7a8138170 2024-04-20T02:15:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53fc688700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:49.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.829+0000 7f53fc688700 4 set_mon_vals no callback set 2024-04-20T02:15:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.830+0000 7fa7acfed700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa78c03a2f0 0x7fa78c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.830+0000 7fa7acfed700 10 monclient: get_auth_request con 0x7fa78c03a2f0 auth_method 0 2024-04-20T02:15:49.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.830+0000 7f53fc688700 10 monclient: shutdown 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.830+0000 7f53fc688700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.830+0000 7f53fc688700 1 -- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53f41385b0 msgr2=0x7f53f40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.830+0000 7f53fc688700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53f41385b0 0x7f53f40a61d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009df0 tx=0x7f53ec00b330 comp rx=0 tx=0).stop 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.832+0000 7f53fc688700 1 -- 172.21.15.47:0/3888643425 shutdown_connections 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.832+0000 7f53fc688700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53f41385b0 0x7f53f40a61d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.832+0000 7f53fc688700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53f4137d90 0x7f53f40a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.832+0000 7f53fc688700 1 --2- 172.21.15.47:0/3888643425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53f4138170 0x7f53f40a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.832+0000 7f53fc688700 1 -- 172.21.15.47:0/3888643425 >> 172.21.15.47:0/3888643425 conn(0x7f53f40aa8b0 msgr2=0x7f53f40a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.834+0000 7fa7a57fa700 10 client.4342.objecter ms_handle_connect 0x7fa78c03a2f0 2024-04-20T02:15:49.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.834+0000 7fa7acfed700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa78c03a2f0 0x7fa78c03c7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa794009710 tx=0x7fa794006c60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.835+0000 7f53fc688700 1 -- 172.21.15.47:0/3888643425 shutdown_connections 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.835+0000 7f53fc688700 1 -- 172.21.15.47:0/3888643425 wait complete. 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 10 monclient: monmap: 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.846 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 1 librados: starting msgr at 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 1 librados: starting objecter 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 1 Processor -- start 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 1 -- start start 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 1 librados: setting wanted keys 2024-04-20T02:15:49.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 1 librados: calling monclient init 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.836+0000 7f53fc688700 10 monclient: init 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient: _add_conns ranks=[4,8,3,0,5,1,7,6,2] 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53f4137d90 0x7f53f4136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): picked mon.noname-c con 0x7f53f4137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 0x7f53f4137430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): picked mon.noname-e con 0x7f53f4138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53f41385b0 0x7f53f41cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): picked mon.noname-g con 0x7f53f41385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4071fe0 con 0x7f53f41385b0 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41d1040 con 0x7f53f4137d90 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41222f0 con 0x7f53f4138170 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53fc688700 10 monclient(hunting): authenticate will time out at 1629.238522s 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 0x7f53f4137430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 0x7f53f4137430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43712/0 (socket says 172.21.15.47:43712) 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 1 -- 172.21.15.47:0/2059517078 learned_addr learned my addr 172.21.15.47:0/2059517078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 10 monclient(hunting): get_auth_request con 0x7f53f4138170 auth_method 0 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.837+0000 7f53f9c23700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 10 monclient(hunting): handle_auth_done global_id 4301 payload 824 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 -- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53f4137d90 msgr2=0x7f53f4136ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53f4137d90 0x7f53f4136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 -- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53f41385b0 msgr2=0x7f53f41cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53f41385b0 0x7f53f41cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 monclient: found mon.noname-e 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 -- 172.21.15.47:0/2059517078 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80094a0 con 0x7f53f4138170 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53f9c23700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 0x7f53f4137430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e8e0 tx=0x7f53f000eca0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53e6ffd700 10 client.?.objecter ms_handle_connect 0x7f53f4138170 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53e6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53f000eee0 con 0x7f53f4138170 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53e6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53e6ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.838+0000 7f53e6ffd700 10 monclient: dump: 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.850 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.840+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa79c00be40 con 0x7fa7a8138170 2024-04-20T02:15:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.840+0000 7fa7a57fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.840+0000 7fa7a57fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f000ccc0 con 0x7f53f4138170 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53f000ce30 con 0x7f53f4138170 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53f8c21700 4 set_mon_vals no callback set 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53e6ffd700 10 monclient: dump: 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.853 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 5 monclient: authenticate success, global_id 4301 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 10 monclient: _renew_subs 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec009a70 con 0x7f53f4138170 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 10 client.4301.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:49.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 10 monclient: _renew_subs 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f409dc20 con 0x7f53f4138170 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.839+0000 7f53fc688700 1 librados: init done 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.841+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f53f000c850 con 0x7f53f4138170 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.841+0000 7f53e6ffd700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f53d803a2f0 0x7f53d803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.841+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f53f004a270 con 0x7f53f4138170 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.841+0000 7f53e6ffd700 10 client.4301.objecter ms_dispatch 0x7f53f4001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.841+0000 7f53e6ffd700 3 client.4301.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.841+0000 7f53e6ffd700 3 client.4301.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.842+0000 7f53e6ffd700 20 client.4301.objecter dump_active .. 0 homeless 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.842+0000 7f53fa424700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f53d803a2f0 0x7f53d803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.842+0000 7f53fa424700 10 monclient: get_auth_request con 0x7f53d803a2f0 auth_method 0 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.842+0000 7f53e6ffd700 10 client.4301.objecter ms_handle_connect 0x7f53d803a2f0 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.842+0000 7f53fa424700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f53d803a2f0 0x7f53d803c7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009c90 tx=0x7f53e8009730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.844+0000 7f53fc688700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.844+0000 7f53fc688700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.844+0000 7f53fc688700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.844+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f409e430 con 0x7f53f4138170 2024-04-20T02:15:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.853+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f53f0013470 con 0x7f53f4138170 2024-04-20T02:15:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.854+0000 7f53e6ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.854+0000 7f53e6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.874+0000 7fd5371f3700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-04-20T02:15:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.874+0000 7fd5371f3700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-04-20T02:15:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.874+0000 7fd5371f3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.874+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd53009e430 con 0x7fd5301385b0 2024-04-20T02:15:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.875+0000 7fd52d7fa700 1 -- 172.21.15.47:0/438313651 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5280122e0 con 0x7fd5301385b0 2024-04-20T02:15:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.875+0000 7fd52d7fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-04-20T02:15:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.875+0000 7fd52d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:49.875 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.878+0000 7fd5371f3700 10 librados: watch_flush enter 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 10 librados: watch_flush exit 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 20 client.4292.objecter shutdown clearing up homeless session... 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 10 client.4292.objecter successfully canceled tick 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd51c03a320 msgr2=0x7fd51c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd51c03a320 0x7fd51c03c7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd520000c00 tx=0x7fd520008030 comp rx=0 tx=0).stop 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 10 monclient: shutdown 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd5301385b0 msgr2=0x7fd530137500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd5301385b0 0x7fd530137500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd528007cf0 tx=0x7fd52800e0e0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 shutdown_connections 2024-04-20T02:15:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd51c03a320 0x7fd51c03c7e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd5301385b0 0x7fd530137500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd530137d90 0x7fd5301c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 --2- 172.21.15.47:0/438313651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd530138170 0x7fd530136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.879+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 >> 172.21.15.47:0/438313651 conn(0x7fd5300aa8b0 msgr2=0x7fd5300a3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.880+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 shutdown_connections 2024-04-20T02:15:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.880+0000 7fd5371f3700 1 -- 172.21.15.47:0/438313651 wait complete. 2024-04-20T02:15:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.880+0000 7fd5371f3700 1 librados: shutdown 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.890+0000 7fab4e4c7700 10 monclient: get_monmap_and_config 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.890+0000 7fab4e4c7700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.890+0000 7fab4e4c7700 10 monclient: monmap: 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:49.892096+0000 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:49.892096+0000 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:49.891 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.891+0000 7fab4e4c7700 1 Processor -- start 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 -- start start 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient: init 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient: _add_conns ranks=[5,7,10,6,1,0,4,2,9,3,8] 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fab480a2870 0x7fab480a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): picked mon.noname-h con 0x7fab480a2870 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:49.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 0x7fab48137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): picked mon.noname-i con 0x7fab480a3190 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fab48137ff0 0x7fab48138480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): picked mon.noname-f con 0x7fab48137ff0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab481389c0 con 0x7fab48137ff0 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab4809f130 con 0x7fab480a2870 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fab480673b0 con 0x7fab480a3190 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.892+0000 7fab4e4c7700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.893+0000 7fab4e4c7700 10 monclient(hunting): authenticate will time out at 1629.294026s 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 0x7fab48137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 0x7fab48137ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:55050/0 (socket says 172.21.15.47:55050) 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 -- 172.21.15.47:0/332528477 learned_addr learned my addr 172.21.15.47:0/332528477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): get_auth_request con 0x7fab480a3190 auth_method 0 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab47fff700 1 --2- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fab480a2870 0x7fab480a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab4ca64700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fab48137ff0 0x7fab48138480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab47fff700 10 monclient(hunting): get_auth_request con 0x7fab480a2870 auth_method 0 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab47fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab47fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab47fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient(hunting): handle_auth_done global_id 4327 payload 274 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 -- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fab480a2870 msgr2=0x7fab480a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 --2- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fab480a2870 0x7fab480a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 -- 172.21.15.47:0/332528477 >> v2:172.21.15.179:3300/0 conn(0x7fab48137ff0 msgr2=0x7fab48138480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 --2- 172.21.15.47:0/332528477 >> v2:172.21.15.179:3300/0 conn(0x7fab48137ff0 0x7fab48138480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 monclient: found mon.noname-i 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 -- 172.21.15.47:0/332528477 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480751e0 con 0x7fab480a3190 2024-04-20T02:15:49.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab477fe700 1 --2- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 0x7fab48137ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098f0 tx=0x7fab3c009c00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=af7d9183fbc55d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.894+0000 7fab46ffd700 1 -- 172.21.15.47:0/332528477 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab3c00bb10 con 0x7fab480a3190 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: dump: 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.898 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 1 -- 172.21.15.47:0/332528477 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c00bc70 con 0x7fab480a3190 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 1 -- 172.21.15.47:0/332528477 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab3c00bde0 con 0x7fab480a3190 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab46ffd700 10 monclient: dump: 2024-04-20T02:15:49.899 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.900 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 5 monclient: authenticate success, global_id 4327 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 4 set_mon_vals no callback set 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 10 monclient: shutdown 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 1 -- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 msgr2=0x7fab48137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.895+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 0x7fab48137ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098f0 tx=0x7fab3c009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 -- 172.21.15.47:0/332528477 shutdown_connections 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/332528477 >> v2:172.21.15.179:3300/0 conn(0x7fab48137ff0 0x7fab48138480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fab480a2870 0x7fab480a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/332528477 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fab480a3190 0x7fab48137ab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 -- 172.21.15.47:0/332528477 >> 172.21.15.47:0/332528477 conn(0x7fab480a4010 msgr2=0x7fab4806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 -- 172.21.15.47:0/332528477 shutdown_connections 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 1 -- 172.21.15.47:0/332528477 wait complete. 2024-04-20T02:15:49.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.896+0000 7fab4e4c7700 10 monclient: monmap: 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.902 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 1 librados: starting msgr at 2024-04-20T02:15:49.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 1 librados: starting objecter 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 1 Processor -- start 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 1 -- start start 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 1 librados: setting wanted keys 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 1 librados: calling monclient init 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.897+0000 7fab4e4c7700 10 monclient: init 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient: _add_conns ranks=[0,3,1,2,5,8,6,4,7] 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 0x7fab480713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): picked mon.noname-a con 0x7fab480a2870 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab480a3190 0x7fab48131c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): picked mon.noname-g con 0x7fab480a3190 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab48137ff0 0x7fab48132180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): picked mon.noname-f con 0x7fab48137ff0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4809c440 con 0x7fab480a2870 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480720c0 con 0x7fab48137ff0 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4809e510 con 0x7fab480a3190 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.898+0000 7fab4e4c7700 10 monclient(hunting): authenticate will time out at 1629.299661s 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 0x7fab480713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 0x7fab480713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56934/0 (socket says 172.21.15.47:56934) 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 -- 172.21.15.47:0/2629770008 learned_addr learned my addr 172.21.15.47:0/2629770008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): get_auth_request con 0x7fab480a2870 auth_method 0 2024-04-20T02:15:49.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab4ca64700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab48137ff0 0x7fab48132180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab477fe700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab480a3190 0x7fab48131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab4ca64700 10 monclient(hunting): get_auth_request con 0x7fab48137ff0 auth_method 0 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab4ca64700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab4ca64700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab4ca64700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.4 2024-04-20T02:15:49.908 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:49.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient(hunting): handle_auth_done global_id 4383 payload 824 2024-04-20T02:15:49.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 -- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab480a3190 msgr2=0x7fab48131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab480a3190 0x7fab48131c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 -- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab48137ff0 msgr2=0x7fab48132180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab48137ff0 0x7fab48132180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.899+0000 7fab47fff700 1 monclient: found mon.noname-a 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab47fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab47fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab47fff700 1 -- 172.21.15.47:0/2629770008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c009550 con 0x7fab480a2870 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab47fff700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 0x7fab480713d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e810 tx=0x7fab3800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 client.?.objecter ms_handle_connect 0x7fab480a2870 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab38013880 con 0x7fab480a2870 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: dump: 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.909 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38013a20 con 0x7fab480a2870 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fab38012370 con 0x7fab480a2870 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab44ff9700 10 monclient: dump: 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.911 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.900+0000 7fab467fc700 4 set_mon_vals no callback set 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 5 monclient: authenticate success, global_id 4383 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 10 monclient: _renew_subs 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4809c100 con 0x7fab480a2870 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 10 client.4383.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 10 monclient: _renew_subs 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4809c5f0 con 0x7fab480a2870 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.901+0000 7fab4e4c7700 1 librados: init done 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fab38013b90 con 0x7fab480a2870 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fab3403a2f0 0x7fab3403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab4e4c7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fab38049cb0 con 0x7fab480a2870 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 10 client.4383.objecter ms_dispatch 0x7fab48001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 3 client.4383.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 3 client.4383.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab4e4c7700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab4e4c7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4809c7e0 con 0x7fab480a2870 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.903+0000 7fab44ff9700 20 client.4383.objecter dump_active .. 0 homeless 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.906+0000 7fab477fe700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fab3403a2f0 0x7fab3403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.906+0000 7fab477fe700 10 monclient: get_auth_request con 0x7fab3403a2f0 auth_method 0 2024-04-20T02:15:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.907+0000 7fab477fe700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fab3403a2f0 0x7fab3403c7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098f0 tx=0x7fab3c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.908+0000 7fab44ff9700 10 client.4383.objecter ms_handle_connect 0x7fab3403a2f0 2024-04-20T02:15:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.909+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fab38013e40 con 0x7fab480a2870 2024-04-20T02:15:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.909+0000 7fab44ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.909+0000 7fab44ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.912+0000 7ffbb04f6700 10 monclient: get_monmap_and_config 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.912+0000 7ffbb04f6700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.912+0000 7ffbb04f6700 10 monclient: monmap: 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:49.913355+0000 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:49.913355+0000 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:49.916 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.912+0000 7ffbb04f6700 1 Processor -- start 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 1 -- start start 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient: init 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient: _add_conns ranks=[2,9,1,3,6,7,0,8,5,4,10] 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 0x7ffba80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): picked mon.noname-b con 0x7ffba8137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffba8138170 0x7ffba80a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): picked mon.noname-a con 0x7ffba8138170 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffba81385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80751e0 con 0x7ffba81385b0 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba809fa40 con 0x7ffba8138170 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba809dfc0 con 0x7ffba8137d90 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.913+0000 7ffbb04f6700 10 monclient(hunting): authenticate will time out at 1629.314875s 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 0x7ffba80a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 0x7ffba80a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53124/0 (socket says 172.21.15.47:53124) 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 -- 172.21.15.47:0/1978756872 learned_addr learned my addr 172.21.15.47:0/1978756872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): get_auth_request con 0x7ffba8137d90 auth_method 0 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbada91700 1 --2- 172.21.15.47:0/1978756872 >> v2:172.21.15.47:3300/0 conn(0x7ffba8138170 0x7ffba80a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbada91700 10 monclient(hunting): get_auth_request con 0x7ffba8138170 auth_method 0 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbada91700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbada91700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbada91700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient(hunting): handle_auth_done global_id 4322 payload 274 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 -- 172.21.15.47:0/1978756872 >> v2:172.21.15.47:3300/0 conn(0x7ffba8138170 msgr2=0x7ffba80a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:49.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 --2- 172.21.15.47:0/1978756872 >> v2:172.21.15.47:3300/0 conn(0x7ffba8138170 0x7ffba80a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 -- 172.21.15.47:0/1978756872 >> v1:172.21.15.179:6789/0 conn(0x7ffba81385b0 legacy=0x7ffba80a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 monclient: found mon.noname-b 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 -- 172.21.15.47:0/1978756872 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80a7bc0 con 0x7ffba8137d90 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbae292700 1 --2- 172.21.15.47:0/1978756872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 0x7ffba80a09e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffba40098f0 tx=0x7ffba4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68f63142528a83b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbad290700 1 -- 172.21.15.47:0/1978756872 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffba400bb10 con 0x7ffba8137d90 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.914+0000 7ffbad290700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: dump: 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.920 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 1 -- 172.21.15.47:0/1978756872 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba400bc70 con 0x7ffba8137d90 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 1 -- 172.21.15.47:0/1978756872 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffba400bde0 con 0x7ffba8137d90 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbad290700 10 monclient: dump: 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.922 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 5 monclient: authenticate success, global_id 4322 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 4 set_mon_vals no callback set 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 10 monclient: shutdown 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1978756872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 msgr2=0x7ffba80a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.915+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1978756872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 0x7ffba80a09e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffba40098f0 tx=0x7ffba4009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.917+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1978756872 shutdown_connections 2024-04-20T02:15:49.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.917+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1978756872 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffba8137d90 0x7ffba80a09e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.917+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1978756872 >> v2:172.21.15.47:3300/0 conn(0x7ffba8138170 0x7ffba80a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.917+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1978756872 >> 172.21.15.47:0/1978756872 conn(0x7ffba80aa8b0 msgr2=0x7ffba80a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.919+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1978756872 shutdown_connections 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.919+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1978756872 wait complete. 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.919+0000 7ffbb04f6700 10 monclient: build_initial_monmap 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.919+0000 7ffbb04f6700 10 monclient: monmap: 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:49.926 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 librados: starting msgr at 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 librados: starting objecter 2024-04-20T02:15:49.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 Processor -- start 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 -- start start 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 librados: setting wanted keys 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 librados: calling monclient init 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient: init 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient: _add_conns ranks=[5,1,3,0,2,4,6,7,8] 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffba8137d90 0x7ffba81c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): picked mon.noname-h con 0x7ffba8137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba8138170 0x7ffba8136ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): picked mon.noname-f con 0x7ffba8138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 0x7ffba8137420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): picked mon.noname-g con 0x7ffba81385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba81242e0 con 0x7ffba8138170 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba809ea50 con 0x7ffba81385b0 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffba809dc20 con 0x7ffba8137d90 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.920+0000 7ffbb04f6700 10 monclient(hunting): authenticate will time out at 1629.321777s 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 0x7ffba8137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba8138170 0x7ffba8136ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 0x7ffba8137420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:48866/0 (socket says 172.21.15.47:48866) 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba8138170 0x7ffba8136ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:48238/0 (socket says 172.21.15.47:48238) 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 -- 172.21.15.47:0/1039131270 learned_addr learned my addr 172.21.15.47:0/1039131270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): get_auth_request con 0x7ffba81385b0 auth_method 0 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): get_auth_request con 0x7ffba8138170 auth_method 0 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbada91700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient(hunting): handle_auth_done global_id 4314 payload 824 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 -- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffba8137d90 msgr2=0x7ffba81c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffba8137d90 0x7ffba81c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 -- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba8138170 msgr2=0x7ffba8136ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba8138170 0x7ffba8136ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient: _finish_hunting 0 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 monclient: found mon.noname-g 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 -- 172.21.15.47:0/1039131270 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009550 con 0x7ffba81385b0 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffbaea93700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 0x7ffba8137420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffba000cb10 tx=0x7ffba000ce20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffb9effd700 10 client.?.objecter ms_handle_connect 0x7ffba81385b0 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffb9effd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffba0003e70 con 0x7ffba81385b0 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.921+0000 7ffb9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: dump: 2024-04-20T02:15:49.931 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: _finish_auth 0 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000a860 con 0x7ffba81385b0 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffba000ae40 con 0x7ffba81385b0 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbaca8f700 4 set_mon_vals no callback set 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffb9effd700 10 monclient: dump: 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:49.933 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 5 monclient: authenticate success, global_id 4314 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 10 monclient: _renew_subs 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba81c9910 con 0x7ffba81385b0 2024-04-20T02:15:49.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 10 client.4314.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 10 monclient: _renew_subs 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba8124000 con 0x7ffba81385b0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.922+0000 7ffbb04f6700 1 librados: init done 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.923+0000 7ffbb04f6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.923+0000 7ffbb04f6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.923+0000 7ffbb04f6700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.923+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba806bca0 con 0x7ffba81385b0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.923+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ffba000a9d0 con 0x7ffba81385b0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb8c03a2f0 0x7ffb8c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7ffba0049f00 con 0x7ffba81385b0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 10 client.4314.objecter ms_dispatch 0x7ffba8001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 3 client.4314.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 3 client.4314.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 20 client.4314.objecter dump_active .. 0 homeless 2024-04-20T02:15:49.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffbae292700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb8c03a2f0 0x7ffb8c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffbae292700 10 monclient: get_auth_request con 0x7ffb8c03a2f0 auth_method 0 2024-04-20T02:15:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffb9effd700 10 client.4314.objecter ms_handle_connect 0x7ffb8c03a2f0 2024-04-20T02:15:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.924+0000 7ffbae292700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb8c03a2f0 0x7ffb8c03c7b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009130 tx=0x7ffba4010000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.929+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ffba000ac80 con 0x7ffba81385b0 2024-04-20T02:15:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.929+0000 7ffb9effd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:49.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.929+0000 7ffb9effd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:49.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.965+0000 7fc50a619700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-04-20T02:15:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.965+0000 7fc50a619700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-04-20T02:15:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.965+0000 7fc50a619700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.965+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc50409ff80 con 0x7fc5040a0d90 2024-04-20T02:15:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.966+0000 7fc500ff9700 1 -- 172.21.15.47:0/3323017544 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4f8013720 con 0x7fc5040a0d90 2024-04-20T02:15:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.966+0000 7fc500ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-04-20T02:15:49.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.966+0000 7fc500ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:49.966 INFO:teuthology.orchestra.run.smithi047.stdout:51539607555 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 10 librados: watch_flush enter 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 10 librados: watch_flush exit 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 20 client.4263.objecter shutdown clearing up homeless session... 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 10 client.4263.objecter successfully canceled tick 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4f003a2f0 msgr2=0x7fc4f003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4f003a2f0 0x7fc4f003c7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4006b20 tx=0x7fc4f4008040 comp rx=0 tx=0).stop 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 10 monclient: shutdown 2024-04-20T02:15:49.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 msgr2=0x7fc5041ba970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5041ba970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800ebe0 tx=0x7fc4f800eef0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 shutdown_connections 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4f003a2f0 0x7fc4f003c7b0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4006b20 tx=0x7fc4f4008040 comp rx=0 tx=0).stop 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc50409b6e0 0x7fc5041c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc5040a2950 0x7fc5041cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 --2- 172.21.15.47:0/3323017544 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc5040a0d90 0x7fc5041ba970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.969+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 >> 172.21.15.47:0/3323017544 conn(0x7fc5040a7570 msgr2=0x7fc5041bd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.970+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 shutdown_connections 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.970+0000 7fc50a619700 1 -- 172.21.15.47:0/3323017544 wait complete. 2024-04-20T02:15:49.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:49.970+0000 7fc50a619700 1 librados: shutdown 2024-04-20T02:15:49.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607555 got 51539607555 for osd.3 2024-04-20T02:15:49.990 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53fc688700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53fc688700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53fc688700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53f4122530 con 0x7f53f4138170 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53e6ffd700 1 -- 172.21.15.47:0/2059517078 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f53f001b3b0 con 0x7f53f4138170 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53e6ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.093+0000 7f53e6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:50.093 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 10 librados: watch_flush enter 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 10 librados: watch_flush exit 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 20 client.4301.objecter shutdown clearing up homeless session... 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 10 client.4301.objecter successfully canceled tick 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f53d803a2f0 msgr2=0x7f53d803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f53d803a2f0 0x7f53d803c7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009c90 tx=0x7f53e8009730 comp rx=0 tx=0).stop 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 10 monclient: shutdown 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 msgr2=0x7f53f4137430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 0x7f53f4137430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f000e8e0 tx=0x7f53f000eca0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 shutdown_connections 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f53d803a2f0 0x7f53d803c7b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53f41385b0 0x7f53f41cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53f4137d90 0x7f53f4136ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 --2- 172.21.15.47:0/2059517078 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53f4138170 0x7f53f4137430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 >> 172.21.15.47:0/2059517078 conn(0x7f53f40aa8b0 msgr2=0x7f53f40a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 shutdown_connections 2024-04-20T02:15:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 -- 172.21.15.47:0/2059517078 wait complete. 2024-04-20T02:15:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.097+0000 7f53fc688700 1 librados: shutdown 2024-04-20T02:15:50.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.1 2024-04-20T02:15:50.111 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.133+0000 7fab4e4c7700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.133+0000 7fab4e4c7700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.133+0000 7fab4e4c7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.133+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fab4809d730 con 0x7fab480a2870 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.133+0000 7fab44ff9700 1 -- 172.21.15.47:0/2629770008 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab3802a090 con 0x7fab480a2870 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.134+0000 7fab44ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.134+0000 7fab44ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:50.133 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:50.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7af251700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-04-20T02:15:50.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7af251700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-04-20T02:15:50.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7af251700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:50.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa7a809e6c0 con 0x7fa7a8138170 2024-04-20T02:15:50.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7a57fa700 1 -- 172.21.15.47:0/2344119567 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa79c015020 con 0x7fa7a8138170 2024-04-20T02:15:50.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7a57fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-04-20T02:15:50.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.136+0000 7fa7a57fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:50.136 INFO:teuthology.orchestra.run.smithi047.stdout:47244640260 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 10 librados: watch_flush enter 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 10 librados: watch_flush exit 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 20 client.4383.objecter shutdown clearing up homeless session... 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 10 client.4383.objecter successfully canceled tick 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fab3403a2f0 msgr2=0x7fab3403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fab3403a2f0 0x7fab3403c7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0098f0 tx=0x7fab3c008030 comp rx=0 tx=0).stop 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 10 monclient: shutdown 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 msgr2=0x7fab480713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 0x7fab480713d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab3800e810 tx=0x7fab3800ebd0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 shutdown_connections 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.137+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fab3403a2f0 0x7fab3403c7b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab48137ff0 0x7fab48132180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab480a2870 0x7fab480713d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 --2- 172.21.15.47:0/2629770008 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fab480a3190 0x7fab48131c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 >> 172.21.15.47:0/2629770008 conn(0x7fab480a4010 msgr2=0x7fab480a02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 shutdown_connections 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 -- 172.21.15.47:0/2629770008 wait complete. 2024-04-20T02:15:50.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fab4e4c7700 1 librados: shutdown 2024-04-20T02:15:50.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fa7af251700 10 librados: watch_flush enter 2024-04-20T02:15:50.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fa7af251700 10 librados: watch_flush exit 2024-04-20T02:15:50.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fa7af251700 20 client.4342.objecter shutdown clearing up homeless session... 2024-04-20T02:15:50.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.138+0000 7fa7af251700 10 client.4342.objecter successfully canceled tick 2024-04-20T02:15:50.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.139+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa78c03a2f0 msgr2=0x7fa78c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.139+0000 7fa7af251700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa78c03a2f0 0x7fa78c03c7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa794009710 tx=0x7fa794006c60 comp rx=0 tx=0).stop 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.139+0000 7fa7af251700 10 monclient: shutdown 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.139+0000 7fa7af251700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.139+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 msgr2=0x7fa7a8136920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.139+0000 7fa7af251700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 0x7fa7a8136920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009f50 tx=0x7fa79c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 shutdown_connections 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa78c03a2f0 0x7fa78c03c7b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa7a81385b0 0x7fa7a81290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa7a8137d90 0x7fa7a81363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 --2- 172.21.15.47:0/2344119567 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa7a8138170 0x7fa7a8136920 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 >> 172.21.15.47:0/2344119567 conn(0x7fa7a80aa8b0 msgr2=0x7fa7a80a1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 shutdown_connections 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 -- 172.21.15.47:0/2344119567 wait complete. 2024-04-20T02:15:50.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.140+0000 7fa7af251700 1 librados: shutdown 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.152+0000 7ffbb04f6700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.152+0000 7ffbb04f6700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.152+0000 7ffbb04f6700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.152+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffba809e430 con 0x7ffba81385b0 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.153+0000 7ffb9effd700 1 -- 172.21.15.47:0/1039131270 <== mon.3 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffba00102e0 con 0x7ffba81385b0 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.153+0000 7ffb9effd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-04-20T02:15:50.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.153+0000 7ffb9effd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:50.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.2 2024-04-20T02:15:50.153 INFO:teuthology.orchestra.run.smithi047.stdout:51539607556 2024-04-20T02:15:50.154 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:50.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.5 2024-04-20T02:15:50.155 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 10 librados: watch_flush enter 2024-04-20T02:15:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 10 librados: watch_flush exit 2024-04-20T02:15:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 20 client.4314.objecter shutdown clearing up homeless session... 2024-04-20T02:15:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 10 client.4314.objecter successfully canceled tick 2024-04-20T02:15:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb8c03a2f0 msgr2=0x7ffb8c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb8c03a2f0 0x7ffb8c03c7b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009130 tx=0x7ffba4010000 comp rx=0 tx=0).stop 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 10 monclient: shutdown 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 msgr2=0x7ffba8137420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 0x7ffba8137420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffba000cb10 tx=0x7ffba000ce20 comp rx=0 tx=0).stop 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 shutdown_connections 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffba81385b0 0x7ffba8137420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffba8137d90 0x7ffba81c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb8c03a2f0 0x7ffb8c03c7b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 --2- 172.21.15.47:0/1039131270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba8138170 0x7ffba8136ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 >> 172.21.15.47:0/1039131270 conn(0x7ffba80aa8b0 msgr2=0x7ffba809cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 shutdown_connections 2024-04-20T02:15:50.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 -- 172.21.15.47:0/1039131270 wait complete. 2024-04-20T02:15:50.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.156+0000 7ffbb04f6700 1 librados: shutdown 2024-04-20T02:15:50.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.0 2024-04-20T02:15:50.169 DEBUG:teuthology.parallel:result is None 2024-04-20T02:15:50.169 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T02:15:50.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.352+0000 7fbcdd019700 10 monclient: get_monmap_and_config 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.352+0000 7fbcdd019700 10 monclient: build_initial_monmap 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.352+0000 7fbcdd019700 10 monclient: monmap: 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:50.353925+0000 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:50.353925+0000 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:50.352 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:50.353 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.353+0000 7fbcdd019700 1 Processor -- start 2024-04-20T02:15:50.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.353+0000 7fbcdd019700 1 -- start start 2024-04-20T02:15:50.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.353+0000 7fbcdd019700 10 monclient: init 2024-04-20T02:15:50.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.353+0000 7fbcdd019700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:50.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient: _add_conns ranks=[1,9,8,0,2,7,4,10,6,3,5] 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbcd80a5370 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 0x7fbcd813ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): picked mon.noname-a con 0x7fbcd80a4280 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 0x7fbcd813d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): picked mon.noname-e con 0x7fbcd80a4660 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80751e0 con 0x7fbcd80a5370 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd80a5750 con 0x7fbcd80a4280 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd809f160 con 0x7fbcd80a4660 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcdd019700 10 monclient(hunting): authenticate will time out at 1629.755153s 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 0x7fbcd813ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 0x7fbcd813d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 0x7fbcd813d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43714/0 (socket says 172.21.15.47:43714) 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 0x7fbcd813ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56946/0 (socket says 172.21.15.47:56946) 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 -- 172.21.15.47:0/4171793268 learned_addr learned my addr 172.21.15.47:0/4171793268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): get_auth_request con 0x7fbcd80a4660 auth_method 0 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): get_auth_request con 0x7fbcd80a4280 auth_method 0 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient(hunting): handle_auth_done global_id 4310 payload 274 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 -- 172.21.15.47:0/4171793268 >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 msgr2=0x7fbcd813ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 --2- 172.21.15.47:0/4171793268 >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 0x7fbcd813ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 -- 172.21.15.47:0/4171793268 >> v1:172.21.15.179:6789/0 conn(0x7fbcd80a5370 legacy=0x7fbcd80a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient: _finish_hunting 0 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 monclient: found mon.noname-e 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 -- 172.21.15.47:0/4171793268 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80a01b0 con 0x7fbcd80a4660 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcce59c700 1 --2- 172.21.15.47:0/4171793268 >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 0x7fbcd813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd759e700 1 --2- 172.21.15.47:0/4171793268 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 0x7fbcd813d2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009e50 tx=0x7fbcd000b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6502f3a7358a9321 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd659c700 1 -- 172.21.15.47:0/4171793268 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcd000e070 con 0x7fbcd80a4660 2024-04-20T02:15:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd659c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd659c700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.354+0000 7fbcd659c700 10 monclient: dump: 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 10 monclient: _finish_auth 0 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 10 monclient: _check_auth_tickets 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 1 -- 172.21.15.47:0/4171793268 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd000bd80 con 0x7fbcd80a4660 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 5 monclient: authenticate success, global_id 4310 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 1 -- 172.21.15.47:0/4171793268 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcd0012860 con 0x7fbcd80a4660 2024-04-20T02:15:50.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 4 set_mon_vals no callback set 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcd659c700 10 monclient: dump: 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.369 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 10 monclient: shutdown 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 -- 172.21.15.47:0/4171793268 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 msgr2=0x7fbcd813d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 --2- 172.21.15.47:0/4171793268 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 0x7fbcd813d2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009e50 tx=0x7fbcd000b330 comp rx=0 tx=0).stop 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 -- 172.21.15.47:0/4171793268 shutdown_connections 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 --2- 172.21.15.47:0/4171793268 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbcd80a4660 0x7fbcd813d2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 --2- 172.21.15.47:0/4171793268 >> v2:172.21.15.47:3300/0 conn(0x7fbcd80a4280 0x7fbcd813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 -- 172.21.15.47:0/4171793268 >> 172.21.15.47:0/4171793268 conn(0x7fbcd80a8790 msgr2=0x7fbcd80a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 -- 172.21.15.47:0/4171793268 shutdown_connections 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.355+0000 7fbcdd019700 1 -- 172.21.15.47:0/4171793268 wait complete. 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.356+0000 7fbcdd019700 10 monclient: build_initial_monmap 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.356+0000 7fbcdd019700 10 monclient: monmap: 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:50.371 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.356+0000 7fbcdd019700 1 librados: starting msgr at 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.356+0000 7fbcdd019700 1 librados: starting objecter 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.356+0000 7fbcdd019700 1 Processor -- start 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 -- start start 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 librados: setting wanted keys 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 librados: calling monclient init 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient: init 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:50.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient: _add_conns ranks=[7,6,0,8,1,3,4,5,2] 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a4280 0x7fbcd81ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): picked mon.noname-i con 0x7fbcd80a4280 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbcd80a4660 0x7fbcd81ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): picked mon.noname-d con 0x7fbcd80a4660 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 0x7fbcd81bae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): picked mon.noname-a con 0x7fbcd80a5370 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd81a9e00 con 0x7fbcd80a5370 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd81ca610 con 0x7fbcd80a4660 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8071240 con 0x7fbcd80a4280 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcdd019700 10 monclient(hunting): authenticate will time out at 1629.758711s 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 0x7fbcd81bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbcd80a4660 0x7fbcd81ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 0x7fbcd81bae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56952/0 (socket says 172.21.15.47:56952) 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 1 -- 172.21.15.47:0/137719783 learned_addr learned my addr 172.21.15.47:0/137719783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): get_auth_request con 0x7fbcd80a5370 auth_method 0 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 10 monclient(hunting): get_auth_request con 0x7fbcd80a4660 auth_method 0 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd6d9d700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a4280 0x7fbcd81ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd6d9d700 10 monclient(hunting): get_auth_request con 0x7fbcd80a4280 auth_method 0 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcd759e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.357+0000 7fbcce59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcce59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 10 monclient(hunting): handle_auth_done global_id 4410 payload 824 2024-04-20T02:15:50.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 -- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a4280 msgr2=0x7fbcd81ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a4280 0x7fbcd81ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 -- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbcd80a4660 msgr2=0x7fbcd81ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbcd80a4660 0x7fbcd81ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 10 monclient: _finish_hunting 0 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 monclient: found mon.noname-a 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 -- 172.21.15.47:0/137719783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00094a0 con 0x7fbcd80a5370 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcce59c700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbcd80a4660 0x7fbcd81ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd759e700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 0x7fbcd81bae20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000b370 tx=0x7fbcd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 client.?.objecter ms_handle_connect 0x7fbcd80a5370 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcd00092e0 con 0x7fbcd80a5370 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:50.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: dump: 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: _finish_auth 0 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd00046c0 con 0x7fbcd80a5370 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 5 monclient: authenticate success, global_id 4410 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 10 monclient: _renew_subs 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd0009b30 con 0x7fbcd80a5370 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcd0004830 con 0x7fbcd80a5370 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcd5d9b700 4 set_mon_vals no callback set 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbccffff700 10 monclient: dump: 2024-04-20T02:15:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 10 client.4410.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 10 monclient: _renew_subs 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd81a9a20 con 0x7fbcd80a5370 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.358+0000 7fbcdd019700 1 librados: init done 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.359+0000 7fbcdd019700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.359+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbcd0004250 con 0x7fbcd80a5370 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.359+0000 7fbcdd019700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.359+0000 7fbcdd019700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.359+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd81a9f40 con 0x7fbcd80a5370 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcb803a220 0x7fbcb803c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fbcd004b820 con 0x7fbcd80a5370 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 10 client.4410.objecter ms_dispatch 0x7fbcd8001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 3 client.4410.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 3 client.4410.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 20 client.4410.objecter dump_active .. 0 homeless 2024-04-20T02:15:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbcd6d9d700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcb803a220 0x7fbcb803c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbcd6d9d700 10 monclient: get_auth_request con 0x7fbcb803a220 auth_method 0 2024-04-20T02:15:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbccffff700 10 client.4410.objecter ms_handle_connect 0x7fbcb803a220 2024-04-20T02:15:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.360+0000 7fbcd6d9d700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcb803a220 0x7fbcb803c6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8007fd0 tx=0x7fbcc8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.365+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fbcd81a9f40 con 0x7fbcd80a5370 2024-04-20T02:15:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.365+0000 7fbccffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.365+0000 7fbccffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:50.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.576+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcd80722a0 con 0x7fbcb803a220 2024-04-20T02:15:50.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.578+0000 7fbccffff700 1 -- 172.21.15.47:0/137719783 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40597 (secure 0 0 0) 0x7fbcc8001560 con 0x7fbcb803a220 2024-04-20T02:15:50.578 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 10 librados: watch_flush enter 2024-04-20T02:15:50.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 10 librados: watch_flush exit 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 20 client.4410.objecter shutdown clearing up homeless session... 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 10 client.4410.objecter successfully canceled tick 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcb803a220 msgr2=0x7fbcb803c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcb803a220 0x7fbcb803c6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8007fd0 tx=0x7fbcc8009040 comp rx=0 tx=0).stop 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 10 monclient: shutdown 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 msgr2=0x7fbcd81bae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 0x7fbcd81bae20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000b370 tx=0x7fbcd0004ad0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 shutdown_connections 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcb803a220 0x7fbcb803c6e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd80a5370 0x7fbcd81bae20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a4280 0x7fbcd81ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 --2- 172.21.15.47:0/137719783 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbcd80a4660 0x7fbcd81ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 >> 172.21.15.47:0/137719783 conn(0x7fbcd80a8790 msgr2=0x7fbcd80a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 shutdown_connections 2024-04-20T02:15:50.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 -- 172.21.15.47:0/137719783 wait complete. 2024-04-20T02:15:50.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.580+0000 7fbcdd019700 1 librados: shutdown 2024-04-20T02:15:50.582 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:15:50.593 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-20T02:15:50.288570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164288,"kb_used_data":1856,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066752,"statfs":{"total":579820584960,"available":579652354048,"internally_reserved":0,"allocated":1900544,"data_stored":1265374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":165927591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000583"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260273+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:15:47.260273+0000","last_peered":"2024-04-20T02:15:47.260273+0000","last_clean":"2024-04-20T02:15:47.260273+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:15:47.260273+0000","last_undegraded":"2024-04-20T02:15:47.260273+0000","last_fullsized":"2024-04-20T02:15:47.260273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260775+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:15:47.260775+0000","last_peered":"2024-04-20T02:15:47.260775+0000","last_clean":"2024-04-20T02:15:47.260775+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:15:47.260775+0000","last_undegraded":"2024-04-20T02:15:47.260775+0000","last_fullsized":"2024-04-20T02:15:47.260775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260018+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:15:47.260018+0000","last_peered":"2024-04-20T02:15:47.260018+0000","last_clean":"2024-04-20T02:15:47.260018+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:15:47.260018+0000","last_undegraded":"2024-04-20T02:15:47.260018+0000","last_fullsized":"2024-04-20T02:15:47.260018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.378678+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:15:47.378678+0000","last_peered":"2024-04-20T02:15:47.378678+0000","last_clean":"2024-04-20T02:15:47.378678+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:15:47.378678+0000","last_undegraded":"2024-04-20T02:15:47.378678+0000","last_fullsized":"2024-04-20T02:15:47.378678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.857583+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:15:47.857583+0000","last_peered":"2024-04-20T02:15:47.857583+0000","last_clean":"2024-04-20T02:15:47.857583+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:15:47.857583+0000","last_undegraded":"2024-04-20T02:15:47.857583+0000","last_fullsized":"2024-04-20T02:15:47.857583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260154+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:15:47.260154+0000","last_peered":"2024-04-20T02:15:47.260154+0000","last_clean":"2024-04-20T02:15:47.260154+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:15:47.260154+0000","last_undegraded":"2024-04-20T02:15:47.260154+0000","last_fullsized":"2024-04-20T02:15:47.260154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.259833+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:15:47.259833+0000","last_peered":"2024-04-20T02:15:47.259833+0000","last_clean":"2024-04-20T02:15:47.259833+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:15:47.259833+0000","last_undegraded":"2024-04-20T02:15:47.259833+0000","last_fullsized":"2024-04-20T02:15:47.259833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260745+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:15:47.260745+0000","last_peered":"2024-04-20T02:15:47.260745+0000","last_clean":"2024-04-20T02:15:47.260745+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:15:47.260745+0000","last_undegraded":"2024-04-20T02:15:47.260745+0000","last_fullsized":"2024-04-20T02:15:47.260745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.261769+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:15:47.261769+0000","last_peered":"2024-04-20T02:15:47.261769+0000","last_clean":"2024-04-20T02:15:47.261769+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:15:47.261769+0000","last_undegraded":"2024-04-20T02:15:47.261769+0000","last_fullsized":"2024-04-20T02:15:47.261769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1003520,"data_stored":996416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]}]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":684032,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":610304,"data_stored":507936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:15:50.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.729+0000 7f2ff850c700 10 monclient: get_monmap_and_config 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.729+0000 7f2ff850c700 10 monclient: build_initial_monmap 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.730+0000 7f2ff850c700 10 monclient: monmap: 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:50.731154+0000 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:50.731154+0000 2024-04-20T02:15:50.729 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:50.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.730+0000 7f2ff850c700 1 Processor -- start 2024-04-20T02:15:50.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.730+0000 7f2ff850c700 1 -- start start 2024-04-20T02:15:50.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.730+0000 7f2ff850c700 10 monclient: init 2024-04-20T02:15:50.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient: _add_conns ranks=[2,3,7,5,1,8,10,6,4,9,0] 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 0x7f2ff0099890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): picked mon.noname-b con 0x7f2ff009df00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ff00a1120 0x7f2ff00a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): picked mon.noname-g con 0x7f2ff00a1120 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff00a1a40 0x7f2ff013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): picked mon.noname-i con 0x7f2ff00a1a40 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff009ea50 con 0x7f2ff009df00 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff009ed30 con 0x7f2ff00a1120 2024-04-20T02:15:50.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff009f010 con 0x7f2ff00a1a40 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff850c700 10 monclient(hunting): authenticate will time out at 1630.132490s 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 0x7f2ff0099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 0x7f2ff0099890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53126/0 (socket says 172.21.15.47:53126) 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 -- 172.21.15.47:0/3670247446 learned_addr learned my addr 172.21.15.47:0/3670247446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): get_auth_request con 0x7f2ff009df00 auth_method 0 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff6aa9700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff00a1a40 0x7f2ff013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff5aa7700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ff00a1120 0x7f2ff00a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff6aa9700 10 monclient(hunting): get_auth_request con 0x7f2ff00a1a40 auth_method 0 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff6aa9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff6aa9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff6aa9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_done global_id 4331 payload 274 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 -- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff00a1a40 msgr2=0x7f2ff013e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff00a1a40 0x7f2ff013e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 -- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ff00a1120 msgr2=0x7f2ff00a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ff00a1120 0x7f2ff00a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient: _finish_hunting 0 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 monclient: found mon.noname-b 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff62a8700 1 -- 172.21.15.47:0/3670247446 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00751e0 con 0x7f2ff009df00 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff5aa7700 10 monclient: get_auth_request con 0x7f2ff00a1120 auth_method 0 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.731+0000 7f2ff5aa7700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ff00a1120 0x7f2ff00a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff62a8700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 0x7f2ff0099890 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009920 tx=0x7f2fe4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa873f8880b64fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 1 -- 172.21.15.47:0/3670247446 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fe4009e70 con 0x7f2ff009df00 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: dump: 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.745 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.746 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: _finish_auth 0 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: _check_auth_tickets 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 1 -- 172.21.15.47:0/3670247446 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe400bc70 con 0x7f2ff009df00 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 5 monclient: authenticate success, global_id 4331 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 1 -- 172.21.15.47:0/3670247446 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fe400bde0 con 0x7f2ff009df00 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 4 set_mon_vals no callback set 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff52a6700 10 monclient: dump: 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.748 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 10 monclient: shutdown 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 1 -- 172.21.15.47:0/3670247446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 msgr2=0x7f2ff0099890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.732+0000 7f2ff850c700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 0x7f2ff0099890 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009920 tx=0x7f2fe4009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 -- 172.21.15.47:0/3670247446 shutdown_connections 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff00a1a40 0x7f2ff013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ff009df00 0x7f2ff0099890 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 --2- 172.21.15.47:0/3670247446 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ff00a1120 0x7f2ff00a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 -- 172.21.15.47:0/3670247446 >> 172.21.15.47:0/3670247446 conn(0x7f2ff00a6720 msgr2=0x7f2ff00a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 -- 172.21.15.47:0/3670247446 shutdown_connections 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 1 -- 172.21.15.47:0/3670247446 wait complete. 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 10 monclient: build_initial_monmap 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.733+0000 7f2ff850c700 10 monclient: monmap: 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:50.750 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 librados: starting msgr at 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 librados: starting objecter 2024-04-20T02:15:50.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 Processor -- start 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 -- start start 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 librados: setting wanted keys 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 librados: calling monclient init 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient: init 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient: _add_conns ranks=[7,6,4,5,0,2,3,8,1] 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff009df00 0x7f2ff006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): picked mon.noname-i con 0x7f2ff009df00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ff00a1120 0x7f2ff0070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): picked mon.noname-d con 0x7f2ff00a1120 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 0x7f2ff0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): picked mon.noname-c con 0x7f2ff00a1a40 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:50.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0071ae0 con 0x7f2ff00a1a40 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00a0410 con 0x7f2ff00a1120 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0090520 con 0x7f2ff009df00 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff850c700 10 monclient(hunting): authenticate will time out at 1630.135781s 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 0x7f2ff0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff5aa7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ff00a1120 0x7f2ff0070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 0x7f2ff0070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44510/0 (socket says 172.21.15.47:44510) 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 1 -- 172.21.15.47:0/4080678358 learned_addr learned my addr 172.21.15.47:0/4080678358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff5aa7700 10 monclient(hunting): get_auth_request con 0x7f2ff00a1120 auth_method 0 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff5aa7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff5aa7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff5aa7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff62a8700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff009df00 0x7f2ff006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 10 monclient(hunting): get_auth_request con 0x7f2ff00a1a40 auth_method 0 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff6aa9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff62a8700 10 monclient(hunting): get_auth_request con 0x7f2ff009df00 auth_method 0 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff62a8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff62a8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.734+0000 7f2ff62a8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff6aa9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff6aa9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff6aa9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff62a8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 -- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff009df00 msgr2=0x7f2ff006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff009df00 0x7f2ff006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 -- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 msgr2=0x7f2ff0070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 0x7f2ff0070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 10 monclient: _finish_hunting 0 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 monclient: found mon.noname-d 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 -- 172.21.15.47:0/4080678358 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4009550 con 0x7f2ff00a1120 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff6aa9700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 0x7f2ff0070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff5aa7700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ff00a1120 0x7f2ff0070510 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c7c0 tx=0x7f2fec00cad0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 client.?.objecter ms_handle_connect 0x7f2ff00a1120 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fec0044a0 con 0x7f2ff00a1120 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: dump: 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.757 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec0036a0 con 0x7f2ff00a1120 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 5 monclient: authenticate success, global_id 4281 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 10 monclient: _renew_subs 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff0072d20 con 0x7f2ff00a1120 2024-04-20T02:15:50.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fec003c80 con 0x7f2ff00a1120 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2fe2ffd700 10 monclient: dump: 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:50.759 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff4aa5700 4 set_mon_vals no callback set 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 10 client.4281.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 10 monclient: _renew_subs 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff0098ed0 con 0x7f2ff00a1120 2024-04-20T02:15:50.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.735+0000 7f2ff850c700 1 librados: init done 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2fec003810 con 0x7f2ff00a1120 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2fd403a2f0 0x7f2fd403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f2fec049ad0 con 0x7f2ff00a1120 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 10 client.4281.objecter ms_dispatch 0x7f2ff0001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 3 client.4281.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 3 client.4281.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2ff850c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2ff850c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2ff850c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff0072a40 con 0x7f2ff00a1120 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2ff62a8700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2fd403a2f0 0x7f2fd403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:50.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2fe2ffd700 20 client.4281.objecter dump_active .. 0 homeless 2024-04-20T02:15:50.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.736+0000 7f2ff62a8700 10 monclient: get_auth_request con 0x7f2fd403a2f0 auth_method 0 2024-04-20T02:15:50.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.737+0000 7f2fe2ffd700 10 client.4281.objecter ms_handle_connect 0x7f2fd403a2f0 2024-04-20T02:15:50.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.737+0000 7f2ff62a8700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2fd403a2f0 0x7f2fd403c7b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009920 tx=0x7f2fe4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:50.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.742+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2fec004030 con 0x7f2ff00a1120 2024-04-20T02:15:50.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.742+0000 7f2fe2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:50.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.742+0000 7f2fe2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.952+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ff00903b0 con 0x7f2fd403a2f0 2024-04-20T02:15:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.954+0000 7f2fe2ffd700 1 -- 172.21.15.47:0/4080678358 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40597 (secure 0 0 0) 0x7f2ff00903b0 con 0x7f2fd403a2f0 2024-04-20T02:15:50.953 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 10 librados: watch_flush enter 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 10 librados: watch_flush exit 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 20 client.4281.objecter shutdown clearing up homeless session... 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 10 client.4281.objecter successfully canceled tick 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2fd403a2f0 msgr2=0x7f2fd403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2fd403a2f0 0x7f2fd403c7b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009920 tx=0x7f2fe4008030 comp rx=0 tx=0).stop 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 10 monclient: shutdown 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ff00a1120 msgr2=0x7f2ff0070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:50.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ff00a1120 0x7f2ff0070510 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c7c0 tx=0x7f2fec00cad0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 shutdown_connections 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2fd403a2f0 0x7f2fd403c7b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2ff00a1a40 0x7f2ff0070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2ff009df00 0x7f2ff006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 --2- 172.21.15.47:0/4080678358 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ff00a1120 0x7f2ff0070510 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 >> 172.21.15.47:0/4080678358 conn(0x7f2ff00a6720 msgr2=0x7f2ff01bfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 shutdown_connections 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 -- 172.21.15.47:0/4080678358 wait complete. 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:50.956+0000 7f2ff850c700 1 librados: shutdown 2024-04-20T02:15:50.957 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:15:50.970 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-20T02:15:50.288570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164288,"kb_used_data":1856,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066752,"statfs":{"total":579820584960,"available":579652354048,"internally_reserved":0,"allocated":1900544,"data_stored":1265374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":165927591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000583"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260273+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:15:47.260273+0000","last_peered":"2024-04-20T02:15:47.260273+0000","last_clean":"2024-04-20T02:15:47.260273+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:15:47.260273+0000","last_undegraded":"2024-04-20T02:15:47.260273+0000","last_fullsized":"2024-04-20T02:15:47.260273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260775+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:15:47.260775+0000","last_peered":"2024-04-20T02:15:47.260775+0000","last_clean":"2024-04-20T02:15:47.260775+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:15:47.260775+0000","last_undegraded":"2024-04-20T02:15:47.260775+0000","last_fullsized":"2024-04-20T02:15:47.260775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260018+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:15:47.260018+0000","last_peered":"2024-04-20T02:15:47.260018+0000","last_clean":"2024-04-20T02:15:47.260018+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:15:47.260018+0000","last_undegraded":"2024-04-20T02:15:47.260018+0000","last_fullsized":"2024-04-20T02:15:47.260018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.378678+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:15:47.378678+0000","last_peered":"2024-04-20T02:15:47.378678+0000","last_clean":"2024-04-20T02:15:47.378678+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:15:47.378678+0000","last_undegraded":"2024-04-20T02:15:47.378678+0000","last_fullsized":"2024-04-20T02:15:47.378678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.857583+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:15:47.857583+0000","last_peered":"2024-04-20T02:15:47.857583+0000","last_clean":"2024-04-20T02:15:47.857583+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:15:47.857583+0000","last_undegraded":"2024-04-20T02:15:47.857583+0000","last_fullsized":"2024-04-20T02:15:47.857583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260154+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:15:47.260154+0000","last_peered":"2024-04-20T02:15:47.260154+0000","last_clean":"2024-04-20T02:15:47.260154+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:15:47.260154+0000","last_undegraded":"2024-04-20T02:15:47.260154+0000","last_fullsized":"2024-04-20T02:15:47.260154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.259833+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:15:47.259833+0000","last_peered":"2024-04-20T02:15:47.259833+0000","last_clean":"2024-04-20T02:15:47.259833+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:15:47.259833+0000","last_undegraded":"2024-04-20T02:15:47.259833+0000","last_fullsized":"2024-04-20T02:15:47.259833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260745+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:15:47.260745+0000","last_peered":"2024-04-20T02:15:47.260745+0000","last_clean":"2024-04-20T02:15:47.260745+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:15:47.260745+0000","last_undegraded":"2024-04-20T02:15:47.260745+0000","last_fullsized":"2024-04-20T02:15:47.260745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.261769+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:15:47.261769+0000","last_peered":"2024-04-20T02:15:47.261769+0000","last_clean":"2024-04-20T02:15:47.261769+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:15:47.261769+0000","last_undegraded":"2024-04-20T02:15:47.261769+0000","last_fullsized":"2024-04-20T02:15:47.261769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1003520,"data_stored":996416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]}]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":684032,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":610304,"data_stored":507936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:15:50.971 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T02:15:50.971 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T02:15:50.971 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T02:15:50.972 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.105+0000 7f43c9fad700 10 monclient: get_monmap_and_config 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.105+0000 7f43c9fad700 10 monclient: build_initial_monmap 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.105+0000 7f43c9fad700 10 monclient: monmap: 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:51.106999+0000 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:51.106999+0000 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:51.105 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:51.106 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.106+0000 7f43c9fad700 1 Processor -- start 2024-04-20T02:15:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.106+0000 7f43c9fad700 1 -- start start 2024-04-20T02:15:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.106+0000 7f43c9fad700 10 monclient: init 2024-04-20T02:15:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient: _add_conns ranks=[1,9,0,10,7,6,2,8,5,3,4] 2024-04-20T02:15:51.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f43c40a2950 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43c40a3440 0x7f43c413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient(hunting): picked mon.noname-a con 0x7f43c40a3440 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f43c413d500 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40751e0 con 0x7f43c413d500 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c409e900 con 0x7f43c40a2950 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c409ef20 con 0x7f43c40a3440 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c9fad700 10 monclient(hunting): authenticate will time out at 1630.508347s 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f43c40a3440 0x7f43c413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f43c413d500 0x7f43c413f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54150/0 (socket says 172.21.15.47:54150) 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c3fff700 1 -- 172.21.15.47:0/3684773740 learned_addr learned my addr 172.21.15.47:0/3684773740 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): get_auth_request con 0x7f43c40a3440 auth_method 0 2024-04-20T02:15:51.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_done global_id 4419 payload 274 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 1 -- 172.21.15.47:0/3684773740 >> v1:172.21.15.179:6789/0 conn(0x7f43c40a2950 legacy=0x7f43c40a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 1 -- 172.21.15.47:0/3684773740 >> v1:172.21.15.47:6789/0 conn(0x7f43c413d500 legacy=0x7f43c413f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 1 monclient: found mon.noname-a 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c27fc700 1 -- 172.21.15.47:0/3684773740 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646966850 0 0) 0x7f43c409e900 con 0x7f43c40a2950 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:51.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c2ffd700 1 -- 172.21.15.47:0/3684773740 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4067390 con 0x7f43c40a3440 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.107+0000 7f43c27fc700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c2ffd700 1 --2- 172.21.15.47:0/3684773740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c413cfc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009840 tx=0x7f43ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c923072b10e5035 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 1 -- 172.21.15.47:0/3684773740 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43ac0044a0 con 0x7f43c40a3440 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: dump: 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.118 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 5 monclient: authenticate success, global_id 4419 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:51.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 1 -- 172.21.15.47:0/3684773740 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac0036a0 con 0x7f43c40a3440 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 4 set_mon_vals no callback set 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 1 -- 172.21.15.47:0/3684773740 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43ac003c80 con 0x7f43c40a3440 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c27fc700 10 monclient: dump: 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.120 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 10 monclient: shutdown 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 1 -- 172.21.15.47:0/3684773740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 msgr2=0x7f43c413cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3684773740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c413cfc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009840 tx=0x7f43ac009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 1 -- 172.21.15.47:0/3684773740 shutdown_connections 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3684773740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c413cfc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.108+0000 7f43c9fad700 1 -- 172.21.15.47:0/3684773740 >> 172.21.15.47:0/3684773740 conn(0x7f43c40a7570 msgr2=0x7f43c40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 1 -- 172.21.15.47:0/3684773740 shutdown_connections 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 1 -- 172.21.15.47:0/3684773740 wait complete. 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 10 monclient: build_initial_monmap 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 10 monclient: monmap: 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:51.122 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 1 librados: starting msgr at 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 1 librados: starting objecter 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.109+0000 7f43c9fad700 1 Processor -- start 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 -- start start 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 librados: setting wanted keys 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 librados: calling monclient init 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient: init 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient: _add_conns ranks=[8,0,1,7,5,2,4,6,3] 2024-04-20T02:15:51.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f43c40a2950 0x7f43c4123130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): picked mon.noname-e con 0x7f43c40a2950 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c4123670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): picked mon.noname-a con 0x7f43c40a3440 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c413d500 0x7f43c4123bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): picked mon.noname-f con 0x7f43c413d500 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c413aa50 con 0x7f43c40a3440 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41266c0 con 0x7f43c413d500 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40720c0 con 0x7f43c40a2950 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c9fad700 10 monclient(hunting): authenticate will time out at 1630.511736s 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c4123670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f43c40a2950 0x7f43c4123130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c4123670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56970/0 (socket says 172.21.15.47:56970) 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 1 -- 172.21.15.47:0/3113900862 learned_addr learned my addr 172.21.15.47:0/3113900862 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): get_auth_request con 0x7f43c40a2950 auth_method 0 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 10 monclient(hunting): get_auth_request con 0x7f43c40a3440 auth_method 0 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.110+0000 7f43c37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c3fff700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c413d500 0x7f43c4123bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 10 monclient(hunting): handle_auth_done global_id 4319 payload 824 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 -- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c413d500 msgr2=0x7f43c4123bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c413d500 0x7f43c4123bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 -- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 msgr2=0x7f43c4123670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c4123670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 monclient: found mon.noname-e 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 -- 172.21.15.47:0/3113900862 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0094a0 con 0x7f43c40a2950 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c3fff700 10 monclient: get_auth_request con 0x7f43c413d500 auth_method 0 2024-04-20T02:15:51.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c3fff700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c413d500 0x7f43c4123bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c2ffd700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c4123670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c37fe700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f43c40a2950 0x7f43c4123130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f43b400c460 tx=0x7f43b400c770 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 client.?.objecter ms_handle_connect 0x7f43c40a2950 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43b4007ea0 con 0x7f43c40a2950 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: dump: 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.127 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: _finish_auth 0 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40052e0 con 0x7f43c40a2950 2024-04-20T02:15:51.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 5 monclient: authenticate success, global_id 4319 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43b4005450 con 0x7f43c40a2950 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c1ffb700 4 set_mon_vals no callback set 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43bbfff700 10 monclient: dump: 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.129 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 10 monclient: _renew_subs 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c409e560 con 0x7f43c40a2950 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 10 client.4319.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 10 monclient: _renew_subs 2024-04-20T02:15:51.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c41263e0 con 0x7f43c40a2950 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.111+0000 7f43c9fad700 1 librados: init done 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f43b4003680 con 0x7f43c40a2950 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43c9fad700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43c9fad700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43c9fad700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c4126900 con 0x7f43c40a2950 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43bbfff700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f43a403a2f0 0x7f43a403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f43b4048f70 con 0x7f43c40a2950 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43bbfff700 10 client.4319.objecter ms_dispatch 0x7f43c4001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43bbfff700 3 client.4319.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.112+0000 7f43bbfff700 3 client.4319.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.113+0000 7f43bbfff700 20 client.4319.objecter dump_active .. 0 homeless 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.113+0000 7f43c2ffd700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f43a403a2f0 0x7f43a403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.113+0000 7f43c2ffd700 10 monclient: get_auth_request con 0x7f43a403a2f0 auth_method 0 2024-04-20T02:15:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.113+0000 7f43c2ffd700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f43a403a2f0 0x7f43a403c7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009b90 tx=0x7f43ac01d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.113+0000 7f43bbfff700 10 client.4319.objecter ms_handle_connect 0x7f43a403a2f0 2024-04-20T02:15:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.118+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f43b4018860 con 0x7f43c40a2950 2024-04-20T02:15:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.118+0000 7f43bbfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.118+0000 7f43bbfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.382+0000 7f43c9fad700 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.382+0000 7f43c9fad700 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.382+0000 7f43c9fad700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.382+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f43c41248b0 con 0x7f43c40a2950 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.383+0000 7f43bbfff700 1 -- 172.21.15.47:0/3113900862 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f43b4017020 con 0x7f43c40a2950 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.383+0000 7f43bbfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-04-20T02:15:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.383+0000 7f43bbfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:51.383 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:51.383 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T02:15:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 10 librados: watch_flush enter 2024-04-20T02:15:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 10 librados: watch_flush exit 2024-04-20T02:15:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 20 client.4319.objecter shutdown clearing up homeless session... 2024-04-20T02:15:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 10 client.4319.objecter successfully canceled tick 2024-04-20T02:15:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f43a403a2f0 msgr2=0x7f43a403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f43a403a2f0 0x7f43a403c7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009b90 tx=0x7f43ac01d040 comp rx=0 tx=0).stop 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 10 monclient: shutdown 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f43c40a2950 msgr2=0x7f43c4123130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f43c40a2950 0x7f43c4123130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f43b400c460 tx=0x7f43b400c770 comp rx=0 tx=0).stop 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 shutdown_connections 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f43a403a2f0 0x7f43a403c7b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c413d500 0x7f43c4123bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f43c40a2950 0x7f43c4123130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 --2- 172.21.15.47:0/3113900862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43c40a3440 0x7f43c4123670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 >> 172.21.15.47:0/3113900862 conn(0x7f43c40a7570 msgr2=0x7f43c406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 shutdown_connections 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 -- 172.21.15.47:0/3113900862 wait complete. 2024-04-20T02:15:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.385+0000 7f43c9fad700 1 librados: shutdown 2024-04-20T02:15:51.398 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T02:15:51.399 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-04-20T02:15:51.412 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-04-20T02:15:51.412 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T02:15:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.548+0000 7f52d46c3700 10 monclient: get_monmap_and_config 2024-04-20T02:15:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.548+0000 7f52d46c3700 10 monclient: build_initial_monmap 2024-04-20T02:15:51.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.548+0000 7f52d46c3700 10 monclient: monmap: 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:51.550073+0000 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:51.550073+0000 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:51.549 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:51.550 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:51.550 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:51.550 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.549+0000 7f52d46c3700 1 Processor -- start 2024-04-20T02:15:51.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.549+0000 7f52d46c3700 1 -- start start 2024-04-20T02:15:51.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.549+0000 7f52d46c3700 10 monclient: init 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient: _add_conns ranks=[3,6,9,7,0,5,4,2,10,1,8] 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52cc0a0600 0x7f52cc0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): picked mon.noname-g con 0x7f52cc0a0600 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 0x7f52cc0a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): picked mon.noname-d con 0x7f52cc0a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52cc0a2950 0x7f52cc0a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): picked mon.noname-a con 0x7f52cc0a2950 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc072830 con 0x7f52cc0a2950 2024-04-20T02:15:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0a1460 con 0x7f52cc0a0600 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc067770 con 0x7f52cc0a01b0 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d46c3700 10 monclient(hunting): authenticate will time out at 1630.951353s 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 0x7f52cc0a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52cc0a2950 0x7f52cc0a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 0x7f52cc0a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52594/0 (socket says 172.21.15.47:52594) 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 -- 172.21.15.47:0/2385149268 learned_addr learned my addr 172.21.15.47:0/2385149268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): get_auth_request con 0x7f52cc0a01b0 auth_method 0 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): get_auth_request con 0x7f52cc0a2950 auth_method 0 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d245f700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52cc0a0600 0x7f52cc0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d245f700 10 monclient(hunting): get_auth_request con 0x7f52cc0a0600 auth_method 0 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d245f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d245f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d245f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 -- 172.21.15.47:0/2385149268 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52cc0a0600 msgr2=0x7f52cc0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52cc0a0600 0x7f52cc0a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 -- 172.21.15.47:0/2385149268 >> v2:172.21.15.47:3300/0 conn(0x7f52cc0a2950 msgr2=0x7f52cc0a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 --2- 172.21.15.47:0/2385149268 >> v2:172.21.15.47:3300/0 conn(0x7f52cc0a2950 0x7f52cc0a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient: _finish_hunting 0 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 monclient: found mon.noname-d 2024-04-20T02:15:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d1c5e700 1 -- 172.21.15.47:0/2385149268 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0751e0 con 0x7f52cc0a01b0 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d245f700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52cc0a0600 0x7f52cc0a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.550+0000 7f52d2c60700 1 --2- 172.21.15.47:0/2385149268 >> v2:172.21.15.47:3300/0 conn(0x7f52cc0a2950 0x7f52cc0a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d1c5e700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 0x7f52cc0a0f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0098f0 tx=0x7f52bc009c00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1316081a4a843c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 1 -- 172.21.15.47:0/2385149268 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f52bc011aa0 con 0x7f52cc0a01b0 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: dump: 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: _finish_auth 0 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: _check_auth_tickets 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 5 monclient: authenticate success, global_id 4290 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 1 -- 172.21.15.47:0/2385149268 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc011c00 con 0x7f52cc0a01b0 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 4 set_mon_vals no callback set 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 1 -- 172.21.15.47:0/2385149268 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f52bc011d70 con 0x7f52cc0a01b0 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d145d700 10 monclient: dump: 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 10 monclient: shutdown 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 -- 172.21.15.47:0/2385149268 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 msgr2=0x7f52cc0a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 0x7f52cc0a0f20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0098f0 tx=0x7f52bc009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 -- 172.21.15.47:0/2385149268 shutdown_connections 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 --2- 172.21.15.47:0/2385149268 >> v2:172.21.15.47:3300/0 conn(0x7f52cc0a2950 0x7f52cc0a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52cc0a0600 0x7f52cc0a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 --2- 172.21.15.47:0/2385149268 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52cc0a01b0 0x7f52cc0a0f20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 -- 172.21.15.47:0/2385149268 >> 172.21.15.47:0/2385149268 conn(0x7f52cc0a7570 msgr2=0x7f52cc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.551+0000 7f52d46c3700 1 -- 172.21.15.47:0/2385149268 shutdown_connections 2024-04-20T02:15:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.552+0000 7f52d46c3700 1 -- 172.21.15.47:0/2385149268 wait complete. 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.552+0000 7f52d46c3700 10 monclient: build_initial_monmap 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.552+0000 7f52d46c3700 10 monclient: monmap: 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.552+0000 7f52d46c3700 1 librados: starting msgr at 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.552+0000 7f52d46c3700 1 librados: starting objecter 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.552+0000 7f52d46c3700 1 Processor -- start 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 -- start start 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 librados: setting wanted keys 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 librados: calling monclient init 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient: init 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient: _add_conns ranks=[5,7,4,2,1,0,3,6,8] 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f52cc0a01b0 0x7f52cc1cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): picked mon.noname-h con 0x7f52cc0a01b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:51.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52cc0a0600 0x7f52cc1bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): picked mon.noname-i con 0x7f52cc0a0600 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 0x7f52cc1c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): picked mon.noname-c con 0x7f52cc0a2950 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc09e890 con 0x7f52cc0a2950 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc09e440 con 0x7f52cc0a01b0 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1ad5c0 con 0x7f52cc0a0600 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:51.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d46c3700 10 monclient(hunting): authenticate will time out at 1630.954560s 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 0x7f52cc1c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 0x7f52cc1c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44520/0 (socket says 172.21.15.47:44520) 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 1 -- 172.21.15.47:0/3477156159 learned_addr learned my addr 172.21.15.47:0/3477156159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): get_auth_request con 0x7f52cc0a2950 auth_method 0 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52cc0a0600 0x7f52cc1bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d2c60700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f52cc0a01b0 0x7f52cc1cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): get_auth_request con 0x7f52cc0a0600 auth_method 0 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 10 monclient(hunting): get_auth_request con 0x7f52cc0a01b0 auth_method 0 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d1c5e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.553+0000 7f52d245f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d245f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 10 monclient(hunting): handle_auth_done global_id 4360 payload 824 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 -- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52cc0a0600 msgr2=0x7f52cc1bfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52cc0a0600 0x7f52cc1bfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 -- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f52cc0a01b0 msgr2=0x7f52cc1cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f52cc0a01b0 0x7f52cc1cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 10 monclient: _finish_hunting 0 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 monclient: found mon.noname-c 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 -- 172.21.15.47:0/3477156159 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009a70 con 0x7f52cc0a2950 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d2c60700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 0x7f52cc1c0140 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52c400efc0 tx=0x7f52c400c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 client.?.objecter ms_handle_connect 0x7f52cc0a2950 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f52c4014480 con 0x7f52cc0a2950 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: dump: 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400f040 con 0x7f52cc0a2950 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 5 monclient: authenticate success, global_id 4360 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 10 monclient: _renew_subs 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc009550 con 0x7f52cc0a2950 2024-04-20T02:15:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f52c4014a90 con 0x7f52cc0a2950 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 10 client.4360.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d0c5c700 4 set_mon_vals no callback set 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52c2ffd700 10 monclient: dump: 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.578 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 10 monclient: _renew_subs 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc1ab1f0 con 0x7f52cc0a2950 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.554+0000 7f52d46c3700 1 librados: init done 2024-04-20T02:15:51.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f52c4014bf0 con 0x7f52cc0a2950 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52d46c3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52d46c3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52d46c3700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f52b003a2f0 0x7f52b003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc1c0680 con 0x7f52cc0a2950 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f52c404a420 con 0x7f52cc0a2950 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 10 client.4360.objecter ms_dispatch 0x7f52cc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 3 client.4360.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 3 client.4360.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52d245f700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f52b003a2f0 0x7f52b003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52d245f700 10 monclient: get_auth_request con 0x7f52b003a2f0 auth_method 0 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.555+0000 7f52c2ffd700 20 client.4360.objecter dump_active .. 0 homeless 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.556+0000 7f52c2ffd700 10 client.4360.objecter ms_handle_connect 0x7f52b003a2f0 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.556+0000 7f52d245f700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f52b003a2f0 0x7f52b003c7b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52c8006f10 tx=0x7f52c8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.560+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f52c4012930 con 0x7f52cc0a2950 2024-04-20T02:15:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.560+0000 7f52c2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.560+0000 7f52c2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.777+0000 7f52d46c3700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.777+0000 7f52d46c3700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.777+0000 7f52d46c3700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.777+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52cc0008d0 con 0x7f52cc0a2950 2024-04-20T02:15:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.778+0000 7f52c2ffd700 1 -- 172.21.15.47:0/3477156159 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11832 (secure 0 0 0) 0x7f52c4012340 con 0x7f52cc0a2950 2024-04-20T02:15:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.778+0000 7f52c2ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:15:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.778+0000 7f52c2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:51.778 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:51.778 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:15:47.245083+0000","last_up_change":"2024-04-20T02:15:43.202903+0000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T02:15:44.345718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T02:15:44.739933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":42309},{"type":"v1","addr":"172.21.15.47:6801","nonce":42309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":42309},{"type":"v1","addr":"172.21.15.47:6803","nonce":42309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":42309},{"type":"v1","addr":"172.21.15.47:6811","nonce":42309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":42309},{"type":"v1","addr":"172.21.15.47:6806","nonce":42309}]},"public_addr":"172.21.15.47:6801/42309","cluster_addr":"172.21.15.47:6803/42309","heartbeat_back_addr":"172.21.15.47:6811/42309","heartbeat_front_addr":"172.21.15.47:6806/42309","state":["exists","up"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":42310},{"type":"v1","addr":"172.21.15.47:6807","nonce":42310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":42310},{"type":"v1","addr":"172.21.15.47:6813","nonce":42310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6819","nonce":42310},{"type":"v1","addr":"172.21.15.47:6821","nonce":42310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":42310},{"type":"v1","addr":"172.21.15.47:6817","nonce":42310}]},"public_addr":"172.21.15.47:6807/42310","cluster_addr":"172.21.15.47:6813/42310","heartbeat_back_addr":"172.21.15.47:6821/42310","heartbeat_front_addr":"172.21.15.47:6817/42310","state":["exists","up"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":42311},{"type":"v1","addr":"172.21.15.47:6812","nonce":42311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":42311},{"type":"v1","addr":"172.21.15.47:6816","nonce":42311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":42311},{"type":"v1","addr":"172.21.15.47:6823","nonce":42311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":42311},{"type":"v1","addr":"172.21.15.47:6820","nonce":42311}]},"public_addr":"172.21.15.47:6812/42311","cluster_addr":"172.21.15.47:6816/42311","heartbeat_back_addr":"172.21.15.47:6823/42311","heartbeat_front_addr":"172.21.15.47:6820/42311","state":["exists","up"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":42294},{"type":"v1","addr":"172.21.15.179:6801","nonce":42294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":42294},{"type":"v1","addr":"172.21.15.179:6803","nonce":42294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":42294},{"type":"v1","addr":"172.21.15.179:6807","nonce":42294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":42294},{"type":"v1","addr":"172.21.15.179:6805","nonce":42294}]},"public_addr":"172.21.15.179:6801/42294","cluster_addr":"172.21.15.179:6803/42294","heartbeat_back_addr":"172.21.15.179:6807/42294","heartbeat_front_addr":"172.21.15.179:6805/42294","state":["exists","up"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":42295},{"type":"v1","addr":"172.21.15.179:6812","nonce":42295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":42295},{"type":"v1","addr":"172.21.15.179:6816","nonce":42295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":42295},{"type":"v1","addr":"172.21.15.179:6823","nonce":42295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":42295},{"type":"v1","addr":"172.21.15.179:6820","nonce":42295}]},"public_addr":"172.21.15.179:6812/42295","cluster_addr":"172.21.15.179:6816/42295","heartbeat_back_addr":"172.21.15.179:6823/42295","heartbeat_front_addr":"172.21.15.179:6820/42295","state":["exists","up"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":42296},{"type":"v1","addr":"172.21.15.179:6809","nonce":42296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":42296},{"type":"v1","addr":"172.21.15.179:6813","nonce":42296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":42296},{"type":"v1","addr":"172.21.15.179:6821","nonce":42296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":42296},{"type":"v1","addr":"172.21.15.179:6817","nonce":42296}]},"public_addr":"172.21.15.179:6809/42296","cluster_addr":"172.21.15.179:6813/42296","heartbeat_back_addr":"172.21.15.179:6821/42296","heartbeat_front_addr":"172.21.15.179:6817/42296","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.384773+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.348973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.447042+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.151528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:15:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.780+0000 7f52d46c3700 10 librados: watch_flush enter 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.780+0000 7f52d46c3700 10 librados: watch_flush exit 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.780+0000 7f52d46c3700 20 client.4360.objecter shutdown clearing up homeless session... 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.780+0000 7f52d46c3700 10 client.4360.objecter successfully canceled tick 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f52b003a2f0 msgr2=0x7f52b003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f52b003a2f0 0x7f52b003c7b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f52c8006f10 tx=0x7f52c8008040 comp rx=0 tx=0).stop 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 10 monclient: shutdown 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 msgr2=0x7f52cc1c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 0x7f52cc1c0140 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52c400efc0 tx=0x7f52c400c3a0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 shutdown_connections 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f52b003a2f0 0x7f52b003c7b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f52cc0a2950 0x7f52cc1c0140 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f52cc0a01b0 0x7f52cc1cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 --2- 172.21.15.47:0/3477156159 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52cc0a0600 0x7f52cc1bfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 >> 172.21.15.47:0/3477156159 conn(0x7f52cc0a7570 msgr2=0x7f52cc09c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 shutdown_connections 2024-04-20T02:15:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 -- 172.21.15.47:0/3477156159 wait complete. 2024-04-20T02:15:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.781+0000 7f52d46c3700 1 librados: shutdown 2024-04-20T02:15:51.794 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T02:15:44.345718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-20T02:15:44.739933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T02:15:51.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T02:15:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.932+0000 7f9c01bcc700 10 monclient: get_monmap_and_config 2024-04-20T02:15:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.932+0000 7f9c01bcc700 10 monclient: build_initial_monmap 2024-04-20T02:15:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.933+0000 7f9c01bcc700 10 monclient: monmap: 2024-04-20T02:15:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:51.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:51.934136+0000 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:51.934136+0000 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:51.933 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:51.934 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.933+0000 7f9c01bcc700 1 Processor -- start 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.933+0000 7f9c01bcc700 1 -- start start 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.933+0000 7f9c01bcc700 10 monclient: init 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient: _add_conns ranks=[4,0,2,5,10,6,1,8,3,7,9] 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9bfc071860 0x7f9bfc071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): picked mon.noname-c con 0x7f9bfc071860 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9bfc1344a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1348a0 0x7f9bfc0a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): picked mon.noname-b con 0x7f9bfc1348a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0751e0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc09ee80 con 0x7f9bfc1348a0 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0a0900 con 0x7f9bfc071860 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9c01bcc700 10 monclient(hunting): authenticate will time out at 1631.335424s 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfaffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9bfc1344a0 0x7f9bfc0a2120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54160/0 (socket says 172.21.15.47:54160) 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfaffd700 1 -- 172.21.15.47:0/1471858268 learned_addr learned my addr 172.21.15.47:0/1471858268 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9bfc071860 0x7f9bfc071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): get_auth_request con 0x7f9bfc071860 auth_method 0 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1348a0 0x7f9bfc0a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): get_auth_request con 0x7f9bfc1348a0 auth_method 0 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfa7fc700 1 -- 172.21.15.47:0/1471858268 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301339259 0 0) 0x7f9bfc0751e0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfa7fc700 10 monclient(hunting): _negotiate 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfa7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfa7fc700 10 monclient(hunting): my global_id is 4455 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfa7fc700 1 -- 172.21.15.47:0/1471858268 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be80035b0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_done global_id 4369 payload 274 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 1 -- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1348a0 msgr2=0x7f9bfc0a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1348a0 0x7f9bfc0a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 1 -- 172.21.15.47:0/1471858268 >> v1:172.21.15.47:6789/0 conn(0x7f9bfc1344a0 legacy=0x7f9bfc0a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 1 monclient: found mon.noname-c 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:51.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfb7fe700 1 -- 172.21.15.47:0/1471858268 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0a0060 con 0x7f9bfc071860 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.934+0000 7f9bfbfff700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1348a0 0x7f9bfc0a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfb7fe700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9bfc071860 0x7f9bfc071c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009870 tx=0x7f9bec009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=baa4dc78d07c729e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 1 -- 172.21.15.47:0/1471858268 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9bec011aa0 con 0x7f9bfc071860 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: dump: 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.947 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 5 monclient: authenticate success, global_id 4369 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 1 -- 172.21.15.47:0/1471858268 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec011c00 con 0x7f9bfc071860 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:51.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 4 set_mon_vals no callback set 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 1 -- 172.21.15.47:0/1471858268 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9bec011d70 con 0x7f9bfc071860 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9bfa7fc700 10 monclient: dump: 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 10 monclient: shutdown 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 1 -- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9bfc071860 msgr2=0x7f9bfc071c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9bfc071860 0x7f9bfc071c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009870 tx=0x7f9bec009b80 comp rx=0 tx=0).stop 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 1 -- 172.21.15.47:0/1471858268 shutdown_connections 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9bfc071860 0x7f9bfc071c40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/1471858268 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1348a0 0x7f9bfc0a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.935+0000 7f9c01bcc700 1 -- 172.21.15.47:0/1471858268 >> 172.21.15.47:0/1471858268 conn(0x7f9bfc0a6d40 msgr2=0x7f9bfc0a3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:51.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.936+0000 7f9c01bcc700 1 -- 172.21.15.47:0/1471858268 shutdown_connections 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.936+0000 7f9c01bcc700 1 -- 172.21.15.47:0/1471858268 wait complete. 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.936+0000 7f9c01bcc700 10 monclient: build_initial_monmap 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.936+0000 7f9c01bcc700 10 monclient: monmap: 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:51.951 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.936+0000 7f9c01bcc700 1 librados: starting msgr at 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.936+0000 7f9c01bcc700 1 librados: starting objecter 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 Processor -- start 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 -- start start 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 librados: setting wanted keys 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 librados: calling monclient init 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient: init 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient: _add_conns ranks=[0,2,1,5,8,4,7,3,6] 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 0x7f9bfc1cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): picked mon.noname-a con 0x7f9bfc071860 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:51.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 0x7f9bfc131280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): picked mon.noname-b con 0x7f9bfc1344a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bfc1348a0 0x7f9bfc1317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): picked mon.noname-f con 0x7f9bfc1348a0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc11d270 con 0x7f9bfc071860 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc09f8a0 con 0x7f9bfc1348a0 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc071370 con 0x7f9bfc1344a0 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9c01bcc700 10 monclient(hunting): authenticate will time out at 1631.338786s 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 0x7f9bfc131280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 0x7f9bfc1cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 0x7f9bfc1cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56992/0 (socket says 172.21.15.47:56992) 2024-04-20T02:15:51.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 1 -- 172.21.15.47:0/2660916328 learned_addr learned my addr 172.21.15.47:0/2660916328 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 0x7f9bfc131280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53148/0 (socket says 172.21.15.47:53148) 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 10 monclient(hunting): get_auth_request con 0x7f9bfc071860 auth_method 0 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfaffd700 10 monclient(hunting): get_auth_request con 0x7f9bfc1344a0 auth_method 0 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.937+0000 7f9bfaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bfc1348a0 0x7f9bfc1317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): get_auth_request con 0x7f9bfc1348a0 auth_method 0 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 10 monclient(hunting): handle_auth_done global_id 4349 payload 824 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 -- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bfc1348a0 msgr2=0x7f9bfc1317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bfc1348a0 0x7f9bfc1317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 -- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 msgr2=0x7f9bfc1cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 0x7f9bfc1cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:51.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 monclient: found mon.noname-b 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 -- 172.21.15.47:0/2660916328 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4009a70 con 0x7f9bfc1344a0 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfb7fe700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 0x7f9bfc1cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bfaffd700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 0x7f9bfc131280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0a7630 tx=0x7f9be400c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 client.?.objecter ms_handle_connect 0x7f9bfc1344a0 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9be400ce00 con 0x7f9bfc1344a0 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: dump: 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.956 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:51.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 5 monclient: authenticate success, global_id 4349 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be40070d0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 10 monclient: _renew_subs 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9be40076b0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec0094a0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf9ffb700 4 set_mon_vals no callback set 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9bf3fff700 10 monclient: dump: 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 10 client.4349.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 10 monclient: _renew_subs 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc11e680 con 0x7f9bfc1344a0 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.938+0000 7f9c01bcc700 1 librados: init done 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9be4007240 con 0x7f9bfc1344a0 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9c01bcc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9bdc03a2f0 0x7f9bdc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9c01bcc700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9c01bcc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc06aac0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f9be4048ea0 con 0x7f9bfc1344a0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 10 client.4349.objecter ms_dispatch 0x7f9bfc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 3 client.4349.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 3 client.4349.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bf3fff700 20 client.4349.objecter dump_active .. 0 homeless 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bfb7fe700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9bdc03a2f0 0x7f9bdc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.939+0000 7f9bfb7fe700 10 monclient: get_auth_request con 0x7f9bdc03a2f0 auth_method 0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.940+0000 7f9bf3fff700 10 client.4349.objecter ms_handle_connect 0x7f9bdc03a2f0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.940+0000 7f9bfb7fe700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9bdc03a2f0 0x7f9bdc03c7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009840 tx=0x7f9bec008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:51.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.945+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f9be4017020 con 0x7f9bfc1344a0 2024-04-20T02:15:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.945+0000 7f9bf3fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:51.945+0000 7f9bf3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.158+0000 7f9c01bcc700 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.158+0000 7f9c01bcc700 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.158+0000 7f9c01bcc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.158+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bfc06ace0 con 0x7f9bfc1344a0 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.158+0000 7f9bf3fff700 1 -- 172.21.15.47:0/2660916328 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9be40074f0 con 0x7f9bfc1344a0 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.159+0000 7f9bf3fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.159+0000 7f9bf3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:52.158 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 10 librados: watch_flush enter 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 10 librados: watch_flush exit 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 20 client.4349.objecter shutdown clearing up homeless session... 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 10 client.4349.objecter successfully canceled tick 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9bdc03a2f0 msgr2=0x7f9bdc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9bdc03a2f0 0x7f9bdc03c7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009840 tx=0x7f9bec008030 comp rx=0 tx=0).stop 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 10 monclient: shutdown 2024-04-20T02:15:52.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 msgr2=0x7f9bfc131280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 0x7f9bfc131280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0a7630 tx=0x7f9be400c700 comp rx=0 tx=0).stop 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 shutdown_connections 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9bdc03a2f0 0x7f9bdc03c7b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9bfc1348a0 0x7f9bfc1317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bfc071860 0x7f9bfc1cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 --2- 172.21.15.47:0/2660916328 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9bfc1344a0 0x7f9bfc131280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 >> 172.21.15.47:0/2660916328 conn(0x7f9bfc0a6d40 msgr2=0x7f9bfc1bf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 shutdown_connections 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 -- 172.21.15.47:0/2660916328 wait complete. 2024-04-20T02:15:52.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.161+0000 7f9c01bcc700 1 librados: shutdown 2024-04-20T02:15:52.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-20T02:15:52.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.317+0000 7ff5ca2bd700 10 monclient: get_monmap_and_config 2024-04-20T02:15:52.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.317+0000 7ff5ca2bd700 10 monclient: build_initial_monmap 2024-04-20T02:15:52.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.317+0000 7ff5ca2bd700 10 monclient: monmap: 2024-04-20T02:15:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:52.318958+0000 2024-04-20T02:15:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:52.318958+0000 2024-04-20T02:15:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:52.318 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:52.319 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:52.320 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.318+0000 7ff5ca2bd700 1 Processor -- start 2024-04-20T02:15:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.318+0000 7ff5ca2bd700 1 -- start start 2024-04-20T02:15:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.318+0000 7ff5ca2bd700 10 monclient: init 2024-04-20T02:15:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:52.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient: _add_conns ranks=[3,10,6,8,5,4,9,1,2,0,7] 2024-04-20T02:15:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c40a3810 0x7ff5c4137d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): picked mon.noname-g con 0x7ff5c40a3810 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff5c40a3bf0 0x7ff5c40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): picked mon.noname-f con 0x7ff5c40a3bf0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 0x7ff5c4134a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): picked mon.noname-d con 0x7ff5c41345b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c406d5a0 con 0x7ff5c40a3bf0 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c406e740 con 0x7ff5c40a3810 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c406e180 con 0x7ff5c41345b0 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5ca2bd700 10 monclient(hunting): authenticate will time out at 1631.720205s 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 0x7ff5c4134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 0x7ff5c4134a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52600/0 (socket says 172.21.15.47:52600) 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 -- 172.21.15.47:0/3319815060 learned_addr learned my addr 172.21.15.47:0/3319815060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): get_auth_request con 0x7ff5c41345b0 auth_method 0 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 1 --2- 172.21.15.47:0/3319815060 >> v2:172.21.15.179:3300/0 conn(0x7ff5c40a3bf0 0x7ff5c40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 1 --2- 172.21.15.47:0/3319815060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c40a3810 0x7ff5c4137d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): get_auth_request con 0x7ff5c40a3810 auth_method 0 2024-04-20T02:15:52.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): get_auth_request con 0x7ff5c40a3bf0 auth_method 0 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient(hunting): handle_auth_done global_id 4299 payload 274 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 -- 172.21.15.47:0/3319815060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c40a3810 msgr2=0x7ff5c4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 --2- 172.21.15.47:0/3319815060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c40a3810 0x7ff5c4137d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 -- 172.21.15.47:0/3319815060 >> v2:172.21.15.179:3300/0 conn(0x7ff5c40a3bf0 msgr2=0x7ff5c40a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 --2- 172.21.15.47:0/3319815060 >> v2:172.21.15.179:3300/0 conn(0x7ff5c40a3bf0 0x7ff5c40a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient: _finish_hunting 0 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 monclient: found mon.noname-d 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 -- 172.21.15.47:0/3319815060 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c40751e0 con 0x7ff5c41345b0 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.319+0000 7ff5c885a700 1 --2- 172.21.15.47:0/3319815060 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 0x7ff5c4134a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009e50 tx=0x7ff5bc00b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=53d74e43694a869c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 1 -- 172.21.15.47:0/3319815060 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5bc00e070 con 0x7ff5c41345b0 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: dump: 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 1 -- 172.21.15.47:0/3319815060 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc00bd80 con 0x7ff5c41345b0 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 5 monclient: authenticate success, global_id 4299 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 4 set_mon_vals no callback set 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 1 -- 172.21.15.47:0/3319815060 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5bc012860 con 0x7ff5c41345b0 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5c2ffd700 10 monclient: dump: 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.350 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 10 monclient: shutdown 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3319815060 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 msgr2=0x7ff5c4134a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3319815060 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 0x7ff5c4134a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009e50 tx=0x7ff5bc00b330 comp rx=0 tx=0).stop 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3319815060 shutdown_connections 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3319815060 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff5c41345b0 0x7ff5c4134a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3319815060 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5c40a3810 0x7ff5c4137d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3319815060 >> v2:172.21.15.179:3300/0 conn(0x7ff5c40a3bf0 0x7ff5c40a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3319815060 >> 172.21.15.47:0/3319815060 conn(0x7ff5c40a8430 msgr2=0x7ff5c40a4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3319815060 shutdown_connections 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.320+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3319815060 wait complete. 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.321+0000 7ff5ca2bd700 10 monclient: build_initial_monmap 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.321+0000 7ff5ca2bd700 10 monclient: monmap: 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.321+0000 7ff5ca2bd700 1 librados: starting msgr at 2024-04-20T02:15:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.321+0000 7ff5ca2bd700 1 librados: starting objecter 2024-04-20T02:15:52.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.321+0000 7ff5ca2bd700 1 Processor -- start 2024-04-20T02:15:52.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 -- start start 2024-04-20T02:15:52.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 librados: setting wanted keys 2024-04-20T02:15:52.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 librados: calling monclient init 2024-04-20T02:15:52.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient: init 2024-04-20T02:15:52.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient: _add_conns ranks=[8,7,2,4,1,6,0,3,5] 2024-04-20T02:15:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 0x7ff5c40a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): picked mon.noname-e con 0x7ff5c40a3810 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff5c40a3bf0 0x7ff5c4133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): picked mon.noname-i con 0x7ff5c40a3bf0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c41345b0 0x7ff5c411e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): picked mon.noname-b con 0x7ff5c41345b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c4120ce0 con 0x7ff5c41345b0 2024-04-20T02:15:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c41c9100 con 0x7ff5c40a3bf0 2024-04-20T02:15:52.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c411fb70 con 0x7ff5c40a3810 2024-04-20T02:15:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5ca2bd700 10 monclient(hunting): authenticate will time out at 1631.723515s 2024-04-20T02:15:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 0x7ff5c40a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c41345b0 0x7ff5c411e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 0x7ff5c40a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43738/0 (socket says 172.21.15.47:43738) 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 1 -- 172.21.15.47:0/3234419934 learned_addr learned my addr 172.21.15.47:0/3234419934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c41345b0 0x7ff5c411e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53150/0 (socket says 172.21.15.47:53150) 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): get_auth_request con 0x7ff5c40a3810 auth_method 0 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): get_auth_request con 0x7ff5c41345b0 auth_method 0 2024-04-20T02:15:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c37fe700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff5c40a3bf0 0x7ff5c4133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c37fe700 10 monclient(hunting): get_auth_request con 0x7ff5c40a3bf0 auth_method 0 2024-04-20T02:15:52.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c885a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 10 monclient(hunting): handle_auth_done global_id 4328 payload 824 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 1 -- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff5c40a3bf0 msgr2=0x7ff5c4133bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.322+0000 7ff5c3fff700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff5c40a3bf0 0x7ff5c4133bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 1 -- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c41345b0 msgr2=0x7ff5c411e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c41345b0 0x7ff5c411e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:52.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 1 monclient: found mon.noname-e 2024-04-20T02:15:52.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:52.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:52.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 1 -- 172.21.15.47:0/3234419934 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009b30 con 0x7ff5c40a3810 2024-04-20T02:15:52.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c3fff700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 0x7ff5c40a1690 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400e9b0 tx=0x7ff5b400ecc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 client.?.objecter ms_handle_connect 0x7ff5c40a3810 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b4009710 con 0x7ff5c40a3810 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: dump: 2024-04-20T02:15:52.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 5 monclient: authenticate success, global_id 4328 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400ccc0 con 0x7ff5c40a3810 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 10 monclient: _renew_subs 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c406d7e0 con 0x7ff5c40a3810 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:52.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b400ce30 con 0x7ff5c40a3810 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c0ff9700 10 monclient: dump: 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.369 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5c27fc700 4 set_mon_vals no callback set 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 10 client.4328.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 10 monclient: _renew_subs 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c40a2280 con 0x7ff5c40a3810 2024-04-20T02:15:52.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.323+0000 7ff5ca2bd700 1 librados: init done 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5ca2bd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5ca2bd700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5ca2bd700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff5b400c850 con 0x7ff5c40a3810 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c406dd00 con 0x7ff5c40a3810 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5c0ff9700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff5b003a2f0 0x7ff5b003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7ff5b404a1f0 con 0x7ff5c40a3810 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5c0ff9700 10 client.4328.objecter ms_dispatch 0x7ff5c4001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5c0ff9700 3 client.4328.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:52.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.324+0000 7ff5c0ff9700 3 client.4328.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.325+0000 7ff5c0ff9700 20 client.4328.objecter dump_active .. 0 homeless 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.325+0000 7ff5c37fe700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff5b003a2f0 0x7ff5b003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.325+0000 7ff5c37fe700 10 monclient: get_auth_request con 0x7ff5b003a2f0 auth_method 0 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.325+0000 7ff5c37fe700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff5b003a2f0 0x7ff5b003c7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40a20e0 tx=0x7ff5b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.325+0000 7ff5c0ff9700 10 client.4328.objecter ms_handle_connect 0x7ff5b003a2f0 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.329+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ff5c406dd00 con 0x7ff5c40a3810 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.329+0000 7ff5c0ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:52.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.329+0000 7ff5c0ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:52.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.540+0000 7ff5ca2bd700 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-04-20T02:15:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.540+0000 7ff5ca2bd700 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-04-20T02:15:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.540+0000 7ff5ca2bd700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.540+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff5c406dd00 con 0x7ff5c40a3810 2024-04-20T02:15:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.541+0000 7ff5c0ff9700 1 -- 172.21.15.47:0/3234419934 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff5c406dd00 con 0x7ff5c40a3810 2024-04-20T02:15:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.541+0000 7ff5c0ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-04-20T02:15:52.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.541+0000 7ff5c0ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:52.542 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 8 2024-04-20T02:15:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 10 librados: watch_flush enter 2024-04-20T02:15:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 10 librados: watch_flush exit 2024-04-20T02:15:52.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 20 client.4328.objecter shutdown clearing up homeless session... 2024-04-20T02:15:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 10 client.4328.objecter successfully canceled tick 2024-04-20T02:15:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff5b003a2f0 msgr2=0x7ff5b003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff5b003a2f0 0x7ff5b003c7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40a20e0 tx=0x7ff5b8008040 comp rx=0 tx=0).stop 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 10 monclient: shutdown 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 msgr2=0x7ff5c40a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 0x7ff5c40a1690 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400e9b0 tx=0x7ff5b400ecc0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 shutdown_connections 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff5b003a2f0 0x7ff5b003c7b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff5c41345b0 0x7ff5c411e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff5c40a3810 0x7ff5c40a1690 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 --2- 172.21.15.47:0/3234419934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff5c40a3bf0 0x7ff5c4133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 >> 172.21.15.47:0/3234419934 conn(0x7ff5c40a8430 msgr2=0x7ff5c40a6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 shutdown_connections 2024-04-20T02:15:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.543+0000 7ff5ca2bd700 1 -- 172.21.15.47:0/3234419934 wait complete. 2024-04-20T02:15:52.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.544+0000 7ff5ca2bd700 1 librados: shutdown 2024-04-20T02:15:52.557 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-04-20T02:15:52.557 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:15:52.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.692+0000 7faddbafd700 10 monclient: get_monmap_and_config 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.692+0000 7faddbafd700 10 monclient: build_initial_monmap 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.692+0000 7faddbafd700 10 monclient: monmap: 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:52.693996+0000 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:52.693996+0000 2024-04-20T02:15:52.693 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:52.694 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.693+0000 7faddbafd700 1 Processor -- start 2024-04-20T02:15:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.693+0000 7faddbafd700 1 -- start start 2024-04-20T02:15:52.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.693+0000 7faddbafd700 10 monclient: init 2024-04-20T02:15:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient: _add_conns ranks=[1,0,4,7,10,3,8,6,2,9,5] 2024-04-20T02:15:52.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fadd409e4e0 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fadd41344f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd4134960 0x7fadd413c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient(hunting): picked mon.noname-c con 0x7fadd4134960 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40751e0 con 0x7fadd41344f0 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40a05a0 con 0x7fadd409e4e0 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd409e8c0 con 0x7fadd4134960 2024-04-20T02:15:52.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7faddbafd700 10 monclient(hunting): authenticate will time out at 1632.095324s 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd9098700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fadd41344f0 0x7fadd4139100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54170/0 (socket says 172.21.15.47:54170) 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd9098700 1 -- 172.21.15.47:0/688856242 learned_addr learned my addr 172.21.15.47:0/688856242 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 1 --2- 172.21.15.47:0/688856242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd4134960 0x7fadd413c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): get_auth_request con 0x7fadd4134960 auth_method 0 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd8897700 1 -- 172.21.15.47:0/688856242 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344060052 0 0) 0x7fadd40751e0 con 0x7fadd41344f0 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd8897700 10 monclient(hunting): _negotiate 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd8897700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd8897700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd8897700 10 monclient(hunting): my global_id is 4473 2024-04-20T02:15:52.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadd8897700 1 -- 172.21.15.47:0/688856242 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadc00035b0 con 0x7fadd41344f0 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient(hunting): handle_auth_done global_id 4378 payload 274 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 1 -- 172.21.15.47:0/688856242 >> v1:172.21.15.179:6789/0 conn(0x7fadd409e4e0 legacy=0x7fadd4133de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 1 -- 172.21.15.47:0/688856242 >> v1:172.21.15.47:6789/0 conn(0x7fadd41344f0 legacy=0x7fadd4139100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient: _finish_hunting 0 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 1 monclient: found mon.noname-c 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.694+0000 7fadda09a700 1 -- 172.21.15.47:0/688856242 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd409de30 con 0x7fadd4134960 2024-04-20T02:15:52.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadda09a700 1 --2- 172.21.15.47:0/688856242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd4134960 0x7fadd413c830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00aed0 tx=0x7fadcc00c330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1db1f33d24e92864 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 1 -- 172.21.15.47:0/688856242 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fadcc00f070 con 0x7fadd4134960 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: dump: 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.718 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.719 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: _finish_auth 0 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: _check_auth_tickets 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 1 -- 172.21.15.47:0/688856242 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc00aa00 con 0x7fadd4134960 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 5 monclient: authenticate success, global_id 4378 2024-04-20T02:15:52.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 1 -- 172.21.15.47:0/688856242 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fadcc00a2e0 con 0x7fadd4134960 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 4 set_mon_vals no callback set 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7fadd8897700 10 monclient: dump: 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.722 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 10 monclient: shutdown 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 -- 172.21.15.47:0/688856242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd4134960 msgr2=0x7fadd413c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 --2- 172.21.15.47:0/688856242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd4134960 0x7fadd413c830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00aed0 tx=0x7fadcc00c330 comp rx=0 tx=0).stop 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 -- 172.21.15.47:0/688856242 shutdown_connections 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 --2- 172.21.15.47:0/688856242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd4134960 0x7fadd413c830 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 -- 172.21.15.47:0/688856242 >> 172.21.15.47:0/688856242 conn(0x7fadd40a6720 msgr2=0x7fadd40a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 -- 172.21.15.47:0/688856242 shutdown_connections 2024-04-20T02:15:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.695+0000 7faddbafd700 1 -- 172.21.15.47:0/688856242 wait complete. 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 10 monclient: build_initial_monmap 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 10 monclient: monmap: 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 1 librados: starting msgr at 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 1 librados: starting objecter 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 1 Processor -- start 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 1 -- start start 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 1 librados: setting wanted keys 2024-04-20T02:15:52.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 1 librados: calling monclient init 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.696+0000 7faddbafd700 10 monclient: init 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient: _add_conns ranks=[2,4,5,8,0,6,3,7,1] 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fadd409e4e0 0x7fadd412ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): picked mon.noname-b con 0x7fadd409e4e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 0x7fadd41304f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): picked mon.noname-c con 0x7fadd41344f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fadd4134960 0x7fadd4130a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): picked mon.noname-h con 0x7fadd4134960 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4070f70 con 0x7fadd409e4e0 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4070c90 con 0x7fadd41344f0 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40704f0 con 0x7fadd4134960 2024-04-20T02:15:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7faddbafd700 10 monclient(hunting): authenticate will time out at 1632.098412s 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 0x7fadd41304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 0x7fadd41304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44538/0 (socket says 172.21.15.47:44538) 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 -- 172.21.15.47:0/2943794727 learned_addr learned my addr 172.21.15.47:0/2943794727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fadd409e4e0 0x7fadd412ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): get_auth_request con 0x7fadd41344f0 auth_method 0 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): get_auth_request con 0x7fadd409e4e0 auth_method 0 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:52.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadda09a700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fadd4134960 0x7fadd4130a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadda09a700 10 monclient(hunting): get_auth_request con 0x7fadd4134960 auth_method 0 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadda09a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadda09a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadda09a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient(hunting): handle_auth_done global_id 4387 payload 824 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 -- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fadd4134960 msgr2=0x7fadd4130a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fadd4134960 0x7fadd4130a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 -- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fadd409e4e0 msgr2=0x7fadd412ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fadd409e4e0 0x7fadd412ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient: _finish_hunting 0 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 monclient: found mon.noname-c 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9098700 1 -- 172.21.15.47:0/2943794727 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc00ab80 con 0x7fadd41344f0 2024-04-20T02:15:52.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.697+0000 7fadd9899700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fadd409e4e0 0x7fadd412ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadd9098700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 0x7fadd41304f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fadc400c500 tx=0x7fadc400c810 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 client.?.objecter ms_handle_connect 0x7fadd41344f0 2024-04-20T02:15:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fadc40070d0 con 0x7fadd41344f0 2024-04-20T02:15:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: dump: 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc4007230 con 0x7fadd41344f0 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 5 monclient: authenticate success, global_id 4387 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 10 monclient: _renew_subs 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd40709b0 con 0x7fadd41344f0 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fadc4007810 con 0x7fadd41344f0 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadcbfff700 4 set_mon_vals no callback set 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 10 client.4387.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7fadca7fc700 10 monclient: dump: 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 10 monclient: _renew_subs 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd406ef80 con 0x7fadd41344f0 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.698+0000 7faddbafd700 1 librados: init done 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fadc40073a0 con 0x7fadd41344f0 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fadb803a2f0 0x7fadb803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fadc4049070 con 0x7fadd41344f0 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 10 client.4387.objecter ms_dispatch 0x7fadd4001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 3 client.4387.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 3 client.4387.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7faddbafd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7faddbafd700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7faddbafd700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd40711b0 con 0x7fadd41344f0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadca7fc700 20 client.4387.objecter dump_active .. 0 homeless 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadd9899700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fadb803a2f0 0x7fadb803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.699+0000 7fadd9899700 10 monclient: get_auth_request con 0x7fadb803a2f0 auth_method 0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.700+0000 7fadca7fc700 10 client.4387.objecter ms_handle_connect 0x7fadb803a2f0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.700+0000 7fadd9899700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fadb803a2f0 0x7fadb803c7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000f60 tx=0x7fadd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.704+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fadc4017070 con 0x7fadd41344f0 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.704+0000 7fadca7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.704+0000 7fadca7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:52.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.912+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadd41331e0 con 0x7fadb803a2f0 2024-04-20T02:15:52.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.915+0000 7fadca7fc700 1 -- 172.21.15.47:0/2943794727 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40597 (secure 0 0 0) 0x7fadd41331e0 con 0x7fadb803a2f0 2024-04-20T02:15:52.914 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.916+0000 7faddbafd700 10 librados: watch_flush enter 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 10 librados: watch_flush exit 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 20 client.4387.objecter shutdown clearing up homeless session... 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 10 client.4387.objecter successfully canceled tick 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fadb803a2f0 msgr2=0x7fadb803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fadb803a2f0 0x7fadb803c7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000f60 tx=0x7fadd0009040 comp rx=0 tx=0).stop 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 10 monclient: shutdown 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 msgr2=0x7fadd41304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 0x7fadd41304f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fadc400c500 tx=0x7fadc400c810 comp rx=0 tx=0).stop 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 shutdown_connections 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fadb803a2f0 0x7fadb803c7b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fadd4134960 0x7fadd4130a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fadd409e4e0 0x7fadd412ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 --2- 172.21.15.47:0/2943794727 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fadd41344f0 0x7fadd41304f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 >> 172.21.15.47:0/2943794727 conn(0x7fadd40a6720 msgr2=0x7fadd412d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 shutdown_connections 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 -- 172.21.15.47:0/2943794727 wait complete. 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:52.917+0000 7faddbafd700 1 librados: shutdown 2024-04-20T02:15:52.918 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:15:52.929 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-04-20T02:15:52.289061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164288,"kb_used_data":1856,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066752,"statfs":{"total":579820584960,"available":579652354048,"internally_reserved":0,"allocated":1900544,"data_stored":1265374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":165927591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260273+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:15:47.260273+0000","last_peered":"2024-04-20T02:15:47.260273+0000","last_clean":"2024-04-20T02:15:47.260273+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:15:47.260273+0000","last_undegraded":"2024-04-20T02:15:47.260273+0000","last_fullsized":"2024-04-20T02:15:47.260273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260775+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:15:47.260775+0000","last_peered":"2024-04-20T02:15:47.260775+0000","last_clean":"2024-04-20T02:15:47.260775+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:15:47.260775+0000","last_undegraded":"2024-04-20T02:15:47.260775+0000","last_fullsized":"2024-04-20T02:15:47.260775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260018+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:15:47.260018+0000","last_peered":"2024-04-20T02:15:47.260018+0000","last_clean":"2024-04-20T02:15:47.260018+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:15:47.260018+0000","last_undegraded":"2024-04-20T02:15:47.260018+0000","last_fullsized":"2024-04-20T02:15:47.260018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.378678+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:15:47.378678+0000","last_peered":"2024-04-20T02:15:47.378678+0000","last_clean":"2024-04-20T02:15:47.378678+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:15:47.378678+0000","last_undegraded":"2024-04-20T02:15:47.378678+0000","last_fullsized":"2024-04-20T02:15:47.378678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.857583+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:15:47.857583+0000","last_peered":"2024-04-20T02:15:47.857583+0000","last_clean":"2024-04-20T02:15:47.857583+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:15:47.857583+0000","last_undegraded":"2024-04-20T02:15:47.857583+0000","last_fullsized":"2024-04-20T02:15:47.857583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260154+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:15:47.260154+0000","last_peered":"2024-04-20T02:15:47.260154+0000","last_clean":"2024-04-20T02:15:47.260154+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:15:47.260154+0000","last_undegraded":"2024-04-20T02:15:47.260154+0000","last_fullsized":"2024-04-20T02:15:47.260154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.259833+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:15:47.259833+0000","last_peered":"2024-04-20T02:15:47.259833+0000","last_clean":"2024-04-20T02:15:47.259833+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:15:47.259833+0000","last_undegraded":"2024-04-20T02:15:47.259833+0000","last_fullsized":"2024-04-20T02:15:47.259833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260745+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:15:47.260745+0000","last_peered":"2024-04-20T02:15:47.260745+0000","last_clean":"2024-04-20T02:15:47.260745+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:15:47.260745+0000","last_undegraded":"2024-04-20T02:15:47.260745+0000","last_fullsized":"2024-04-20T02:15:47.260745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.261769+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:15:47.261769+0000","last_peered":"2024-04-20T02:15:47.261769+0000","last_clean":"2024-04-20T02:15:47.261769+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:15:47.261769+0000","last_undegraded":"2024-04-20T02:15:47.261769+0000","last_fullsized":"2024-04-20T02:15:47.261769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1003520,"data_stored":996416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]}]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":684032,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":610304,"data_stored":507936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:15:52.930 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:15:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.066+0000 7f609fef3700 10 monclient: get_monmap_and_config 2024-04-20T02:15:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.066+0000 7f609fef3700 10 monclient: build_initial_monmap 2024-04-20T02:15:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.066+0000 7f609fef3700 10 monclient: monmap: 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:53.067377+0000 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:53.067377+0000 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:53.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.066+0000 7f609fef3700 1 Processor -- start 2024-04-20T02:15:53.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 -- start start 2024-04-20T02:15:53.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient: init 2024-04-20T02:15:53.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:53.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient: _add_conns ranks=[9,7,5,6,0,2,1,4,8,10,3] 2024-04-20T02:15:53.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60980a07f0 0x7f609809ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): picked mon.noname-a con 0x7f60980a07f0 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f60980a0bd0 0x7f60980a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): picked mon.noname-i con 0x7f60980a0bd0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a1030 0x7f60980a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): picked mon.noname-h con 0x7f60980a1030 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60980a36f0 con 0x7f60980a07f0 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f609809ed20 con 0x7f60980a1030 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f609809f000 con 0x7f60980a0bd0 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609fef3700 10 monclient(hunting): authenticate will time out at 1632.468654s 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60980a07f0 0x7f609809ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f60980a07f0 0x7f609809ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56994/0 (socket says 172.21.15.47:56994) 2024-04-20T02:15:53.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 1 -- 172.21.15.47:0/3825425934 learned_addr learned my addr 172.21.15.47:0/3825425934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): get_auth_request con 0x7f60980a07f0 auth_method 0 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609e490700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a1030 0x7f60980a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609dc8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609d48e700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f60980a0bd0 0x7f60980a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609e490700 10 monclient(hunting): get_auth_request con 0x7f60980a1030 auth_method 0 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609e490700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609e490700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609e490700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609d48e700 10 monclient(hunting): get_auth_request con 0x7f60980a0bd0 auth_method 0 2024-04-20T02:15:53.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609d48e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609d48e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.067+0000 7f609d48e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609e490700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609e490700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609e490700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 10 monclient(hunting): handle_auth_done global_id 4482 payload 274 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 -- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f60980a0bd0 msgr2=0x7f60980a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f60980a0bd0 0x7f60980a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 -- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a1030 msgr2=0x7f60980a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a1030 0x7f60980a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 10 monclient: _finish_hunting 0 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 monclient: found mon.noname-a 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:53.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 -- 172.21.15.47:0/3825425934 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980751e0 con 0x7f60980a07f0 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609d48e700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f60980a0bd0 0x7f60980a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609dc8f700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60980a07f0 0x7f609809ffa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60940098f0 tx=0x7f6094009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b9184836f02d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 1 -- 172.21.15.47:0/3825425934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6094011aa0 con 0x7f60980a07f0 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: dump: 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.084 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: _finish_auth 0 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: _check_auth_tickets 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609fef3700 5 monclient: authenticate success, global_id 4482 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609fef3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 1 -- 172.21.15.47:0/3825425934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6094011c00 con 0x7f60980a07f0 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:53.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609fef3700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609fef3700 4 set_mon_vals no callback set 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 1 -- 172.21.15.47:0/3825425934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6094011d70 con 0x7f60980a07f0 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.068+0000 7f609cc8d700 10 monclient: dump: 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.086 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 10 monclient: shutdown 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 -- 172.21.15.47:0/3825425934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60980a07f0 msgr2=0x7f609809ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60980a07f0 0x7f609809ffa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f60940098f0 tx=0x7f6094009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 -- 172.21.15.47:0/3825425934 shutdown_connections 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a1030 0x7f60980a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f60980a07f0 0x7f609809ffa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 --2- 172.21.15.47:0/3825425934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f60980a0bd0 0x7f60980a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 -- 172.21.15.47:0/3825425934 >> 172.21.15.47:0/3825425934 conn(0x7f609806cbf0 msgr2=0x7f609806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 -- 172.21.15.47:0/3825425934 shutdown_connections 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 1 -- 172.21.15.47:0/3825425934 wait complete. 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 10 monclient: build_initial_monmap 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.069+0000 7f609fef3700 10 monclient: monmap: 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:53.088 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 librados: starting msgr at 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 librados: starting objecter 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 Processor -- start 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 -- start start 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 librados: setting wanted keys 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 librados: calling monclient init 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient: init 2024-04-20T02:15:53.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient: _add_conns ranks=[5,8,3,2,6,7,4,1,0] 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a07f0 0x7f60981cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): picked mon.noname-h con 0x7f60980a07f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 0x7f6098137d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): picked mon.noname-e con 0x7f60980a0bd0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60980a1030 0x7f6098138250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): picked mon.noname-g con 0x7f60980a1030 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60981251a0 con 0x7f60980a1030 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60981253f0 con 0x7f60980a07f0 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6098071fc0 con 0x7f60980a0bd0 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.070+0000 7f609fef3700 10 monclient(hunting): authenticate will time out at 1632.471950s 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 0x7f6098137d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 0x7f6098137d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:43750/0 (socket says 172.21.15.47:43750) 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 -- 172.21.15.47:0/271635097 learned_addr learned my addr 172.21.15.47:0/271635097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): get_auth_request con 0x7f60980a0bd0 auth_method 0 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609dc8f700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a07f0 0x7f60981cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609e490700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60980a1030 0x7f6098138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609dc8f700 10 monclient(hunting): get_auth_request con 0x7f60980a07f0 auth_method 0 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609dc8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609dc8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609dc8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609e490700 10 monclient(hunting): get_auth_request con 0x7f60980a1030 auth_method 0 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609e490700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609e490700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609e490700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient(hunting): handle_auth_done global_id 4337 payload 824 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 -- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a07f0 msgr2=0x7f60981cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a07f0 0x7f60981cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 -- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60980a1030 msgr2=0x7f6098138250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60980a1030 0x7f6098138250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient: _finish_hunting 0 2024-04-20T02:15:53.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 monclient: found mon.noname-e 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 -- 172.21.15.47:0/271635097 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6094009550 con 0x7f60980a0bd0 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609d48e700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 0x7f6098137d10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f608800d830 tx=0x7f608800dbf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 client.?.objecter ms_handle_connect 0x7f60980a0bd0 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6088014a90 con 0x7f60980a0bd0 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: dump: 2024-04-20T02:15:53.093 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.094 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6088014bf0 con 0x7f60980a0bd0 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 5 monclient: authenticate success, global_id 4337 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 10 monclient: _renew_subs 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6098067da0 con 0x7f60980a0bd0 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6088014d60 con 0x7f60980a0bd0 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608e7fc700 10 monclient: dump: 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.095 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f608ffff700 4 set_mon_vals no callback set 2024-04-20T02:15:53.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 10 client.4337.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 10 monclient: _renew_subs 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609809e380 con 0x7f60980a0bd0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.071+0000 7f609fef3700 1 librados: init done 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6088014620 con 0x7f60980a0bd0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f609fef3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f609fef3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f609fef3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098125530 con 0x7f60980a0bd0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f608e7fc700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f607c03a320 0x7f607c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f608804a470 con 0x7f60980a0bd0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f608e7fc700 10 client.4337.objecter ms_dispatch 0x7f6098001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f608e7fc700 3 client.4337.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.072+0000 7f608e7fc700 3 client.4337.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:53.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.073+0000 7f608e7fc700 20 client.4337.objecter dump_active .. 0 homeless 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.073+0000 7f609dc8f700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f607c03a320 0x7f607c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.073+0000 7f609dc8f700 10 monclient: get_auth_request con 0x7f607c03a320 auth_method 0 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.073+0000 7f609dc8f700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f607c03a320 0x7f607c03c7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f6094008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.073+0000 7f608e7fc700 10 client.4337.objecter ms_handle_connect 0x7f607c03a320 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.077+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f6098125530 con 0x7f60980a0bd0 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.077+0000 7f608e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.077+0000 7f608e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:53.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.292+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6098067c20 con 0x7f607c03a320 2024-04-20T02:15:53.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.294+0000 7f608e7fc700 1 -- 172.21.15.47:0/271635097 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40597 (secure 0 0 0) 0x7f6094011910 con 0x7f607c03a320 2024-04-20T02:15:53.294 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.296+0000 7f609fef3700 10 librados: watch_flush enter 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.296+0000 7f609fef3700 10 librados: watch_flush exit 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.296+0000 7f609fef3700 20 client.4337.objecter shutdown clearing up homeless session... 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.296+0000 7f609fef3700 10 client.4337.objecter successfully canceled tick 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f607c03a320 msgr2=0x7f607c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f607c03a320 0x7f607c03c7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6094000c00 tx=0x7f6094008030 comp rx=0 tx=0).stop 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 10 monclient: shutdown 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 msgr2=0x7f6098137d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 0x7f6098137d10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f608800d830 tx=0x7f608800dbf0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 shutdown_connections 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f607c03a320 0x7f607c03c7e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60980a1030 0x7f6098138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f60980a07f0 0x7f60981cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 --2- 172.21.15.47:0/271635097 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f60980a0bd0 0x7f6098137d10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 >> 172.21.15.47:0/271635097 conn(0x7f609806cbf0 msgr2=0x7f60980a57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 shutdown_connections 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 -- 172.21.15.47:0/271635097 wait complete. 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.297+0000 7f609fef3700 1 librados: shutdown 2024-04-20T02:15:53.298 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:15:53.309 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-04-20T02:15:52.289061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":164288,"kb_used_data":1856,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566066752,"statfs":{"total":579820584960,"available":579652354048,"internally_reserved":0,"allocated":1900544,"data_stored":1265374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":165927591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260273+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:15:47.260273+0000","last_peered":"2024-04-20T02:15:47.260273+0000","last_clean":"2024-04-20T02:15:47.260273+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:15:47.260273+0000","last_undegraded":"2024-04-20T02:15:47.260273+0000","last_fullsized":"2024-04-20T02:15:47.260273+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260775+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:15:47.260775+0000","last_peered":"2024-04-20T02:15:47.260775+0000","last_clean":"2024-04-20T02:15:47.260775+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:15:47.260775+0000","last_undegraded":"2024-04-20T02:15:47.260775+0000","last_fullsized":"2024-04-20T02:15:47.260775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260018+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:15:47.260018+0000","last_peered":"2024-04-20T02:15:47.260018+0000","last_clean":"2024-04-20T02:15:47.260018+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:15:47.260018+0000","last_undegraded":"2024-04-20T02:15:47.260018+0000","last_fullsized":"2024-04-20T02:15:47.260018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.378678+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:15:47.378678+0000","last_peered":"2024-04-20T02:15:47.378678+0000","last_clean":"2024-04-20T02:15:47.378678+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:15:47.378678+0000","last_undegraded":"2024-04-20T02:15:47.378678+0000","last_fullsized":"2024-04-20T02:15:47.378678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.857583+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:15:47.857583+0000","last_peered":"2024-04-20T02:15:47.857583+0000","last_clean":"2024-04-20T02:15:47.857583+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:15:47.857583+0000","last_undegraded":"2024-04-20T02:15:47.857583+0000","last_fullsized":"2024-04-20T02:15:47.857583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260154+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:15:47.260154+0000","last_peered":"2024-04-20T02:15:47.260154+0000","last_clean":"2024-04-20T02:15:47.260154+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:15:47.260154+0000","last_undegraded":"2024-04-20T02:15:47.260154+0000","last_fullsized":"2024-04-20T02:15:47.260154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.259833+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:15:47.259833+0000","last_peered":"2024-04-20T02:15:47.259833+0000","last_clean":"2024-04-20T02:15:47.259833+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:15:47.259833+0000","last_undegraded":"2024-04-20T02:15:47.259833+0000","last_fullsized":"2024-04-20T02:15:47.259833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.260745+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:15:47.260745+0000","last_peered":"2024-04-20T02:15:47.260745+0000","last_clean":"2024-04-20T02:15:47.260745+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:15:47.260745+0000","last_undegraded":"2024-04-20T02:15:47.260745+0000","last_fullsized":"2024-04-20T02:15:47.260745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-20T02:15:47.261769+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:15:47.261769+0000","last_peered":"2024-04-20T02:15:47.261769+0000","last_clean":"2024-04-20T02:15:47.261769+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:15:47.261769+0000","last_undegraded":"2024-04-20T02:15:47.261769+0000","last_fullsized":"2024-04-20T02:15:47.261769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1003520,"data_stored":996416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]}]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":684032,"data_stored":574630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":610304,"data_stored":507936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":45702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:15:53.310 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-04-20T02:15:53.311 DEBUG:tasks.mon_thrash:Yielding 2024-04-20T02:15:53.311 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T02:15:53.322 INFO:tasks.workunit:Pulling workunits from ref 15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T02:15:53.323 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T02:15:53.323 DEBUG:teuthology.orchestra.run.smithi179:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T02:15:53.323 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-04-20T02:15:53.324 INFO:tasks.mon_thrash.mon_thrasher:seed: 1713579353, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: True, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-04-20T02:15:53.324 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:15:53.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:15:53.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T02:15:53.343 INFO:teuthology.orchestra.run.smithi179.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T02:15:53.344 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-20T02:15:53.404 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-20T02:15:53.404 DEBUG:teuthology.orchestra.run.smithi179:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-20T02:15:53.468 INFO:tasks.workunit:timeout=3h 2024-04-20T02:15:53.469 INFO:tasks.workunit:cleanup=True 2024-04-20T02:15:53.469 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T02:15:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.459+0000 7fa046000700 10 monclient: get_monmap_and_config 2024-04-20T02:15:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.459+0000 7fa046000700 10 monclient: build_initial_monmap 2024-04-20T02:15:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.459+0000 7fa046000700 10 monclient: monmap: 2024-04-20T02:15:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:53.474 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:53.460961+0000 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:53.460961+0000 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:53.475 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.460+0000 7fa046000700 1 Processor -- start 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.460+0000 7fa046000700 1 -- start start 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.460+0000 7fa046000700 10 monclient: init 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient: _add_conns ranks=[5,2,6,9,0,10,7,8,4,3,1] 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071660 0x7fa040071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): picked mon.noname-h con 0x7fa040071660 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 0x7fa0400a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): picked mon.noname-b con 0x7fa040071f80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 0x7fa0400a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): picked mon.noname-d con 0x7fa040137de0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006ca10 con 0x7fa040071f80 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006e140 con 0x7fa040071660 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006e420 con 0x7fa040137de0 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa046000700 10 monclient(hunting): authenticate will time out at 1632.862248s 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 0x7fa0400a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 0x7fa0400a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 0x7fa0400a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:37428/0 (socket says 172.21.15.47:37428) 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 0x7fa0400a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38684/0 (socket says 172.21.15.47:38684) 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 -- 172.21.15.47:0/1529683861 learned_addr learned my addr 172.21.15.47:0/1529683861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): get_auth_request con 0x7fa040137de0 auth_method 0 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): get_auth_request con 0x7fa040071f80 auth_method 0 2024-04-20T02:15:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071660 0x7fa040071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): get_auth_request con 0x7fa040071660 auth_method 0 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient(hunting): handle_auth_done global_id 4308 payload 274 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 -- 172.21.15.47:0/1529683861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071660 msgr2=0x7fa040071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071660 0x7fa040071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 -- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 msgr2=0x7fa0400a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 0x7fa0400a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 monclient: found mon.noname-d 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:53.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 -- 172.21.15.47:0/1529683861 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400751e0 con 0x7fa040137de0 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03effd700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 0x7fa0400a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.461+0000 7fa03ffff700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 0x7fa0400a3dc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa038009e80 tx=0x7fa03800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=767b450153f378b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 1 -- 172.21.15.47:0/1529683861 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa03800e070 con 0x7fa040137de0 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: dump: 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 5 monclient: authenticate success, global_id 4308 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 1 -- 172.21.15.47:0/1529683861 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03800bd80 con 0x7fa040137de0 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 1 -- 172.21.15.47:0/1529683861 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa038012860 con 0x7fa040137de0 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 4 set_mon_vals no callback set 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa03e7fc700 10 monclient: dump: 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.482 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 10 monclient: shutdown 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 -- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 msgr2=0x7fa0400a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 0x7fa0400a3dc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa038009e80 tx=0x7fa03800b330 comp rx=0 tx=0).stop 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 -- 172.21.15.47:0/1529683861 shutdown_connections 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa040137de0 0x7fa0400a3dc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071660 0x7fa040071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 --2- 172.21.15.47:0/1529683861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa040071f80 0x7fa0400a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 -- 172.21.15.47:0/1529683861 >> 172.21.15.47:0/1529683861 conn(0x7fa0400a84a0 msgr2=0x7fa0400a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:53.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 -- 172.21.15.47:0/1529683861 shutdown_connections 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.462+0000 7fa046000700 1 -- 172.21.15.47:0/1529683861 wait complete. 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.463+0000 7fa046000700 10 monclient: build_initial_monmap 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.463+0000 7fa046000700 10 monclient: monmap: 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.463+0000 7fa046000700 1 librados: starting msgr at 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.463+0000 7fa046000700 1 librados: starting objecter 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 Processor -- start 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 -- start start 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 librados: setting wanted keys 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 librados: calling monclient init 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient: init 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient: _add_conns ranks=[7,5,4,0,8,3,2,6,1] 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa040071660 0x7fa0401cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): picked mon.noname-i con 0x7fa040071660 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071f80 0x7fa0401ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): picked mon.noname-h con 0x7fa040071f80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 0x7fa040120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): picked mon.noname-c con 0x7fa040137de0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa040124550 con 0x7fa040137de0 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006d3c0 con 0x7fa040071f80 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa04006d780 con 0x7fa040071660 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.464+0000 7fa046000700 10 monclient(hunting): authenticate will time out at 1632.865943s 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 0x7fa040120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 0x7fa040120b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43156/0 (socket says 172.21.15.47:43156) 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 -- 172.21.15.47:0/1915216331 learned_addr learned my addr 172.21.15.47:0/1915216331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): get_auth_request con 0x7fa040137de0 auth_method 0 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071f80 0x7fa0401ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): get_auth_request con 0x7fa040071f80 auth_method 0 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa040071660 0x7fa0401cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): get_auth_request con 0x7fa040071660 auth_method 0 2024-04-20T02:15:53.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient(hunting): handle_auth_done global_id 4396 payload 824 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 -- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa040071660 msgr2=0x7fa0401cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa040071660 0x7fa0401cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 -- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071f80 msgr2=0x7fa0401ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071f80 0x7fa0401ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:53.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 monclient: found mon.noname-c 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 -- 172.21.15.47:0/1915216331 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340094a0 con 0x7fa040137de0 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03ffff700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 0x7fa040120b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa038009e20 tx=0x7fa038004ad0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 client.?.objecter ms_handle_connect 0x7fa040137de0 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0380092e0 con 0x7fa040137de0 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: dump: 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0380046c0 con 0x7fa040137de0 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa046000700 5 monclient: authenticate success, global_id 4396 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa038004830 con 0x7fa040137de0 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa027fff700 10 monclient: dump: 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 10 monclient: _renew_subs 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.465+0000 7fa03dffb700 4 set_mon_vals no callback set 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa038009b30 con 0x7fa040137de0 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 10 client.4396.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 10 monclient: _renew_subs 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa040124a10 con 0x7fa040137de0 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa046000700 1 librados: init done 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.466+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa038004250 con 0x7fa040137de0 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa02c03a2f0 0x7fa02c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fa03804b650 con 0x7fa040137de0 2024-04-20T02:15:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 10 client.4396.objecter ms_dispatch 0x7fa040001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 3 client.4396.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 3 client.4396.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa046000700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa046000700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa046000700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040123280 con 0x7fa040137de0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 20 client.4396.objecter dump_active .. 0 homeless 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa03f7fe700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa02c03a2f0 0x7fa02c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa03f7fe700 10 monclient: get_auth_request con 0x7fa02c03a2f0 auth_method 0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa027fff700 10 client.4396.objecter ms_handle_connect 0x7fa02c03a2f0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.467+0000 7fa03f7fe700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa02c03a2f0 0x7fa02c03c7b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa030009be0 tx=0x7fa030009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.472+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa03801c440 con 0x7fa040137de0 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.472+0000 7fa027fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.472+0000 7fa027fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:53.527 INFO:tasks.workunit.client.0.smithi179.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.777+0000 7fa046000700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.777+0000 7fa046000700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.777+0000 7fa046000700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.777+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa040124690 con 0x7fa040137de0 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.778+0000 7fa027fff700 1 -- 172.21.15.47:0/1915216331 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fa040124690 con 0x7fa040137de0 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.778+0000 7fa027fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:15:53.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.778+0000 7fa027fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:53.778 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":8,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:15:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 10 librados: watch_flush enter 2024-04-20T02:15:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 10 librados: watch_flush exit 2024-04-20T02:15:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 20 client.4396.objecter shutdown clearing up homeless session... 2024-04-20T02:15:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 10 client.4396.objecter successfully canceled tick 2024-04-20T02:15:53.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa02c03a2f0 msgr2=0x7fa02c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa02c03a2f0 0x7fa02c03c7b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa030009be0 tx=0x7fa030009210 comp rx=0 tx=0).stop 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 10 monclient: shutdown 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 msgr2=0x7fa040120b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 0x7fa040120b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa038009e20 tx=0x7fa038004ad0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 shutdown_connections 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa02c03a2f0 0x7fa02c03c7b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa040137de0 0x7fa040120b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa040071660 0x7fa0401cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 --2- 172.21.15.47:0/1915216331 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa040071f80 0x7fa0401ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 >> 172.21.15.47:0/1915216331 conn(0x7fa0400a84a0 msgr2=0x7fa040134f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.780+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 shutdown_connections 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.781+0000 7fa046000700 1 -- 172.21.15.47:0/1915216331 wait complete. 2024-04-20T02:15:53.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.781+0000 7fa046000700 1 librados: shutdown 2024-04-20T02:15:53.793 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:15:53.793 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:15:53.794 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:15:53.794 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.930+0000 7f8d4fd8c700 10 monclient: get_monmap_and_config 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.930+0000 7f8d4fd8c700 10 monclient: build_initial_monmap 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.930+0000 7f8d4fd8c700 10 monclient: monmap: 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:53.931842+0000 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:53.931842+0000 2024-04-20T02:15:53.930 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.931+0000 7f8d4fd8c700 1 Processor -- start 2024-04-20T02:15:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.931+0000 7f8d4fd8c700 1 -- start start 2024-04-20T02:15:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.931+0000 7f8d4fd8c700 10 monclient: init 2024-04-20T02:15:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:53.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient: _add_conns ranks=[10,0,4,7,3,8,2,1,5,6,9] 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8d480a19d0 0x7f8d480a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): picked mon.noname-f con 0x7f8d480a19d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8d480a42d0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d48135db0 0x7f8d480708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): picked mon.noname-c con 0x7f8d48135db0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480751e0 con 0x7f8d480a42d0 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d48136810 con 0x7f8d480a19d0 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48072020 con 0x7f8d48135db0 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4fd8c700 10 monclient(hunting): authenticate will time out at 1633.333127s 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4d327700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8d480a42d0 0x7f8d480701b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39274/0 (socket says 172.21.15.47:39274) 2024-04-20T02:15:53.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4d327700 1 -- 172.21.15.47:0/2953854482 learned_addr learned my addr 172.21.15.47:0/2953854482 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 --2- 172.21.15.47:0/2953854482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d48135db0 0x7f8d480708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): get_auth_request con 0x7f8d48135db0 auth_method 0 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 1 --2- 172.21.15.47:0/2953854482 >> v2:172.21.15.179:3300/0 conn(0x7f8d480a19d0 0x7f8d480a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): get_auth_request con 0x7f8d480a19d0 auth_method 0 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4cb26700 1 -- 172.21.15.47:0/2953854482 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470068834 0 0) 0x7f8d480751e0 con 0x7f8d480a42d0 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4db28700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4cb26700 10 monclient(hunting): _negotiate 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4cb26700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4cb26700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4cb26700 10 monclient(hunting): my global_id is 4491 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4cb26700 1 -- 172.21.15.47:0/2953854482 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d340035b0 con 0x7f8d480a42d0 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient(hunting): handle_auth_done global_id 4405 payload 274 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 -- 172.21.15.47:0/2953854482 >> v2:172.21.15.179:3300/0 conn(0x7f8d480a19d0 msgr2=0x7f8d480a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 --2- 172.21.15.47:0/2953854482 >> v2:172.21.15.179:3300/0 conn(0x7f8d480a19d0 0x7f8d480a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 -- 172.21.15.47:0/2953854482 >> v1:172.21.15.47:6789/0 conn(0x7f8d480a42d0 legacy=0x7f8d480701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient: _finish_hunting 0 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 monclient: found mon.noname-c 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:53.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 -- 172.21.15.47:0/2953854482 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48072300 con 0x7f8d48135db0 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.932+0000 7f8d4e329700 1 --2- 172.21.15.47:0/2953854482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d48135db0 0x7f8d480708c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009d90 tx=0x7f8d4000b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=977d246853d066a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 1 -- 172.21.15.47:0/2953854482 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d4000e070 con 0x7f8d48135db0 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: dump: 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.946 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: _finish_auth 0 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: _check_auth_tickets 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 5 monclient: authenticate success, global_id 4405 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 1 -- 172.21.15.47:0/2953854482 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d400098f0 con 0x7f8d48135db0 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:53.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 1 -- 172.21.15.47:0/2953854482 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d400128a0 con 0x7f8d48135db0 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 4 set_mon_vals no callback set 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4cb26700 10 monclient: dump: 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 10 monclient: shutdown 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/2953854482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d48135db0 msgr2=0x7f8d480708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/2953854482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d48135db0 0x7f8d480708c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009d90 tx=0x7f8d4000b330 comp rx=0 tx=0).stop 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/2953854482 shutdown_connections 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/2953854482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d48135db0 0x7f8d480708c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/2953854482 >> v2:172.21.15.179:3300/0 conn(0x7f8d480a19d0 0x7f8d480a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/2953854482 >> 172.21.15.47:0/2953854482 conn(0x7f8d480a4a50 msgr2=0x7f8d480a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/2953854482 shutdown_connections 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.933+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/2953854482 wait complete. 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 10 monclient: build_initial_monmap 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 10 monclient: monmap: 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:53.950 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 1 librados: starting msgr at 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 1 librados: starting objecter 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 1 Processor -- start 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 1 -- start start 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 1 librados: setting wanted keys 2024-04-20T02:15:53.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 1 librados: calling monclient init 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.934+0000 7f8d4fd8c700 10 monclient: init 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient: _add_conns ranks=[2,3,5,8,7,6,1,4,0] 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 0x7f8d481cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): picked mon.noname-b con 0x7f8d480a19d0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d480a42d0 0x7f8d48134ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): picked mon.noname-g con 0x7f8d480a42d0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d48135db0 0x7f8d48135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): picked mon.noname-h con 0x7f8d48135db0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48135540 con 0x7f8d480a19d0 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4806e580 con 0x7f8d480a42d0 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480715f0 con 0x7f8d48135db0 2024-04-20T02:15:53.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4fd8c700 10 monclient(hunting): authenticate will time out at 1633.336496s 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 0x7f8d481cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 0x7f8d481cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38698/0 (socket says 172.21.15.47:38698) 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 -- 172.21.15.47:0/22287599 learned_addr learned my addr 172.21.15.47:0/22287599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): get_auth_request con 0x7f8d480a19d0 auth_method 0 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4e329700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d48135db0 0x7f8d48135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4e329700 10 monclient(hunting): get_auth_request con 0x7f8d48135db0 auth_method 0 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4e329700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4e329700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4e329700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4d327700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d480a42d0 0x7f8d48134ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient(hunting): handle_auth_done global_id 4385 payload 824 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d48135db0 msgr2=0x7f8d48135000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8d48135db0 0x7f8d48135000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d480a42d0 msgr2=0x7f8d48134ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d480a42d0 0x7f8d48134ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient: _finish_hunting 0 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 monclient: found mon.noname-b 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4db28700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40009a70 con 0x7f8d480a19d0 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4d327700 10 monclient: get_auth_request con 0x7f8d480a42d0 auth_method 0 2024-04-20T02:15:53.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.935+0000 7f8d4d327700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8d480a42d0 0x7f8d48134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4db28700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 0x7f8d481cc120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00e9b0 tx=0x7f8d3c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 client.?.objecter ms_handle_connect 0x7f8d480a19d0 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d3c009710 con 0x7f8d480a19d0 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: dump: 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c00ccc0 con 0x7f8d480a19d0 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d3c00ce30 con 0x7f8d480a19d0 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 5 monclient: authenticate success, global_id 4385 2024-04-20T02:15:53.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3bfff700 4 set_mon_vals no callback set 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d3a7fc700 10 monclient: dump: 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.957 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 10 monclient: _renew_subs 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d48071230 con 0x7f8d480a19d0 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 10 client.4385.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 10 monclient: _renew_subs 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481241c0 con 0x7f8d480a19d0 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.936+0000 7f8d4fd8c700 1 librados: init done 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8d3c00c850 con 0x7f8d480a19d0 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4fd8c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 0x7f8d4806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480756c0 con 0x7f8d4806aac0 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d2c03a2f0 0x7f8d2c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8d3c04a290 con 0x7f8d480a19d0 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 0x7f8d4806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 10 client.4385.objecter ms_dispatch 0x7f8d48001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 3 client.4385.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 3 client.4385.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient: get_auth_request con 0x7f8d4806aac0 auth_method 0 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 20 client.4385.objecter dump_active .. 0 homeless 2024-04-20T02:15:53.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4e329700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d2c03a2f0 0x7f8d2c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4e329700 10 monclient: get_auth_request con 0x7f8d2c03a2f0 auth_method 0 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 10 monclient(hunting): handle_auth_done global_id 4500 payload 824 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d4806a920 con 0x7f8d4806aac0 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 10 client.4385.objecter ms_handle_connect 0x7f8d4806aac0 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d4d327700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 0x7f8d4806c7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d440010e0 tx=0x7f8d4400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.937+0000 7f8d3a7fc700 10 client.4385.objecter resend_mon_ops 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d4400b070 con 0x7f8d4806aac0 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 10 monclient: dump: 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.960 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 10 client.4385.objecter ms_handle_connect 0x7f8d2c03a2f0 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d4e329700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d2c03a2f0 0x7f8d2c03c7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8d40014040 tx=0x7f8d40011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8d44005b70 con 0x7f8d4806aac0 2024-04-20T02:15:53.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 msgr2=0x7f8d4806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 0x7f8d4806c7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d440010e0 tx=0x7f8d4400da80 comp rx=0 tx=0).stop 2024-04-20T02:15:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 msgr2=0x7f8d4806c7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:53.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.938+0000 7f8d3a7fc700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806aac0 0x7f8d4806c7a0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8d440010e0 tx=0x7f8d4400da80 comp rx=0 tx=0).stop 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4fd8c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 0x7f8d4806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4fd8c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48068430 con 0x7f8d4806cce0 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 0x7f8d4806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient: get_auth_request con 0x7f8d4806cce0 auth_method 0 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:53.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 10 monclient(hunting): handle_auth_done global_id 4509 payload 824 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 1 -- 172.21.15.47:0/22287599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8d4806d630 con 0x7f8d4806cce0 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d4d327700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 0x7f8d4806d0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d481ccc20 tx=0x7f8d44000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d3a7fc700 10 client.4385.objecter ms_handle_connect 0x7f8d4806cce0 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d3a7fc700 10 client.4385.objecter resend_mon_ops 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d4401bd50 con 0x7f8d4806cce0 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d3a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.946+0000 7f8d3a7fc700 10 monclient: dump: 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:53.965 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9355 (secure 0 0 0) 0x7f8d4401b8e0 con 0x7f8d4806cce0 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 msgr2=0x7f8d4806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 0x7f8d4806d0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d481ccc20 tx=0x7f8d44000ed0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 msgr2=0x7f8d4806d0f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d3a7fc700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d4806cce0 0x7f8d4806d0f0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d481ccc20 tx=0x7f8d44000ed0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:53.967 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 19, 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:53.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:53.969 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:53.970 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:53.971 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:53.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.975 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:53.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.983 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:53.985 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.987 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:53.988 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 10 librados: watch_flush enter 2024-04-20T02:15:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 10 librados: watch_flush exit 2024-04-20T02:15:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 20 client.4385.objecter shutdown clearing up homeless session... 2024-04-20T02:15:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 10 client.4385.objecter successfully canceled tick 2024-04-20T02:15:53.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d2c03a2f0 msgr2=0x7f8d2c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8d2c03a2f0 0x7f8d2c03c7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8d40014040 tx=0x7f8d40011000 comp rx=0 tx=0).stop 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 10 monclient: shutdown 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 msgr2=0x7f8d481cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 0x7f8d481cc120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00e9b0 tx=0x7f8d3c00ed70 comp rx=0 tx=0).stop 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4e329700 1 -- 172.21.15.47:0/22287599 reap_dead start 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 shutdown_connections 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 --2- 172.21.15.47:0/22287599 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d480a19d0 0x7f8d481cc120 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 >> 172.21.15.47:0/22287599 conn(0x7f8d480a4a50 msgr2=0x7f8d480a6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 shutdown_connections 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 -- 172.21.15.47:0/22287599 wait complete. 2024-04-20T02:15:53.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:53.947+0000 7f8d4fd8c700 1 librados: shutdown 2024-04-20T02:15:53.992 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:53.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:15:53.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:15:54.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.132+0000 7f9e56325700 10 monclient: get_monmap_and_config 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.132+0000 7f9e56325700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.132+0000 7f9e56325700 10 monclient: monmap: 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:54.133482+0000 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:54.133482+0000 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.132 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:54.133 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 Processor -- start 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 -- start start 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient: init 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient: _add_conns ranks=[5,2,0,4,10,3,7,9,6,8,1] 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e50137d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): picked mon.noname-h con 0x7f9e50066610 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5009efc0 0x7f9e50138280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): picked mon.noname-b con 0x7f9e5009efc0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:54.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9e500a15d0 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e500751e0 con 0x7f9e500a15d0 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500669f0 con 0x7f9e5009efc0 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501387c0 con 0x7f9e50066610 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e56325700 10 monclient(hunting): authenticate will time out at 1633.534707s 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e548c2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9e500a15d0 0x7f9e500a1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39286/0 (socket says 172.21.15.47:39286) 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5009efc0 0x7f9e50138280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e548c2700 1 -- 172.21.15.47:0/2718293845 learned_addr learned my addr 172.21.15.47:0/2718293845 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e4f7fe700 10 monclient(hunting): get_auth_request con 0x7f9e5009efc0 auth_method 0 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.133+0000 7f9e4ffff700 1 --2- 172.21.15.47:0/2718293845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e50137d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): get_auth_request con 0x7f9e50066610 auth_method 0 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_done global_id 4394 payload 274 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 1 -- 172.21.15.47:0/2718293845 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511892959 0 0) 0x7f9e500751e0 con 0x7f9e500a15d0 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 1 -- 172.21.15.47:0/2718293845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 msgr2=0x7f9e50137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 1 --2- 172.21.15.47:0/2718293845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e50137d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 1 -- 172.21.15.47:0/2718293845 >> v1:172.21.15.47:6789/0 conn(0x7f9e500a15d0 legacy=0x7f9e500a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 1 monclient: found mon.noname-b 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 1 -- 172.21.15.47:0/2718293845 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5009fff0 con 0x7f9e5009efc0 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4f7fe700 1 --2- 172.21.15.47:0/2718293845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5009efc0 0x7f9e50138280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009870 tx=0x7f9e44009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441444f58d26dbe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 1 -- 172.21.15.47:0/2718293845 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e44011aa0 con 0x7f9e5009efc0 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:54.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: dump: 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.149 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 1 -- 172.21.15.47:0/2718293845 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e44011c00 con 0x7f9e5009efc0 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e56325700 5 monclient: authenticate success, global_id 4394 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e56325700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 1 -- 172.21.15.47:0/2718293845 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e44011d70 con 0x7f9e5009efc0 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e4effd700 10 monclient: dump: 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.150 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e56325700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.134+0000 7f9e56325700 4 set_mon_vals no callback set 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 10 monclient: shutdown 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 -- 172.21.15.47:0/2718293845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5009efc0 msgr2=0x7f9e50138280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 --2- 172.21.15.47:0/2718293845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5009efc0 0x7f9e50138280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009870 tx=0x7f9e44009b80 comp rx=0 tx=0).stop 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 -- 172.21.15.47:0/2718293845 shutdown_connections 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 --2- 172.21.15.47:0/2718293845 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e50137d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 --2- 172.21.15.47:0/2718293845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5009efc0 0x7f9e50138280 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 -- 172.21.15.47:0/2718293845 >> 172.21.15.47:0/2718293845 conn(0x7f9e500a6b30 msgr2=0x7f9e500a6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 -- 172.21.15.47:0/2718293845 shutdown_connections 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.135+0000 7f9e56325700 1 -- 172.21.15.47:0/2718293845 wait complete. 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 10 monclient: monmap: 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 1 librados: starting msgr at 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 1 librados: starting objecter 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 1 Processor -- start 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 1 -- start start 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 1 librados: setting wanted keys 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 1 librados: calling monclient init 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.136+0000 7f9e56325700 10 monclient: init 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient: _add_conns ranks=[5,6,4,7,8,3,2,1,0] 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e5011d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): picked mon.noname-h con 0x7f9e50066610 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9e5009efc0 0x7f9e501cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): picked mon.noname-d con 0x7f9e5009efc0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9e500a15d0 0x7f9e501314a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): picked mon.noname-c con 0x7f9e500a15d0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:54.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5011e910 con 0x7f9e500a15d0 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50135ea0 con 0x7f9e50066610 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501cca70 con 0x7f9e5009efc0 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e56325700 10 monclient(hunting): authenticate will time out at 1633.538330s 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9e5009efc0 0x7f9e501cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9e500a15d0 0x7f9e501314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9e500a15d0 0x7f9e501314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43170/0 (socket says 172.21.15.47:43170) 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9e5009efc0 0x7f9e501cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:37442/0 (socket says 172.21.15.47:37442) 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 -- 172.21.15.47:0/2456902491 learned_addr learned my addr 172.21.15.47:0/2456902491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): get_auth_request con 0x7f9e500a15d0 auth_method 0 2024-04-20T02:15:54.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): get_auth_request con 0x7f9e5009efc0 auth_method 0 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e5011d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): get_auth_request con 0x7f9e50066610 auth_method 0 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient(hunting): handle_auth_done global_id 4414 payload 824 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9e5009efc0 msgr2=0x7f9e501cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9e5009efc0 0x7f9e501cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 msgr2=0x7f9e5011d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9e50066610 0x7f9e5011d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 monclient: found mon.noname-c 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e548c2700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440094a0 con 0x7f9e500a15d0 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.137+0000 7f9e4f7fe700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9e5009efc0 0x7f9e501cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e548c2700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9e500a15d0 0x7f9e501314a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800cb60 tx=0x7f9e4800ce70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 client.?.objecter ms_handle_connect 0x7f9e500a15d0 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e4800abf0 con 0x7f9e500a15d0 2024-04-20T02:15:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: dump: 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.159 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4800ad70 con 0x7f9e500a15d0 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 5 monclient: authenticate success, global_id 4414 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 10 monclient: _renew_subs 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e50132020 con 0x7f9e500a15d0 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e48007540 con 0x7f9e500a15d0 2024-04-20T02:15:54.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4cff9700 10 monclient: dump: 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e4e7fc700 4 set_mon_vals no callback set 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 10 client.4414.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 10 monclient: _renew_subs 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5011ee60 con 0x7f9e500a15d0 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.138+0000 7f9e56325700 1 librados: init done 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9e480070d0 con 0x7f9e500a15d0 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e56325700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e56325700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 0x7f9e501c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500682a0 con 0x7f9e501319e0 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 0x7f9e501c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9e3c03a2f0 0x7f9e3c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient: get_auth_request con 0x7f9e501319e0 auth_method 0 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f9e480498b0 con 0x7f9e500a15d0 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 10 client.4414.objecter ms_dispatch 0x7f9e50001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 3 client.4414.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 3 client.4414.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 20 client.4414.objecter dump_active .. 0 homeless 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4f7fe700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9e3c03a2f0 0x7f9e3c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_done global_id 4403 payload 824 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e501365d0 con 0x7f9e501319e0 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4f7fe700 10 monclient: get_auth_request con 0x7f9e3c03a2f0 auth_method 0 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 10 client.4414.objecter ms_handle_connect 0x7f9e501319e0 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4cff9700 10 client.4414.objecter resend_mon_ops 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.139+0000 7f9e4ffff700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 0x7f9e501c9340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9e400119d0 tx=0x7f9e40011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e400036a0 con 0x7f9e501319e0 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 10 monclient: dump: 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4f7fe700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9e3c03a2f0 0x7f9e3c03c7b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e50066de0 tx=0x7f9e44008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 10 client.4414.objecter ms_handle_connect 0x7f9e3c03a2f0 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9e40004240 con 0x7f9e501319e0 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 msgr2=0x7f9e501c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 0x7f9e501c9340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9e400119d0 tx=0x7f9e40011d90 comp rx=0 tx=0).stop 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 msgr2=0x7f9e501c9340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.140+0000 7f9e4cff9700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e501319e0 0x7f9e501c9340 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9e400119d0 tx=0x7f9e40011d90 comp rx=0 tx=0).stop 2024-04-20T02:15:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e56325700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e56325700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 0x7f9e5002bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e56325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5006a400 con 0x7f9e5006aac0 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 0x7f9e5002bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient: get_auth_request con 0x7f9e5006aac0 auth_method 0 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 10 monclient(hunting): handle_auth_done global_id 4412 payload 824 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 1 -- 172.21.15.47:0/2456902491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9e5011e050 con 0x7f9e5006aac0 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4cff9700 10 client.4414.objecter ms_handle_connect 0x7f9e5006aac0 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4cff9700 10 client.4414.objecter resend_mon_ops 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4ffff700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 0x7f9e5002bbc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e400119d0 tx=0x7f9e40004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e4000fd50 con 0x7f9e5006aac0 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4cff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.148+0000 7f9e4cff9700 10 monclient: dump: 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.170 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f9e4000feb0 con 0x7f9e5006aac0 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 msgr2=0x7f9e5002bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 0x7f9e5002bbc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e400119d0 tx=0x7f9e40004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 msgr2=0x7f9e5002bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e4cff9700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9e5006aac0 0x7f9e5002bbc0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e400119d0 tx=0x7f9e40004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:54.172 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 19, 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.173 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:54.175 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.176 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:54.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.178 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.180 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.182 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.185 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:54.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.192 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.193 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.195 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:54.195 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:54.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 10 librados: watch_flush enter 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 10 librados: watch_flush exit 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 20 client.4414.objecter shutdown clearing up homeless session... 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 10 client.4414.objecter successfully canceled tick 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9e3c03a2f0 msgr2=0x7f9e3c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9e3c03a2f0 0x7f9e3c03c7b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9e50066de0 tx=0x7f9e44008030 comp rx=0 tx=0).stop 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 10 monclient: shutdown 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9e500a15d0 msgr2=0x7f9e501314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 --2- 172.21.15.47:0/2456902491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9e500a15d0 0x7f9e501314a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800cb60 tx=0x7f9e4800ce70 comp rx=0 tx=0).stop 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e548c2700 1 -- 172.21.15.47:0/2456902491 reap_dead start 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e548c2700 1 -- 172.21.15.47:0/2456902491 reap_dead start 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 shutdown_connections 2024-04-20T02:15:54.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 >> 172.21.15.47:0/2456902491 conn(0x7f9e500a6b30 msgr2=0x7f9e500a4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 shutdown_connections 2024-04-20T02:15:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 -- 172.21.15.47:0/2456902491 wait complete. 2024-04-20T02:15:54.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.149+0000 7f9e56325700 1 librados: shutdown 2024-04-20T02:15:54.198 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:54.198 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:15:54.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.341+0000 7f08e2d99700 10 monclient: get_monmap_and_config 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.341+0000 7f08e2d99700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.342+0000 7f08e2d99700 10 monclient: monmap: 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:54.343146+0000 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:54.343146+0000 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.341 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:54.342 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.342+0000 7f08e2d99700 1 Processor -- start 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.342+0000 7f08e2d99700 1 -- start start 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.342+0000 7f08e2d99700 10 monclient: init 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient: _add_conns ranks=[6,0,10,8,3,5,2,1,9,4,7] 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f08dc071710 0x7f08dc09e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): picked mon.noname-d con 0x7f08dc071710 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f08dc06f090 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f08dc1399b0 0x7f08dc13bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): picked mon.noname-f con 0x7f08dc1399b0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc0751e0 con 0x7f08dc06f090 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc09ee80 con 0x7f08dc1399b0 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0a0680 con 0x7f08dc071710 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e2d99700 10 monclient(hunting): authenticate will time out at 1633.744844s 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e0b35700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f08dc071710 0x7f08dc09e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08dbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f08dc06f090 0x7f08dc071af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39288/0 (socket says 172.21.15.47:39288) 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08dbfff700 1 -- 172.21.15.47:0/943478939 learned_addr learned my addr 172.21.15.47:0/943478939 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e0b35700 10 monclient(hunting): get_auth_request con 0x7f08dc071710 auth_method 0 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e0b35700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e0b35700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.343+0000 7f08e0b35700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e1336700 1 --2- 172.21.15.47:0/943478939 >> v2:172.21.15.179:3300/0 conn(0x7f08dc1399b0 0x7f08dc13bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 10 monclient(hunting): handle_auth_done global_id 4326 payload 274 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 1 -- 172.21.15.47:0/943478939 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182258368 0 0) 0x7f08dc0751e0 con 0x7f08dc06f090 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 1 -- 172.21.15.47:0/943478939 >> v2:172.21.15.179:3300/0 conn(0x7f08dc1399b0 msgr2=0x7f08dc13bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 1 --2- 172.21.15.47:0/943478939 >> v2:172.21.15.179:3300/0 conn(0x7f08dc1399b0 0x7f08dc13bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 1 -- 172.21.15.47:0/943478939 >> v1:172.21.15.47:6789/0 conn(0x7f08dc06f090 legacy=0x7f08dc071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 1 monclient: found mon.noname-d 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 1 -- 172.21.15.47:0/943478939 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0a0030 con 0x7f08dc071710 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:15:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e0b35700 1 --2- 172.21.15.47:0/943478939 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f08dc071710 0x7f08dc09e940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009870 tx=0x7f08cc009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=768ffa7d72ce62a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 1 -- 172.21.15.47:0/943478939 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08cc011aa0 con 0x7f08dc071710 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: dump: 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e2d99700 5 monclient: authenticate success, global_id 4326 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e2d99700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 1 -- 172.21.15.47:0/943478939 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc011c00 con 0x7f08dc071710 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 1 -- 172.21.15.47:0/943478939 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08cc011d70 con 0x7f08dc071710 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e2d99700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08e2d99700 4 set_mon_vals no callback set 2024-04-20T02:15:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.344+0000 7f08db7fe700 10 monclient: dump: 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 10 monclient: shutdown 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 -- 172.21.15.47:0/943478939 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f08dc071710 msgr2=0x7f08dc09e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 --2- 172.21.15.47:0/943478939 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f08dc071710 0x7f08dc09e940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009870 tx=0x7f08cc009b80 comp rx=0 tx=0).stop 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 -- 172.21.15.47:0/943478939 shutdown_connections 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 --2- 172.21.15.47:0/943478939 >> v2:172.21.15.179:3300/0 conn(0x7f08dc1399b0 0x7f08dc13bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 --2- 172.21.15.47:0/943478939 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f08dc071710 0x7f08dc09e940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 -- 172.21.15.47:0/943478939 >> 172.21.15.47:0/943478939 conn(0x7f08dc0a7570 msgr2=0x7f08dc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 -- 172.21.15.47:0/943478939 shutdown_connections 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.345+0000 7f08e2d99700 1 -- 172.21.15.47:0/943478939 wait complete. 2024-04-20T02:15:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 10 monclient: monmap: 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 1 librados: starting msgr at 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 1 librados: starting objecter 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 1 Processor -- start 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 1 -- start start 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 1 librados: setting wanted keys 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 1 librados: calling monclient init 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.346+0000 7f08e2d99700 10 monclient: init 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient: _add_conns ranks=[4,1,3,2,8,7,5,0,6] 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 0x7f08dc1c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): picked mon.noname-c con 0x7f08dc06f090 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08dc071710 0x7f08dc131ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): picked mon.noname-f con 0x7f08dc071710 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f08dc1399b0 0x7f08dc132000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): picked mon.noname-g con 0x7f08dc1399b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc1cac70 con 0x7f08dc071710 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc11dac0 con 0x7f08dc1399b0 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc09d7a0 con 0x7f08dc06f090 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e2d99700 10 monclient(hunting): authenticate will time out at 1633.748286s 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 0x7f08dc1c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 0x7f08dc1c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43178/0 (socket says 172.21.15.47:43178) 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 -- 172.21.15.47:0/1856400646 learned_addr learned my addr 172.21.15.47:0/1856400646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): get_auth_request con 0x7f08dc06f090 auth_method 0 2024-04-20T02:15:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08dbfff700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08dc071710 0x7f08dc131ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient(hunting): handle_auth_done global_id 4423 payload 824 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e1336700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f08dc1399b0 0x7f08dc132000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f08dc1399b0 msgr2=0x7f08dc132000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f08dc1399b0 0x7f08dc132000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08dc071710 msgr2=0x7f08dc131ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08dc071710 0x7f08dc131ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 monclient: found mon.noname-c 2024-04-20T02:15:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0094a0 con 0x7f08dc06f090 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08dbfff700 10 monclient: get_auth_request con 0x7f08dc071710 auth_method 0 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08dbfff700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08dc071710 0x7f08dc131ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e1336700 10 monclient: get_auth_request con 0x7f08dc1399b0 auth_method 0 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e1336700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f08dc1399b0 0x7f08dc132000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08e0b35700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 0x7f08dc1c9f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009bc0 tx=0x7f08cc004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08d97fa700 10 client.?.objecter ms_handle_connect 0x7f08dc06f090 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.347+0000 7f08d97fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08cc011a40 con 0x7f08dc06f090 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: dump: 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc01f070 con 0x7f08dc06f090 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 5 monclient: authenticate success, global_id 4423 2024-04-20T02:15:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08cc024480 con 0x7f08dc06f090 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 10 monclient: _renew_subs 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc09f340 con 0x7f08dc06f090 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08daffd700 4 set_mon_vals no callback set 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08d97fa700 10 monclient: dump: 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.367 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 10 client.4423.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 10 monclient: _renew_subs 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc11e9b0 con 0x7f08dc06f090 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.348+0000 7f08e2d99700 1 librados: init done 2024-04-20T02:15:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08e2d99700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08e2d99700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 0x7f08dc09b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc068310 con 0x7f08dc06aac0 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f08cc021030 con 0x7f08dc06f090 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 0x7f08dc09b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient: get_auth_request con 0x7f08dc06aac0 auth_method 0 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f08c803a320 0x7f08c803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f08cc04b2c0 con 0x7f08dc06f090 2024-04-20T02:15:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 10 client.4423.objecter ms_dispatch 0x7f08dc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 3 client.4423.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 3 client.4423.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 20 client.4423.objecter dump_active .. 0 homeless 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 10 monclient(hunting): handle_auth_done global_id 4432 payload 824 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08dc11edb0 con 0x7f08dc06aac0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08e1336700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f08c803a320 0x7f08c803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08e1336700 10 monclient: get_auth_request con 0x7f08c803a320 auth_method 0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 10 client.4423.objecter ms_handle_connect 0x7f08dc06aac0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08d97fa700 10 client.4423.objecter resend_mon_ops 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.349+0000 7f08dbfff700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 0x7f08dc09b710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08dc04ee00 tx=0x7f08d000da80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08e1336700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f08c803a320 0x7f08c803c7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08dc06f040 tx=0x7f08d4007360 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 10 client.4423.objecter ms_handle_connect 0x7f08c803a320 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08d00070d0 con 0x7f08dc06aac0 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 10 monclient: dump: 2024-04-20T02:15:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f08d0004a80 con 0x7f08dc06aac0 2024-04-20T02:15:54.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:54.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 msgr2=0x7f08dc09b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 0x7f08dc09b710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08dc04ee00 tx=0x7f08d000da80 comp rx=0 tx=0).stop 2024-04-20T02:15:54.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 msgr2=0x7f08dc09b710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.350+0000 7f08d97fa700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06aac0 0x7f08dc09b710 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08dc04ee00 tx=0x7f08d000da80 comp rx=0 tx=0).stop 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08e2d99700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08e2d99700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 0x7f08dc09c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08e2d99700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc06a400 con 0x7f08dc09bc50 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 0x7f08dc09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient: get_auth_request con 0x7f08dc09bc50 auth_method 0 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 10 monclient(hunting): handle_auth_done global_id 4441 payload 824 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 1 -- 172.21.15.47:0/1856400646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f08dc0a4e20 con 0x7f08dc09bc50 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08d97fa700 10 client.4423.objecter ms_handle_connect 0x7f08dc09bc50 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08dbfff700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 0x7f08dc09c060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08dc09ce70 tx=0x7f08d0000ed0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08d97fa700 10 client.4423.objecter resend_mon_ops 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08d0020070 con 0x7f08dc09bc50 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08d97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08d97fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.358+0000 7f08d97fa700 10 monclient: dump: 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.376 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f08d001bae0 con 0x7f08dc09bc50 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 msgr2=0x7f08dc09c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 0x7f08dc09c060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08dc09ce70 tx=0x7f08d0000ed0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 msgr2=0x7f08dc09c060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08d97fa700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc09bc50 0x7f08dc09c060 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08dc09ce70 tx=0x7f08d0000ed0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.378 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 20, 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.392 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.397 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:54.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.400 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:54.401 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 10 librados: watch_flush enter 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 10 librados: watch_flush exit 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 20 client.4423.objecter shutdown clearing up homeless session... 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 10 client.4423.objecter successfully canceled tick 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f08c803a320 msgr2=0x7f08c803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f08c803a320 0x7f08c803c7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08dc06f040 tx=0x7f08d4007360 comp rx=0 tx=0).stop 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 10 monclient: shutdown 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 msgr2=0x7f08dc1c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 0x7f08dc1c9f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009bc0 tx=0x7f08cc004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e1336700 1 -- 172.21.15.47:0/1856400646 reap_dead start 2024-04-20T02:15:54.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 shutdown_connections 2024-04-20T02:15:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 --2- 172.21.15.47:0/1856400646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08dc06f090 0x7f08dc1c9f60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 >> 172.21.15.47:0/1856400646 conn(0x7f08dc0a7570 msgr2=0x7f08dc0a60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 shutdown_connections 2024-04-20T02:15:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 -- 172.21.15.47:0/1856400646 wait complete. 2024-04-20T02:15:54.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.359+0000 7f08e2d99700 1 librados: shutdown 2024-04-20T02:15:54.404 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:54.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:15:54.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.543+0000 7fcccc5f6700 10 monclient: get_monmap_and_config 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.543+0000 7fcccc5f6700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.543+0000 7fcccc5f6700 10 monclient: monmap: 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:54.544583+0000 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:54.544583+0000 2024-04-20T02:15:54.543 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 Processor -- start 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 -- start start 2024-04-20T02:15:54.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient: init 2024-04-20T02:15:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient: _add_conns ranks=[7,2,4,1,0,3,8,5,10,6,9] 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fccc40a3960 0x7fccc4070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): picked mon.noname-i con 0x7fccc40a3960 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 0x7fccc40a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): picked mon.noname-b con 0x7fccc40a3d40 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 0x7fccc4134b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): picked mon.noname-c con 0x7fccc4134700 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406ca20 con 0x7fccc40a3d40 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406e110 con 0x7fccc4134700 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406e3f0 con 0x7fccc40a3960 2024-04-20T02:15:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccc5f6700 10 monclient(hunting): authenticate will time out at 1633.945820s 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fcccab93700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 0x7fccc4134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.544+0000 7fccc9b91700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 0x7fccc40a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 0x7fccc40a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38748/0 (socket says 172.21.15.47:38748) 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 0x7fccc4134b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43200/0 (socket says 172.21.15.47:43200) 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 -- 172.21.15.47:0/746958705 learned_addr learned my addr 172.21.15.47:0/746958705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): get_auth_request con 0x7fccc40a3d40 auth_method 0 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): get_auth_request con 0x7fccc4134700 auth_method 0 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fccc40a3960 0x7fccc4070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): get_auth_request con 0x7fccc40a3960 auth_method 0 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient(hunting): handle_auth_done global_id 4421 payload 274 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 -- 172.21.15.47:0/746958705 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fccc40a3960 msgr2=0x7fccc4070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fccc40a3960 0x7fccc4070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 -- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 msgr2=0x7fccc4134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 0x7fccc4134b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 monclient: found mon.noname-b 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 -- 172.21.15.47:0/746958705 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40751e0 con 0x7fccc40a3d40 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccab93700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 0x7fccc4134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccca392700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fccc40a3960 0x7fccc4070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9b91700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 0x7fccc40a41c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009920 tx=0x7fccb4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a80f8b9a893496 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 1 -- 172.21.15.47:0/746958705 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb4011aa0 con 0x7fccc40a3d40 2024-04-20T02:15:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: dump: 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 1 -- 172.21.15.47:0/746958705 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4011c00 con 0x7fccc40a3d40 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccc5f6700 5 monclient: authenticate success, global_id 4421 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccc5f6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccc5f6700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 1 -- 172.21.15.47:0/746958705 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb4011d70 con 0x7fccc40a3d40 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fcccc5f6700 4 set_mon_vals no callback set 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.545+0000 7fccc9390700 10 monclient: dump: 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 10 monclient: shutdown 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 -- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 msgr2=0x7fccc40a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 0x7fccc40a41c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009920 tx=0x7fccb4009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 -- 172.21.15.47:0/746958705 shutdown_connections 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc4134700 0x7fccc4134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fccc40a3960 0x7fccc4070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/746958705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fccc40a3d40 0x7fccc40a41c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 -- 172.21.15.47:0/746958705 >> 172.21.15.47:0/746958705 conn(0x7fccc40a8610 msgr2=0x7fccc40a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 -- 172.21.15.47:0/746958705 shutdown_connections 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.546+0000 7fcccc5f6700 1 -- 172.21.15.47:0/746958705 wait complete. 2024-04-20T02:15:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 10 monclient: monmap: 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 1 librados: starting msgr at 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 1 librados: starting objecter 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 1 Processor -- start 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 1 -- start start 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 1 librados: setting wanted keys 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 1 librados: calling monclient init 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.547+0000 7fcccc5f6700 10 monclient: init 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient: _add_conns ranks=[5,4,6,3,2,8,7,1,0] 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fccc40a3960 0x7fccc4133ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): picked mon.noname-h con 0x7fccc40a3960 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc40a3d40 0x7fccc4134010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): picked mon.noname-c con 0x7fccc40a3d40 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc4134700 0x7fccc411e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): picked mon.noname-d con 0x7fccc4134700 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41cd210 con 0x7fccc40a3d40 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4134550 con 0x7fccc40a3960 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4071f80 con 0x7fccc4134700 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccc5f6700 10 monclient(hunting): authenticate will time out at 1633.949233s 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc40a3d40 0x7fccc4134010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc40a3d40 0x7fccc4134010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43206/0 (socket says 172.21.15.47:43206) 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 -- 172.21.15.47:0/3058613393 learned_addr learned my addr 172.21.15.47:0/3058613393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): get_auth_request con 0x7fccc40a3d40 auth_method 0 2024-04-20T02:15:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc4134700 0x7fccc411e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fccc40a3960 0x7fccc4133ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): get_auth_request con 0x7fccc4134700 auth_method 0 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): get_auth_request con 0x7fccc40a3960 auth_method 0 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fcccab93700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient(hunting): handle_auth_done global_id 4459 payload 824 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc4134700 msgr2=0x7fccc411e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc4134700 0x7fccc411e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fccc40a3960 msgr2=0x7fccc4133ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fccc40a3960 0x7fccc4133ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 monclient: found mon.noname-c 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc00094a0 con 0x7fccc40a3d40 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccc9b91700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc40a3d40 0x7fccc4134010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccb4013000 tx=0x7fccb4004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccbaffd700 10 client.?.objecter ms_handle_connect 0x7fccc40a3d40 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.548+0000 7fccbaffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb4004390 con 0x7fccc40a3d40 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: dump: 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 5 monclient: authenticate success, global_id 4459 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 10 monclient: _renew_subs 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb4004550 con 0x7fccc40a3d40 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccbc009a70 con 0x7fccc40a3d40 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccb4024470 con 0x7fccc40a3d40 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccc8b8f700 4 set_mon_vals no callback set 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 10 client.4459.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fccbaffd700 10 monclient: dump: 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.570 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 10 monclient: _renew_subs 2024-04-20T02:15:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb4009550 con 0x7fccc40a3d40 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.549+0000 7fcccc5f6700 1 librados: init done 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fcccc5f6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 0x7fccc411eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccc4068380 con 0x7fccc406aac0 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fccb4021070 con 0x7fccc40a3d40 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 0x7fccc411eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient: get_auth_request con 0x7fccc406aac0 auth_method 0 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcca803a320 0x7fcca803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 10 monclient(hunting): handle_auth_done global_id 4344 payload 824 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fccb404b2a0 con 0x7fccc40a3d40 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccc40a5490 con 0x7fccc406aac0 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 10 client.4459.objecter ms_dispatch 0x7fccc4001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 3 client.4459.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 3 client.4459.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fcccab93700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcca803a320 0x7fcca803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fcccab93700 10 monclient: get_auth_request con 0x7fcca803a320 auth_method 0 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 20 client.4459.objecter dump_active .. 0 homeless 2024-04-20T02:15:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 10 client.4459.objecter ms_handle_connect 0x7fccc406aac0 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 10 client.4459.objecter resend_mon_ops 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccca392700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 0x7fccc411eae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011b50 tx=0x7fccc0011f10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc0014440 con 0x7fccc406aac0 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.550+0000 7fccbaffd700 10 monclient: dump: 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 10 client.4459.objecter ms_handle_connect 0x7fcca803a320 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fcccab93700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcca803a320 0x7fcca803c7e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fccc409ef50 tx=0x7fccbc00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fccc000fb10 con 0x7fccc406aac0 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 msgr2=0x7fccc411eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 0x7fccc411eae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011b50 tx=0x7fccc0011f10 comp rx=0 tx=0).stop 2024-04-20T02:15:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 msgr2=0x7fccc411eae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.551+0000 7fccbaffd700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc406aac0 0x7fccc411eae0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011b50 tx=0x7fccc0011f10 comp rx=0 tx=0).stop 2024-04-20T02:15:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fcccc5f6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 0x7fccc409f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fcccc5f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406a400 con 0x7fccc411f020 2024-04-20T02:15:54.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 0x7fccc409f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient: get_auth_request con 0x7fccc411f020 auth_method 0 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 10 monclient(hunting): handle_auth_done global_id 4353 payload 824 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 1 -- 172.21.15.47:0/3058613393 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fccc40a4700 con 0x7fccc411f020 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccca392700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 0x7fccc409f500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011f50 tx=0x7fccc0004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccbaffd700 10 client.4459.objecter ms_handle_connect 0x7fccc411f020 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccbaffd700 10 client.4459.objecter resend_mon_ops 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccc00229f0 con 0x7fccc411f020 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccbaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccbaffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.559+0000 7fccbaffd700 10 monclient: dump: 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.580 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fccc0022580 con 0x7fccc411f020 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 msgr2=0x7fccc409f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 0x7fccc409f500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011f50 tx=0x7fccc0004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 msgr2=0x7fccc409f500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fccbaffd700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fccc411f020 0x7fccc409f500 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccc0011f50 tx=0x7fccc0004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 20, 2024-04-20T02:15:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.598 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:54.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.601 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.602 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:54.603 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 10 librados: watch_flush enter 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 10 librados: watch_flush exit 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 20 client.4459.objecter shutdown clearing up homeless session... 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 10 client.4459.objecter successfully canceled tick 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcca803a320 msgr2=0x7fcca803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcca803a320 0x7fcca803c7e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fccc409ef50 tx=0x7fccbc00b040 comp rx=0 tx=0).stop 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 10 monclient: shutdown 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc40a3d40 msgr2=0x7fccc4134010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 --2- 172.21.15.47:0/3058613393 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fccc40a3d40 0x7fccc4134010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccb4013000 tx=0x7fccb4004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccab93700 1 -- 172.21.15.47:0/3058613393 reap_dead start 2024-04-20T02:15:54.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccab93700 1 -- 172.21.15.47:0/3058613393 reap_dead start 2024-04-20T02:15:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 shutdown_connections 2024-04-20T02:15:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 >> 172.21.15.47:0/3058613393 conn(0x7fccc40a8610 msgr2=0x7fccc4070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 shutdown_connections 2024-04-20T02:15:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 -- 172.21.15.47:0/3058613393 wait complete. 2024-04-20T02:15:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.560+0000 7fcccc5f6700 1 librados: shutdown 2024-04-20T02:15:54.607 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:54.607 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:15:54.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.746+0000 7fe8230bf700 10 monclient: get_monmap_and_config 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.746+0000 7fe8230bf700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.746+0000 7fe8230bf700 10 monclient: monmap: 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:54.747869+0000 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:54.747869+0000 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:54.747 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.747+0000 7fe8230bf700 1 Processor -- start 2024-04-20T02:15:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.747+0000 7fe8230bf700 1 -- start start 2024-04-20T02:15:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.747+0000 7fe8230bf700 10 monclient: init 2024-04-20T02:15:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient: _add_conns ranks=[4,10,9,7,3,6,8,2,1,5,0] 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 0x7fe81c135e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): picked mon.noname-c con 0x7fe81c0a18a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe81c0a1c80 0x7fe81c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): picked mon.noname-f con 0x7fe81c0a1c80 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 0x7fe81c0a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): picked mon.noname-a con 0x7fe81c0a2100 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c135bd0 con 0x7fe81c0a2100 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c071eb0 con 0x7fe81c0a1c80 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c072190 con 0x7fe81c0a18a0 2024-04-20T02:15:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe8230bf700 10 monclient(hunting): authenticate will time out at 1634.149565s 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 0x7fe81c135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 0x7fe81c0a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 0x7fe81c135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43216/0 (socket says 172.21.15.47:43216) 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 0x7fe81c0a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43384/0 (socket says 172.21.15.47:43384) 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 1 -- 172.21.15.47:0/1415468030 learned_addr learned my addr 172.21.15.47:0/1415468030 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): get_auth_request con 0x7fe81c0a18a0 auth_method 0 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): get_auth_request con 0x7fe81c0a2100 auth_method 0 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 1 --2- 172.21.15.47:0/1415468030 >> v2:172.21.15.179:3300/0 conn(0x7fe81c0a1c80 0x7fe81c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): get_auth_request con 0x7fe81c0a1c80 auth_method 0 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe82165c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.748+0000 7fe81bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 10 monclient(hunting): handle_auth_done global_id 4468 payload 274 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 -- 172.21.15.47:0/1415468030 >> v2:172.21.15.179:3300/0 conn(0x7fe81c0a1c80 msgr2=0x7fe81c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 --2- 172.21.15.47:0/1415468030 >> v2:172.21.15.179:3300/0 conn(0x7fe81c0a1c80 0x7fe81c0a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 -- 172.21.15.47:0/1415468030 >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 msgr2=0x7fe81c0a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 --2- 172.21.15.47:0/1415468030 >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 0x7fe81c0a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 monclient: found mon.noname-c 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 -- 172.21.15.47:0/1415468030 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0751e0 con 0x7fe81c0a18a0 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe82165c700 1 --2- 172.21.15.47:0/1415468030 >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 0x7fe81c0a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe820e5b700 1 --2- 172.21.15.47:0/1415468030 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 0x7fe81c135e20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe81c09ee20 tx=0x7fe80c009bd0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=5cb6d559dc081fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 1 -- 172.21.15.47:0/1415468030 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe80c011aa0 con 0x7fe81c0a18a0 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: dump: 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.763 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 1 -- 172.21.15.47:0/1415468030 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c011c00 con 0x7fe81c0a18a0 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 5 monclient: authenticate success, global_id 4468 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 1 -- 172.21.15.47:0/1415468030 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe80c011d70 con 0x7fe81c0a18a0 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 4 set_mon_vals no callback set 2024-04-20T02:15:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe81b7fe700 10 monclient: dump: 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 10 monclient: shutdown 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 1 -- 172.21.15.47:0/1415468030 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 msgr2=0x7fe81c135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.749+0000 7fe8230bf700 1 --2- 172.21.15.47:0/1415468030 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 0x7fe81c135e20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe81c09ee20 tx=0x7fe80c009bd0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 -- 172.21.15.47:0/1415468030 shutdown_connections 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 --2- 172.21.15.47:0/1415468030 >> v2:172.21.15.47:3300/0 conn(0x7fe81c0a2100 0x7fe81c0a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 --2- 172.21.15.47:0/1415468030 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe81c0a18a0 0x7fe81c135e20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 --2- 172.21.15.47:0/1415468030 >> v2:172.21.15.179:3300/0 conn(0x7fe81c0a1c80 0x7fe81c0a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 -- 172.21.15.47:0/1415468030 >> 172.21.15.47:0/1415468030 conn(0x7fe81c0a50b0 msgr2=0x7fe81c066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 -- 172.21.15.47:0/1415468030 shutdown_connections 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 1 -- 172.21.15.47:0/1415468030 wait complete. 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.750+0000 7fe8230bf700 10 monclient: monmap: 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 1 librados: starting msgr at 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 1 librados: starting objecter 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 1 Processor -- start 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 1 -- start start 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 1 librados: setting wanted keys 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 1 librados: calling monclient init 2024-04-20T02:15:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.751+0000 7fe8230bf700 10 monclient: init 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient: _add_conns ranks=[5,8,2,6,4,7,0,3,1] 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe81c0a18a0 0x7fe81c09de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): picked mon.noname-h con 0x7fe81c0a18a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c0a1c80 0x7fe81c11e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): picked mon.noname-e con 0x7fe81c0a1c80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 0x7fe81c11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): picked mon.noname-b con 0x7fe81c0a2100 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c11ffd0 con 0x7fe81c0a2100 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c071450 con 0x7fe81c0a18a0 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c09ee50 con 0x7fe81c0a1c80 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe8230bf700 10 monclient(hunting): authenticate will time out at 1634.153534s 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 0x7fe81c11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c0a1c80 0x7fe81c11e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 0x7fe81c11ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38762/0 (socket says 172.21.15.47:38762) 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 1 -- 172.21.15.47:0/2237784846 learned_addr learned my addr 172.21.15.47:0/2237784846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): get_auth_request con 0x7fe81c0a2100 auth_method 0 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): get_auth_request con 0x7fe81c0a1c80 auth_method 0 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe82165c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe81bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe81c0a18a0 0x7fe81c09de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): get_auth_request con 0x7fe81c0a18a0 auth_method 0 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.752+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 10 monclient(hunting): handle_auth_done global_id 4430 payload 824 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c0a1c80 msgr2=0x7fe81c11e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c0a1c80 0x7fe81c11e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe81c0a18a0 msgr2=0x7fe81c09de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe81c0a18a0 0x7fe81c09de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 monclient: found mon.noname-b 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81400f050 con 0x7fe81c0a2100 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe82165c700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 0x7fe81c11ea00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe81400f490 tx=0x7fe81400f7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 client.?.objecter ms_handle_connect 0x7fe81c0a2100 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe814012070 con 0x7fe81c0a2100 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: dump: 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814013040 con 0x7fe81c0a2100 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 5 monclient: authenticate success, global_id 4430 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 10 monclient: _renew_subs 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c009550 con 0x7fe81c0a2100 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe81400fe90 con 0x7fe81c0a2100 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8197fa700 10 monclient: dump: 2024-04-20T02:15:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe81affd700 4 set_mon_vals no callback set 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 10 client.4430.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 10 monclient: _renew_subs 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1217d0 con 0x7fe81c0a2100 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.753+0000 7fe8230bf700 1 librados: init done 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe814018460 con 0x7fe81c0a2100 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8230bf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8230bf700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 0x7fe81c11ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c121960 con 0x7fe81c06aac0 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe7fc03a2f0 0x7fe7fc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fe81404e870 con 0x7fe81c0a2100 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 10 client.4430.objecter ms_dispatch 0x7fe81c001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 3 client.4430.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 3 client.4430.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 0x7fe81c11ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient: get_auth_request con 0x7fe81c06aac0 auth_method 0 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe8197fa700 20 client.4430.objecter dump_active .. 0 homeless 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe81bfff700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe7fc03a2f0 0x7fe7fc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe81bfff700 10 monclient: get_auth_request con 0x7fe7fc03a2f0 auth_method 0 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 10 monclient(hunting): handle_auth_done global_id 4355 payload 824 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.754+0000 7fe820e5b700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe81c070bb0 con 0x7fe81c06aac0 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 client.4430.objecter ms_handle_connect 0x7fe81c06aac0 2024-04-20T02:15:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 client.4430.objecter resend_mon_ops 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe820e5b700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 0x7fe81c11ef40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009fd0 tx=0x7fe80c004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 client.4430.objecter ms_handle_connect 0x7fe7fc03a2f0 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe81bfff700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe7fc03a2f0 0x7fe7fc03c7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe81c04fd10 tx=0x7fe804009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe80c020070 con 0x7fe81c06aac0 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 monclient: dump: 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe80c004440 con 0x7fe81c06aac0 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 msgr2=0x7fe81c11ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 0x7fe81c11ef40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009fd0 tx=0x7fe80c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 msgr2=0x7fe81c11ef40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.755+0000 7fe8197fa700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c06aac0 0x7fe81c11ef40 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009fd0 tx=0x7fe80c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe8230bf700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:54.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe8230bf700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 0x7fe81c1c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe8230bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c068430 con 0x7fe81c1c86e0 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 0x7fe81c1c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient: get_auth_request con 0x7fe81c1c86e0 auth_method 0 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 10 monclient(hunting): handle_auth_done global_id 4364 payload 824 2024-04-20T02:15:54.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.763+0000 7fe820e5b700 1 -- 172.21.15.47:0/2237784846 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe81c067150 con 0x7fe81c1c86e0 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 client.4430.objecter ms_handle_connect 0x7fe81c1c86e0 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe820e5b700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 0x7fe81c1c8af0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009130 tx=0x7fe80c011000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 client.4430.objecter resend_mon_ops 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe80c003ee0 con 0x7fe81c1c86e0 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 monclient: dump: 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fe80c02a830 con 0x7fe81c1c86e0 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 msgr2=0x7fe81c1c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 0x7fe81c1c8af0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009130 tx=0x7fe80c011000 comp rx=0 tx=0).stop 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 msgr2=0x7fe81c1c8af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8197fa700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe81c1c86e0 0x7fe81c1c8af0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 20, 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:54.802 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:54.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.805 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:54.806 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 10 librados: watch_flush enter 2024-04-20T02:15:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 10 librados: watch_flush exit 2024-04-20T02:15:54.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 20 client.4430.objecter shutdown clearing up homeless session... 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 10 client.4430.objecter successfully canceled tick 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe7fc03a2f0 msgr2=0x7fe7fc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe7fc03a2f0 0x7fe7fc03c7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe81c04fd10 tx=0x7fe804009210 comp rx=0 tx=0).stop 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 10 monclient: shutdown 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 msgr2=0x7fe81c11ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe8230bf700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 0x7fe81c11ea00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe81400f490 tx=0x7fe81400f7a0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.764+0000 7fe82165c700 1 -- 172.21.15.47:0/2237784846 reap_dead start 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.765+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 shutdown_connections 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.765+0000 7fe8230bf700 1 --2- 172.21.15.47:0/2237784846 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe81c0a2100 0x7fe81c11ea00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.765+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 >> 172.21.15.47:0/2237784846 conn(0x7fe81c0a50b0 msgr2=0x7fe81c134d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.765+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 shutdown_connections 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.765+0000 7fe8230bf700 1 -- 172.21.15.47:0/2237784846 wait complete. 2024-04-20T02:15:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.765+0000 7fe8230bf700 1 librados: shutdown 2024-04-20T02:15:54.810 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:54.810 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:15:54.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:15:54.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.949+0000 7f0bd95b7700 10 monclient: get_monmap_and_config 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.949+0000 7f0bd95b7700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.949+0000 7f0bd95b7700 10 monclient: monmap: 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:54.950640+0000 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:54.950640+0000 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:54.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 Processor -- start 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 -- start start 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient: init 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient: _add_conns ranks=[7,0,9,6,1,3,2,4,10,8,5] 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0bd4137d90 0x7f0bd40a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): picked mon.noname-i con 0x7f0bd4137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bd4138170 addr v1:172.21.15.47:6789/0 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0bd41385b0 0x7f0bd40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): picked mon.noname-a con 0x7f0bd41385b0 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40751e0 con 0x7f0bd4138170 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd409fa40 con 0x7f0bd41385b0 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd409dfc0 con 0x7f0bd4137d90 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd95b7700 10 monclient(hunting): authenticate will time out at 1634.351857s 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd27fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0bd4138170 0x7f0bd40a5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39292/0 (socket says 172.21.15.47:39292) 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.950+0000 7f0bd27fc700 1 -- 172.21.15.47:0/3825805848 learned_addr learned my addr 172.21.15.47:0/3825805848 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 --2- 172.21.15.47:0/3825805848 >> v2:172.21.15.47:3300/0 conn(0x7f0bd41385b0 0x7f0bd40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): get_auth_request con 0x7f0bd41385b0 auth_method 0 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0bd4137d90 0x7f0bd40a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): get_auth_request con 0x7f0bd4137d90 auth_method 0 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 1 -- 172.21.15.47:0/3825805848 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273287578 0 0) 0x7f0bd40751e0 con 0x7f0bd4138170 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient(hunting): my global_id is 4554 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 1 -- 172.21.15.47:0/3825805848 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bbc0035b0 con 0x7f0bd4138170 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_done global_id 4545 payload 274 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 -- 172.21.15.47:0/3825805848 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0bd4137d90 msgr2=0x7f0bd40a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0bd4137d90 0x7f0bd40a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 -- 172.21.15.47:0/3825805848 >> v1:172.21.15.47:6789/0 conn(0x7f0bd4138170 legacy=0x7f0bd40a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 monclient: found mon.noname-a 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 -- 172.21.15.47:0/3825805848 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40a7bc0 con 0x7f0bd41385b0 2024-04-20T02:15:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0bd4137d90 0x7f0bd40a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd37fe700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bd41385b0 0x7f0bd40a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009dc0 tx=0x7f0bcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c29df989187525d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 1 -- 172.21.15.47:0/3825805848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bcc00e070 con 0x7f0bd41385b0 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: dump: 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 1 -- 172.21.15.47:0/3825805848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc012430 con 0x7f0bd41385b0 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd95b7700 5 monclient: authenticate success, global_id 4545 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd95b7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 1 -- 172.21.15.47:0/3825805848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bcc012a10 con 0x7f0bd41385b0 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd95b7700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd95b7700 4 set_mon_vals no callback set 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.951+0000 7f0bd1ffb700 10 monclient: dump: 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 10 monclient: shutdown 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 -- 172.21.15.47:0/3825805848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bd41385b0 msgr2=0x7f0bd40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bd41385b0 0x7f0bd40a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009dc0 tx=0x7f0bcc00b330 comp rx=0 tx=0).stop 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 -- 172.21.15.47:0/3825805848 shutdown_connections 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bd41385b0 0x7f0bd40a63a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/3825805848 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0bd4137d90 0x7f0bd40a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 -- 172.21.15.47:0/3825805848 >> 172.21.15.47:0/3825805848 conn(0x7f0bd40aa8b0 msgr2=0x7f0bd40a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 -- 172.21.15.47:0/3825805848 shutdown_connections 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.952+0000 7f0bd95b7700 1 -- 172.21.15.47:0/3825805848 wait complete. 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 10 monclient: build_initial_monmap 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 10 monclient: monmap: 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 1 librados: starting msgr at 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 1 librados: starting objecter 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 1 Processor -- start 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 1 -- start start 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 1 librados: setting wanted keys 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 1 librados: calling monclient init 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.953+0000 7f0bd95b7700 10 monclient: init 2024-04-20T02:15:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient: _add_conns ranks=[2,6,8,0,4,1,5,3,7] 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0bd4137d90 0x7f0bd41cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): picked mon.noname-b con 0x7f0bd4137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bd4138170 0x7f0bd4136f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): picked mon.noname-d con 0x7f0bd4138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0bd41385b0 0x7f0bd4137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): picked mon.noname-e con 0x7f0bd41385b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4124440 con 0x7f0bd4137d90 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41cce60 con 0x7f0bd4138170 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4124690 con 0x7f0bd41385b0 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd95b7700 10 monclient(hunting): authenticate will time out at 1634.355227s 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0bd4137d90 0x7f0bd41cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bd4138170 0x7f0bd4136f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0bd41385b0 0x7f0bd4137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0bd41385b0 0x7f0bd4137440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37874/0 (socket says 172.21.15.47:37874) 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0bd4137d90 0x7f0bd41cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38774/0 (socket says 172.21.15.47:38774) 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): get_auth_request con 0x7f0bd4138170 auth_method 0 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 1 -- 172.21.15.47:0/2982510641 learned_addr learned my addr 172.21.15.47:0/2982510641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): get_auth_request con 0x7f0bd4137d90 auth_method 0 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): get_auth_request con 0x7f0bd41385b0 auth_method 0 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient(hunting): handle_auth_done global_id 4362 payload 824 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0bd41385b0 msgr2=0x7f0bd4137440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0bd41385b0 0x7f0bd4137440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0bd4137d90 msgr2=0x7f0bd41cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0bd4137d90 0x7f0bd41cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient: _finish_hunting 0 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 monclient: found mon.noname-d 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40094a0 con 0x7f0bd4138170 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0bd4137d90 0x7f0bd41cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bd27fc700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bd4138170 0x7f0bd4136f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c420 tx=0x7f0bc800c730 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bbb7fe700 10 client.?.objecter ms_handle_connect 0x7f0bd4138170 2024-04-20T02:15:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.954+0000 7f0bbb7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bc8005db0 con 0x7f0bd4138170 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: dump: 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 5 monclient: authenticate success, global_id 4362 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc80070d0 con 0x7f0bd4138170 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bc80076b0 con 0x7f0bd4138170 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 10 monclient: _renew_subs 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc009a70 con 0x7f0bd4138170 2024-04-20T02:15:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd17fa700 4 set_mon_vals no callback set 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 10 client.4362.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bbb7fe700 10 monclient: dump: 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 10 monclient: _renew_subs 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4122f00 con 0x7f0bd4138170 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.955+0000 7f0bd95b7700 1 librados: init done 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0bc8007240 con 0x7f0bd4138170 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd95b7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 0x7f0bd41208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4068180 con 0x7f0bd406aac0 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bc003a2f0 0x7f0bc003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f0bc8048e70 con 0x7f0bd4138170 2024-04-20T02:15:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 10 client.4362.objecter ms_dispatch 0x7f0bd4001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 3 client.4362.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 3 client.4362.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bbb7fe700 20 client.4362.objecter dump_active .. 0 homeless 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 0x7f0bd41208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd37fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bc003a2f0 0x7f0bc003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd37fe700 10 monclient: get_auth_request con 0x7f0bc003a2f0 auth_method 0 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient: get_auth_request con 0x7f0bd406aac0 auth_method 0 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.956+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_done global_id 4537 payload 824 2024-04-20T02:15:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bd2ffd700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bd409f760 con 0x7f0bd406aac0 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bd37fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bc003a2f0 0x7f0bc003c7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d90 tx=0x7f0bcc011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 client.4362.objecter ms_handle_connect 0x7f0bc003a2f0 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 client.4362.objecter ms_handle_connect 0x7f0bd406aac0 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 client.4362.objecter resend_mon_ops 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 0x7f0bd41208d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4011f00 tx=0x7f0bc400f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bc40044a0 con 0x7f0bd406aac0 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 monclient: dump: 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0bc4004030 con 0x7f0bd406aac0 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 msgr2=0x7f0bd41208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 0x7f0bd41208d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4011f00 tx=0x7f0bc400f520 comp rx=0 tx=0).stop 2024-04-20T02:15:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 msgr2=0x7f0bd41208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.957+0000 7f0bbb7fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd406aac0 0x7f0bd41208d0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4011f00 tx=0x7f0bc400f520 comp rx=0 tx=0).stop 2024-04-20T02:15:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.965+0000 7f0bd95b7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.965+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 0x7f0bd4121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.965+0000 7f0bd95b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.965+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd406a400 con 0x7f0bd4120e10 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.965+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 0x7f0bd4121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.965+0000 7f0bd2ffd700 10 monclient: get_auth_request con 0x7f0bd4120e10 auth_method 0 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 10 monclient(hunting): handle_auth_done global_id 4546 payload 824 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 1 -- 172.21.15.47:0/2982510641 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0bd4121760 con 0x7f0bd4120e10 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bd2ffd700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 0x7f0bd4121220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0bc401b000 tx=0x7f0bc4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bbb7fe700 10 client.4362.objecter ms_handle_connect 0x7f0bd4120e10 2024-04-20T02:15:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bbb7fe700 10 client.4362.objecter resend_mon_ops 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bc401ead0 con 0x7f0bd4120e10 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bbb7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.966+0000 7f0bbb7fe700 10 monclient: dump: 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f0bc401e660 con 0x7f0bd4120e10 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 msgr2=0x7f0bd4121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 0x7f0bd4121220 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0bc401b000 tx=0x7f0bc4004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 msgr2=0x7f0bd4121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bbb7fe700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bd4120e10 0x7f0bd4121220 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0bc401b000 tx=0x7f0bc4004750 comp rx=0 tx=0).stop 2024-04-20T02:15:54.987 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 20, 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:55.009 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 10 librados: watch_flush enter 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 10 librados: watch_flush exit 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 20 client.4362.objecter shutdown clearing up homeless session... 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 10 client.4362.objecter successfully canceled tick 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bc003a2f0 msgr2=0x7f0bc003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bc003a2f0 0x7f0bc003c7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d90 tx=0x7f0bcc011000 comp rx=0 tx=0).stop 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 10 monclient: shutdown 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bd4138170 msgr2=0x7f0bd4136f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 --2- 172.21.15.47:0/2982510641 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bd4138170 0x7f0bd4136f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800c420 tx=0x7f0bc800c730 comp rx=0 tx=0).stop 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd37fe700 1 -- 172.21.15.47:0/2982510641 reap_dead start 2024-04-20T02:15:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd37fe700 1 -- 172.21.15.47:0/2982510641 reap_dead start 2024-04-20T02:15:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 shutdown_connections 2024-04-20T02:15:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 >> 172.21.15.47:0/2982510641 conn(0x7f0bd40aa8b0 msgr2=0x7f0bd409cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 shutdown_connections 2024-04-20T02:15:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 -- 172.21.15.47:0/2982510641 wait complete. 2024-04-20T02:15:55.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:54.967+0000 7f0bd95b7700 1 librados: shutdown 2024-04-20T02:15:55.012 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:55.012 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:15:55.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:15:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.151+0000 7fb5ce301700 10 monclient: get_monmap_and_config 2024-04-20T02:15:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.151+0000 7fb5ce301700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.151+0000 7fb5ce301700 10 monclient: monmap: 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:55.152567+0000 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:55.152567+0000 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:55.152 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 Processor -- start 2024-04-20T02:15:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 -- start start 2024-04-20T02:15:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient: init 2024-04-20T02:15:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient: _add_conns ranks=[1,6,9,2,5,8,10,7,0,4,3] 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb5c80a19d0 addr v1:172.21.15.179:6789/0 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a1db0 0x7fb5c813ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): picked mon.noname-d con 0x7fb5c80a1db0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 0x7fb5c813d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): picked mon.noname-a con 0x7fb5c813b150 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80751e0 con 0x7fb5c80a19d0 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c8136610 con 0x7fb5c813b150 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80a2230 con 0x7fb5c80a1db0 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5ce301700 10 monclient(hunting): authenticate will time out at 1634.553808s 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5cc89e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 0x7fb5c813d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a1db0 0x7fb5c813ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.152+0000 7fb5cc89e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 0x7fb5c813d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43400/0 (socket says 172.21.15.47:43400) 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 1 -- 172.21.15.47:0/2296702378 learned_addr learned my addr 172.21.15.47:0/2296702378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): get_auth_request con 0x7fb5c80a1db0 auth_method 0 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): get_auth_request con 0x7fb5c813b150 auth_method 0 2024-04-20T02:15:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_done global_id 4371 payload 274 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 1 -- 172.21.15.47:0/2296702378 >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 msgr2=0x7fb5c813d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/2296702378 >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 0x7fb5c813d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 1 -- 172.21.15.47:0/2296702378 >> v1:172.21.15.179:6789/0 conn(0x7fb5c80a19d0 legacy=0x7fb5c8135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 1 monclient: found mon.noname-d 2024-04-20T02:15:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 1 -- 172.21.15.47:0/2296702378 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80a4040 con 0x7fb5c80a1db0 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5cc89e700 1 --2- 172.21.15.47:0/2296702378 >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 0x7fb5c813d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/2296702378 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a1db0 0x7fb5c813ac10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009840 tx=0x7fb5bc009b50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=663b1109c6cc7a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 1 -- 172.21.15.47:0/2296702378 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5bc011aa0 con 0x7fb5c80a1db0 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: dump: 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 1 -- 172.21.15.47:0/2296702378 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc011c00 con 0x7fb5c80a1db0 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5ce301700 5 monclient: authenticate success, global_id 4371 2024-04-20T02:15:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5ce301700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5ce301700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 1 -- 172.21.15.47:0/2296702378 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5bc011d70 con 0x7fb5c80a1db0 2024-04-20T02:15:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5ce301700 4 set_mon_vals no callback set 2024-04-20T02:15:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.153+0000 7fb5c6ffd700 10 monclient: dump: 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.170 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 10 monclient: shutdown 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 -- 172.21.15.47:0/2296702378 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a1db0 msgr2=0x7fb5c813ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 --2- 172.21.15.47:0/2296702378 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a1db0 0x7fb5c813ac10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009840 tx=0x7fb5bc009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 -- 172.21.15.47:0/2296702378 shutdown_connections 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 --2- 172.21.15.47:0/2296702378 >> v2:172.21.15.47:3300/0 conn(0x7fb5c813b150 0x7fb5c813d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 --2- 172.21.15.47:0/2296702378 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a1db0 0x7fb5c813ac10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 -- 172.21.15.47:0/2296702378 >> 172.21.15.47:0/2296702378 conn(0x7fb5c80a4a50 msgr2=0x7fb5c80a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 -- 172.21.15.47:0/2296702378 shutdown_connections 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.154+0000 7fb5ce301700 1 -- 172.21.15.47:0/2296702378 wait complete. 2024-04-20T02:15:55.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 10 monclient: monmap: 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.172 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 1 librados: starting msgr at 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 1 librados: starting objecter 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 1 Processor -- start 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 1 -- start start 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 1 librados: setting wanted keys 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 1 librados: calling monclient init 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.155+0000 7fb5ce301700 10 monclient: init 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient: _add_conns ranks=[6,2,8,0,7,1,5,4,3] 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a19d0 0x7fb5c81ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): picked mon.noname-d con 0x7fb5c80a19d0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5c80a1db0 0x7fb5c8133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): picked mon.noname-b con 0x7fb5c80a1db0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:55.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb5c813b150 0x7fb5c8133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): picked mon.noname-e con 0x7fb5c813b150 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c813a670 con 0x7fb5c80a1db0 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8071230 con 0x7fb5c80a19d0 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c811fa10 con 0x7fb5c813b150 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5ce301700 10 monclient(hunting): authenticate will time out at 1634.557325s 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5c80a1db0 0x7fb5c8133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a19d0 0x7fb5c81ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb5c813b150 0x7fb5c8133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5c80a1db0 0x7fb5c8133900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38780/0 (socket says 172.21.15.47:38780) 2024-04-20T02:15:55.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 1 -- 172.21.15.47:0/6722596 learned_addr learned my addr 172.21.15.47:0/6722596 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): get_auth_request con 0x7fb5c80a1db0 auth_method 0 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): get_auth_request con 0x7fb5c80a19d0 auth_method 0 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): get_auth_request con 0x7fb5c813b150 auth_method 0 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5cc89e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient(hunting): handle_auth_done global_id 4380 payload 824 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb5c813b150 msgr2=0x7fb5c8133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb5c813b150 0x7fb5c8133e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5c80a1db0 msgr2=0x7fb5c8133900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5c80a1db0 0x7fb5c8133900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 monclient: found mon.noname-d 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009b30 con 0x7fb5c80a19d0 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb5c80a1db0 0x7fb5c8133900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c7fff700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a19d0 0x7fb5c81ca6d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c440 tx=0x7fb5b8001e70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.156+0000 7fb5c4ff9700 10 client.?.objecter ms_handle_connect 0x7fb5c80a19d0 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5b8001120 con 0x7fb5c80a19d0 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: dump: 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 5 monclient: authenticate success, global_id 4380 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8005540 con 0x7fb5c80a19d0 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5b8003b10 con 0x7fb5c80a19d0 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 10 monclient: _renew_subs 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc0094a0 con 0x7fb5c80a19d0 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c67fc700 4 set_mon_vals no callback set 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5c4ff9700 10 monclient: dump: 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 10 client.4380.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 10 monclient: _renew_subs 2024-04-20T02:15:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81389d0 con 0x7fb5c80a19d0 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.157+0000 7fb5ce301700 1 librados: init done 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5ce301700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5ce301700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 0x7fb5c809c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8068430 con 0x7fb5c806aac0 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb5b8003680 con 0x7fb5c80a19d0 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 0x7fb5c809c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient: get_auth_request con 0x7fb5c806aac0 auth_method 0 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb5b403a350 0x7fb5b403c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fb5b8049300 con 0x7fb5c80a19d0 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 10 client.4380.objecter ms_dispatch 0x7fb5c8001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 3 client.4380.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 3 client.4380.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c4ff9700 20 client.4380.objecter dump_active .. 0 homeless 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5cc89e700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb5b403a350 0x7fb5b403c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5cc89e700 10 monclient: get_auth_request con 0x7fb5b403a350 auth_method 0 2024-04-20T02:15:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_done global_id 4350 payload 824 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.158+0000 7fb5c77fe700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5c8072300 con 0x7fb5c806aac0 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 client.4380.objecter ms_handle_connect 0x7fb5c806aac0 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 client.4380.objecter resend_mon_ops 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 0x7fb5c809c940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009840 tx=0x7fb5bc004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 client.4380.objecter ms_handle_connect 0x7fb5b403a350 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5cc89e700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb5b403a350 0x7fb5b403c810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81202f0 tx=0x7fb5c000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5bc020070 con 0x7fb5c806aac0 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 monclient: dump: 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb5bc003970 con 0x7fb5c806aac0 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 msgr2=0x7fb5c809c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 0x7fb5c809c940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009840 tx=0x7fb5bc004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 msgr2=0x7fb5c809c940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.159+0000 7fb5c4ff9700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c806aac0 0x7fb5c809c940 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5ce301700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5ce301700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 0x7fb5c809d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5ce301700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c806a400 con 0x7fb5c809ce80 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 0x7fb5c809d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5c77fe700 10 monclient: get_auth_request con 0x7fb5c809ce80 auth_method 0 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.167+0000 7fb5c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c77fe700 10 monclient(hunting): handle_auth_done global_id 4359 payload 824 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c77fe700 1 -- 172.21.15.47:0/6722596 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5c80a4480 con 0x7fb5c809ce80 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c77fe700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 0x7fb5c809d290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81cb130 tx=0x7fb5bc011000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 client.4380.objecter ms_handle_connect 0x7fb5c809ce80 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 client.4380.objecter resend_mon_ops 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5bc01b960 con 0x7fb5c809ce80 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 monclient: dump: 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fb5bc01b4f0 con 0x7fb5c809ce80 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.168+0000 7fb5c4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 msgr2=0x7fb5c809d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5c4ff9700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 0x7fb5c809d290 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81cb130 tx=0x7fb5bc011000 comp rx=0 tx=0).stop 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5c4ff9700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 msgr2=0x7fb5c809d290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5c4ff9700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5c809ce80 0x7fb5c809d290 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81cb130 tx=0x7fb5bc011000 comp rx=0 tx=0).stop 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:55.190 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 20, 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:55.191 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.192 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:55.193 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:55.212 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 10 librados: watch_flush enter 2024-04-20T02:15:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 10 librados: watch_flush exit 2024-04-20T02:15:55.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 20 client.4380.objecter shutdown clearing up homeless session... 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 10 client.4380.objecter successfully canceled tick 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb5b403a350 msgr2=0x7fb5b403c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb5b403a350 0x7fb5b403c810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81202f0 tx=0x7fb5c000b040 comp rx=0 tx=0).stop 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 10 monclient: shutdown 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a19d0 msgr2=0x7fb5c81ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 --2- 172.21.15.47:0/6722596 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5c80a19d0 0x7fb5c81ca6d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c440 tx=0x7fb5b8001e70 comp rx=0 tx=0).stop 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5cc89e700 1 -- 172.21.15.47:0/6722596 reap_dead start 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5cc89e700 1 -- 172.21.15.47:0/6722596 reap_dead start 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 shutdown_connections 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 >> 172.21.15.47:0/6722596 conn(0x7fb5c80a4a50 msgr2=0x7fb5c809f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 shutdown_connections 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 -- 172.21.15.47:0/6722596 wait complete. 2024-04-20T02:15:55.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.169+0000 7fb5ce301700 1 librados: shutdown 2024-04-20T02:15:55.215 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:55.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:15:55.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:15:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.361+0000 7f707e849700 10 monclient: get_monmap_and_config 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.361+0000 7f707e849700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.361+0000 7f707e849700 10 monclient: monmap: 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:55.362812+0000 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:55.362812+0000 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.361 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:55.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 1 Processor -- start 2024-04-20T02:15:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 1 -- start start 2024-04-20T02:15:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 10 monclient: init 2024-04-20T02:15:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 10 monclient: _add_conns ranks=[6,8,3,5,4,0,7,10,9,2,1] 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 0x7f707809b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.362+0000 7f707e849700 10 monclient(hunting): picked mon.noname-d con 0x7f7078135ad0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f70781362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): picked mon.noname-e con 0x7f7078135eb0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70780a2950 0x7f70780a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): picked mon.noname-g con 0x7f70780a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707809bc20 con 0x7f70780a2950 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7078067db0 con 0x7f7078135ad0 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f70780a0ae0 con 0x7f7078135eb0 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707e849700 10 monclient(hunting): authenticate will time out at 1634.764062s 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 0x7f707809b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f70781362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 0x7f707809b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:37500/0 (socket says 172.21.15.47:37500) 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 1 -- 172.21.15.47:0/149807132 learned_addr learned my addr 172.21.15.47:0/149807132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): get_auth_request con 0x7f7078135eb0 auth_method 0 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): get_auth_request con 0x7f7078135ad0 auth_method 0 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707cde6700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70780a2950 0x7f70780a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707cde6700 10 monclient(hunting): get_auth_request con 0x7f70780a2950 auth_method 0 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707cde6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707cde6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f707cde6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient(hunting): handle_auth_done global_id 4391 payload 274 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 -- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 msgr2=0x7f707809b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 0x7f707809b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 -- 172.21.15.47:0/149807132 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70780a2950 msgr2=0x7f70780a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70780a2950 0x7f70780a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 monclient: found mon.noname-e 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 -- 172.21.15.47:0/149807132 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780751e0 con 0x7f7078135eb0 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7077fff700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 0x7f707809b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f70777fe700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f70781362b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068009bb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=696010f49e73b882 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7076ffd700 1 -- 172.21.15.47:0/149807132 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7068011aa0 con 0x7f7078135eb0 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.363+0000 7f7076ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: dump: 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 5 monclient: authenticate success, global_id 4391 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 1 -- 172.21.15.47:0/149807132 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068011c00 con 0x7f7078135eb0 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 4 set_mon_vals no callback set 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 1 -- 172.21.15.47:0/149807132 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7068011d70 con 0x7f7078135eb0 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f7076ffd700 10 monclient: dump: 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 10 monclient: shutdown 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 -- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 msgr2=0x7f70781362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f70781362b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068009bb0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 -- 172.21.15.47:0/149807132 shutdown_connections 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70780a2950 0x7f70780a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7078135ad0 0x7f707809b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 --2- 172.21.15.47:0/149807132 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f70781362b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 -- 172.21.15.47:0/149807132 >> 172.21.15.47:0/149807132 conn(0x7f70780a7570 msgr2=0x7f70780a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 -- 172.21.15.47:0/149807132 shutdown_connections 2024-04-20T02:15:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.364+0000 7f707e849700 1 -- 172.21.15.47:0/149807132 wait complete. 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.365+0000 7f707e849700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.365+0000 7f707e849700 10 monclient: monmap: 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.365+0000 7f707e849700 1 librados: starting msgr at 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.365+0000 7f707e849700 1 librados: starting objecter 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.365+0000 7f707e849700 1 Processor -- start 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 -- start start 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 librados: setting wanted keys 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 librados: calling monclient init 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient: init 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient: _add_conns ranks=[1,5,8,6,0,2,7,4,3] 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780a2950 0x7f70781ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): picked mon.noname-f con 0x7f70780a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7078135ad0 0x7f70781315c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): picked mon.noname-h con 0x7f7078135ad0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f7078131b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): picked mon.noname-e con 0x7f7078135eb0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078122a00 con 0x7f70780a2950 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7078121a50 con 0x7f7078135ad0 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7078066ea0 con 0x7f7078135eb0 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707e849700 10 monclient(hunting): authenticate will time out at 1634.767529s 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f7078131b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f7078131b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37902/0 (socket says 172.21.15.47:37902) 2024-04-20T02:15:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 -- 172.21.15.47:0/3191952171 learned_addr learned my addr 172.21.15.47:0/3191952171 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): get_auth_request con 0x7f7078135eb0 auth_method 0 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780a2950 0x7f70781ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f70777fe700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7078135ad0 0x7f70781315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): get_auth_request con 0x7f70780a2950 auth_method 0 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f70777fe700 10 monclient(hunting): get_auth_request con 0x7f7078135ad0 auth_method 0 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f70777fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f70777fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f70777fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient(hunting): handle_auth_done global_id 4400 payload 824 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7078135ad0 msgr2=0x7f70781315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7078135ad0 0x7f70781315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780a2950 msgr2=0x7f70781ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70780a2950 0x7f70781ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 monclient: found mon.noname-e 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f707cde6700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7060009550 con 0x7f7078135eb0 2024-04-20T02:15:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.366+0000 7f70777fe700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7078135ad0 0x7f70781315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707cde6700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f7078131b00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7070007e30 tx=0x7f707000e3c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 client.?.objecter ms_handle_connect 0x7f7078135eb0 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7070009960 con 0x7f7078135eb0 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: dump: 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707000ebe0 con 0x7f7078135eb0 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 5 monclient: authenticate success, global_id 4400 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 10 monclient: _renew_subs 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70680094a0 con 0x7f7078135eb0 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f707000ed50 con 0x7f7078135eb0 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f7074ff9700 10 monclient: dump: 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f70767fc700 4 set_mon_vals no callback set 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 10 client.4400.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 10 monclient: _renew_subs 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078122fc0 con 0x7f7078135eb0 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.367+0000 7f707e849700 1 librados: init done 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f707e849700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f707e849700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 0x7f707811f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7078068310 con 0x7f707806aac0 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f707001e460 con 0x7f7078135eb0 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 0x7f707811f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f705803a320 0x7f705803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient: get_auth_request con 0x7f707806aac0 auth_method 0 2024-04-20T02:15:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f707004a640 con 0x7f7078135eb0 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 10 client.4400.objecter ms_dispatch 0x7f7078001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 3 client.4400.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 3 client.4400.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 20 client.4400.objecter dump_active .. 0 homeless 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f70777fe700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f705803a320 0x7f705803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f70777fe700 10 monclient: get_auth_request con 0x7f705803a320 auth_method 0 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 10 monclient(hunting): handle_auth_done global_id 4451 payload 824 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7078067a40 con 0x7f707806aac0 2024-04-20T02:15:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7077fff700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 0x7f707811f800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7060011bd0 tx=0x7f7060011f90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 10 client.4400.objecter ms_handle_connect 0x7f707806aac0 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.368+0000 7f7074ff9700 10 client.4400.objecter resend_mon_ops 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f706000fe50 con 0x7f707806aac0 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 10 monclient: dump: 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 10 client.4400.objecter ms_handle_connect 0x7f705803a320 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f70777fe700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f705803a320 0x7f705803c7e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f706000f9e0 con 0x7f707806aac0 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 msgr2=0x7f707811f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 0x7f707811f800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7060011bd0 tx=0x7f7060011f90 comp rx=0 tx=0).stop 2024-04-20T02:15:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 msgr2=0x7f707811f800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.369+0000 7f7074ff9700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707806aac0 0x7f707811f800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f707e849700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f707e849700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 0x7f707802bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f707e849700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f707806a400 con 0x7f707802bbc0 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 0x7f707802bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient: get_auth_request con 0x7f707802bbc0 auth_method 0 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 10 monclient(hunting): handle_auth_done global_id 4460 payload 824 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.377+0000 7f7077fff700 1 -- 172.21.15.47:0/3191952171 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f707809a9d0 con 0x7f707802bbc0 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 client.4400.objecter ms_handle_connect 0x7f707802bbc0 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 client.4400.objecter resend_mon_ops 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7077fff700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 0x7f707802bfd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f706001c000 tx=0x7f7060004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7060014ac0 con 0x7f707802bbc0 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 monclient: dump: 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f7060014650 con 0x7f707802bbc0 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 msgr2=0x7f707802bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 0x7f707802bfd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f706001c000 tx=0x7f7060004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 msgr2=0x7f707802bfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f7074ff9700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f707802bbc0 0x7f707802bfd0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f706001c000 tx=0x7f7060004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 21, 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.420 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.421 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:55.421 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f707e849700 10 librados: watch_flush enter 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f707e849700 10 librados: watch_flush exit 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f707e849700 20 client.4400.objecter shutdown clearing up homeless session... 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.378+0000 7f707e849700 10 client.4400.objecter successfully canceled tick 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f705803a320 msgr2=0x7f705803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f705803a320 0x7f705803c7e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068008030 comp rx=0 tx=0).stop 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 10 monclient: shutdown 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 msgr2=0x7f7078131b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f7078131b00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7070007e30 tx=0x7f707000e3c0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707cde6700 1 -- 172.21.15.47:0/3191952171 reap_dead start 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 shutdown_connections 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 --2- 172.21.15.47:0/3191952171 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7078135eb0 0x7f7078131b00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 >> 172.21.15.47:0/3191952171 conn(0x7f70780a7570 msgr2=0x7f70780a5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 shutdown_connections 2024-04-20T02:15:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 -- 172.21.15.47:0/3191952171 wait complete. 2024-04-20T02:15:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.379+0000 7f707e849700 1 librados: shutdown 2024-04-20T02:15:55.424 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:55.424 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:15:55.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.564+0000 7f4968145700 10 monclient: get_monmap_and_config 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.564+0000 7f4968145700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.564+0000 7f4968145700 10 monclient: monmap: 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:55.565248+0000 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:55.565248+0000 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.563 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:55.564 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.564+0000 7f4968145700 1 Processor -- start 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.564+0000 7f4968145700 1 -- start start 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.564+0000 7f4968145700 10 monclient: init 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient: _add_conns ranks=[5,7,2,6,8,10,3,0,1,4,9] 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f49600a0830 0x7f49600a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): picked mon.noname-h con 0x7f49600a0830 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49600a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): picked mon.noname-i con 0x7f49600a01b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 0x7f49600a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): picked mon.noname-b con 0x7f49600a1150 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4960067820 con 0x7f49600a1150 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f496009deb0 con 0x7f49600a0830 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4960067330 con 0x7f49600a01b0 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4968145700 10 monclient(hunting): authenticate will time out at 1634.966587s 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 0x7f49600a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 0x7f49600a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38796/0 (socket says 172.21.15.47:38796) 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 1 -- 172.21.15.47:0/649708646 learned_addr learned my addr 172.21.15.47:0/649708646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): get_auth_request con 0x7f49600a1150 auth_method 0 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4965ee1700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f49600a0830 0x7f49600a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49666e2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4965ee1700 10 monclient(hunting): get_auth_request con 0x7f49600a0830 auth_method 0 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4965ee1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4965ee1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f4965ee1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49656e0700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49600a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49656e0700 10 monclient(hunting): get_auth_request con 0x7f49600a01b0 auth_method 0 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49656e0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49656e0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.565+0000 7f49656e0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49656e0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49656e0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49656e0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 10 monclient(hunting): handle_auth_done global_id 4457 payload 274 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 -- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 msgr2=0x7f49600a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49600a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 -- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f49600a0830 msgr2=0x7f49600a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f49600a0830 0x7f49600a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 monclient: found mon.noname-b 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 -- 172.21.15.47:0/649708646 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600751e0 con 0x7f49600a1150 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f49666e2700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 0x7f49600a2e90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4958009e80 tx=0x7f495800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4db295c75845b877 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 1 -- 172.21.15.47:0/649708646 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f495800e070 con 0x7f49600a1150 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: dump: 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 1 -- 172.21.15.47:0/649708646 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495800bd80 con 0x7f49600a1150 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 1 -- 172.21.15.47:0/649708646 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4958012860 con 0x7f49600a1150 2024-04-20T02:15:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4968145700 5 monclient: authenticate success, global_id 4457 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4968145700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4968145700 4 set_mon_vals no callback set 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.566+0000 7f4964edf700 10 monclient: dump: 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 10 monclient: shutdown 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 -- 172.21.15.47:0/649708646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 msgr2=0x7f49600a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 0x7f49600a2e90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4958009e80 tx=0x7f495800b330 comp rx=0 tx=0).stop 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 -- 172.21.15.47:0/649708646 shutdown_connections 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a1150 0x7f49600a2e90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f49600a0830 0x7f49600a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 --2- 172.21.15.47:0/649708646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49600a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 -- 172.21.15.47:0/649708646 >> 172.21.15.47:0/649708646 conn(0x7f49600a7570 msgr2=0x7f49600a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 -- 172.21.15.47:0/649708646 shutdown_connections 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 1 -- 172.21.15.47:0/649708646 wait complete. 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.567+0000 7f4968145700 10 monclient: monmap: 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 librados: starting msgr at 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 librados: starting objecter 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 Processor -- start 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 -- start start 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 librados: setting wanted keys 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 librados: calling monclient init 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient: init 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient: _add_conns ranks=[7,2,3,8,5,4,1,0,6] 2024-04-20T02:15:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49601cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): picked mon.noname-i con 0x7f49600a01b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a0830 0x7f49601bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): picked mon.noname-b con 0x7f49600a0830 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49600a1150 0x7f49601c0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): picked mon.noname-g con 0x7f49600a1150 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49601abe90 con 0x7f49600a0830 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49601acdf0 con 0x7f49600a1150 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f49601c06a0 con 0x7f49600a01b0 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f4968145700 10 monclient(hunting): authenticate will time out at 1634.969854s 2024-04-20T02:15:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a0830 0x7f49601bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a0830 0x7f49601bfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38800/0 (socket says 172.21.15.47:38800) 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 1 -- 172.21.15.47:0/1035237967 learned_addr learned my addr 172.21.15.47:0/1035237967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 10 monclient(hunting): get_auth_request con 0x7f49600a0830 auth_method 0 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.568+0000 7f49656e0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49601cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): get_auth_request con 0x7f49600a01b0 auth_method 0 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49666e2700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49600a1150 0x7f49601c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49666e2700 10 monclient(hunting): get_auth_request con 0x7f49600a1150 auth_method 0 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49666e2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49666e2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49666e2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 10 monclient(hunting): handle_auth_done global_id 4466 payload 824 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 msgr2=0x7f49601cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49600a01b0 0x7f49601cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49600a1150 msgr2=0x7f49601c0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49600a1150 0x7f49601c0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 monclient: found mon.noname-b 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4958009b30 con 0x7f49600a0830 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49666e2700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f49600a1150 0x7f49601c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49656e0700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a0830 0x7f49601bfc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9b0 tx=0x7f495000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 client.?.objecter ms_handle_connect 0x7f49600a0830 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f495000cc60 con 0x7f49600a0830 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: dump: 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495000ce20 con 0x7f49600a0830 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 5 monclient: authenticate success, global_id 4466 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 10 monclient: _renew_subs 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49500138e0 con 0x7f49600a0830 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49601ad660 con 0x7f49600a0830 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4957fff700 4 set_mon_vals no callback set 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 10 client.4466.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f49567fc700 10 monclient: dump: 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 10 monclient: _renew_subs 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496009e440 con 0x7f49600a0830 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.569+0000 7f4968145700 1 librados: init done 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4968145700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4968145700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 0x7f49601a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4960068310 con 0x7f496006aac0 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4950013470 con 0x7f49600a0830 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 0x7f49601a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient: get_auth_request con 0x7f496006aac0 auth_method 0 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f49567fc700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f494403a2f0 0x7f494403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f495004a140 con 0x7f49600a0830 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f49567fc700 10 client.4466.objecter ms_dispatch 0x7f4960001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f49567fc700 3 client.4466.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.570+0000 7f49567fc700 3 client.4466.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 20 client.4466.objecter dump_active .. 0 homeless 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49666e2700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f494403a2f0 0x7f494403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f4965ee1700 10 monclient(hunting): handle_auth_done global_id 4435 payload 824 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f4965ee1700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f496009f1c0 con 0x7f496006aac0 2024-04-20T02:15:55.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49666e2700 10 monclient: get_auth_request con 0x7f494403a2f0 auth_method 0 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 client.4466.objecter ms_handle_connect 0x7f496006aac0 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 client.4466.objecter resend_mon_ops 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f4965ee1700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 0x7f49601a9720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f495c0119a0 tx=0x7f495c011d60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f495c0036a0 con 0x7f496006aac0 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 monclient: dump: 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 client.4466.objecter ms_handle_connect 0x7f494403a2f0 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49666e2700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f494403a2f0 0x7f494403c7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49600a15d0 tx=0x7f4958009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f495c004240 con 0x7f496006aac0 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 msgr2=0x7f49601a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 0x7f49601a9720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f495c0119a0 tx=0x7f495c011d60 comp rx=0 tx=0).stop 2024-04-20T02:15:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 msgr2=0x7f49601a9720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.571+0000 7f49567fc700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f496006aac0 0x7f49601a9720 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f495c0119a0 tx=0x7f495c011d60 comp rx=0 tx=0).stop 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.579+0000 7f4968145700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.579+0000 7f4968145700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 0x7f49601a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.579+0000 7f4968145700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.579+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f496006a400 con 0x7f49601acf60 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 0x7f49601a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient: get_auth_request con 0x7f49601acf60 auth_method 0 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 10 monclient(hunting): handle_auth_done global_id 4444 payload 824 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 1 -- 172.21.15.47:0/1035237967 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f49600a4450 con 0x7f49601acf60 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f4965ee1700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 0x7f49601a9c60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f495c0091e0 tx=0x7f495c004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f49567fc700 10 client.4466.objecter ms_handle_connect 0x7f49601acf60 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f49567fc700 10 client.4466.objecter resend_mon_ops 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f495c01e450 con 0x7f49601acf60 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f49567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f49567fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.580+0000 7f49567fc700 10 monclient: dump: 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.600 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f495c00fb20 con 0x7f49601acf60 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 msgr2=0x7f49601a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 0x7f49601a9c60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f495c0091e0 tx=0x7f495c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 msgr2=0x7f49601a9c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49567fc700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f49601acf60 0x7f49601a9c60 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f495c0091e0 tx=0x7f495c004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 8, 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:15:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 21, 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:15:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:15:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:15:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:15:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:15:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:15:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:15:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:15:55.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.622 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:15:55.623 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 10 librados: watch_flush enter 2024-04-20T02:15:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 10 librados: watch_flush exit 2024-04-20T02:15:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 20 client.4466.objecter shutdown clearing up homeless session... 2024-04-20T02:15:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 10 client.4466.objecter successfully canceled tick 2024-04-20T02:15:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f494403a2f0 msgr2=0x7f494403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f494403a2f0 0x7f494403c7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49600a15d0 tx=0x7f4958009000 comp rx=0 tx=0).stop 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 10 monclient: shutdown 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a0830 msgr2=0x7f49601bfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 --2- 172.21.15.47:0/1035237967 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f49600a0830 0x7f49601bfc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9b0 tx=0x7f495000ecc0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49666e2700 1 -- 172.21.15.47:0/1035237967 reap_dead start 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f49666e2700 1 -- 172.21.15.47:0/1035237967 reap_dead start 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 shutdown_connections 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 >> 172.21.15.47:0/1035237967 conn(0x7f49600a7570 msgr2=0x7f49600a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 shutdown_connections 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 -- 172.21.15.47:0/1035237967 wait complete. 2024-04-20T02:15:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.581+0000 7f4968145700 1 librados: shutdown 2024-04-20T02:15:55.627 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:15:55.627 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h', 'e', 'a'] 2024-04-20T02:15:55.627 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-20T02:15:55.627 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-04-20T02:15:55.627 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h store 2024-04-20T02:15:55.627 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h sync_force --yes-i-really-mean-it 2024-04-20T02:15:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.767+0000 7f6105d35700 10 monclient: get_monmap_and_config 2024-04-20T02:15:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.767+0000 7f6105d35700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.767+0000 7f6105d35700 10 monclient: monmap: 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:55.768482+0000 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:55.768482+0000 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:55.768 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 Processor -- start 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 -- start start 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient: init 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient: _add_conns ranks=[8,9,5,1,10,0,2,4,6,7,3] 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 0x7f61000a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): picked mon.noname-e con 0x7f61000a0a70 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f61000a2870 0x7f61000a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): picked mon.noname-a con 0x7f61000a2870 addr v2:172.21.15.47:3300/0 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61000a31f0 0x7f610013ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): picked mon.noname-h con 0x7f61000a31f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61000a1390 con 0x7f61000a2870 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61000a36a0 con 0x7f61000a31f0 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6100072560 con 0x7f61000a0a70 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f6105d35700 10 monclient(hunting): authenticate will time out at 1635.169688s 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 0x7f61000a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60feffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f61000a2870 0x7f61000a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 0x7f61000a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37904/0 (socket says 172.21.15.47:37904) 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60feffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f61000a2870 0x7f61000a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43412/0 (socket says 172.21.15.47:43412) 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 1 -- 172.21.15.47:0/4020431418 learned_addr learned my addr 172.21.15.47:0/4020431418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 10 monclient(hunting): get_auth_request con 0x7f61000a0a70 auth_method 0 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60ff7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60feffd700 10 monclient(hunting): get_auth_request con 0x7f61000a2870 auth_method 0 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60feffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60feffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.768+0000 7f60feffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60ff7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60ff7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61000a31f0 0x7f610013ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60ff7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): get_auth_request con 0x7f61000a31f0 auth_method 0 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 10 monclient(hunting): handle_auth_done global_id 4572 payload 274 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 -- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 msgr2=0x7f61000a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 0x7f61000a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 -- 172.21.15.47:0/4020431418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61000a31f0 msgr2=0x7f610013ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61000a31f0 0x7f610013ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 monclient: found mon.noname-a 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 -- 172.21.15.47:0/4020431418 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61000751e0 con 0x7f61000a2870 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60ff7fe700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 0x7f61000a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60feffd700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61000a2870 0x7f61000a2cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009840 tx=0x7f60f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5bc1a984a05af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 1 -- 172.21.15.47:0/4020431418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f4011aa0 con 0x7f61000a2870 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: dump: 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 1 -- 172.21.15.47:0/4020431418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f4011c00 con 0x7f61000a2870 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f6105d35700 5 monclient: authenticate success, global_id 4572 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f6105d35700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f6105d35700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 1 -- 172.21.15.47:0/4020431418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f4011d70 con 0x7f61000a2870 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f6105d35700 4 set_mon_vals no callback set 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.769+0000 7f60fe7fc700 10 monclient: dump: 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 10 monclient: shutdown 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 -- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61000a2870 msgr2=0x7f61000a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61000a2870 0x7f61000a2cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009840 tx=0x7f60f4009b50 comp rx=0 tx=0).stop 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 -- 172.21.15.47:0/4020431418 shutdown_connections 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61000a31f0 0x7f610013ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a0a70 0x7f61000a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 --2- 172.21.15.47:0/4020431418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61000a2870 0x7f61000a2cb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 -- 172.21.15.47:0/4020431418 >> 172.21.15.47:0/4020431418 conn(0x7f61000a4080 msgr2=0x7f610006c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 -- 172.21.15.47:0/4020431418 shutdown_connections 2024-04-20T02:15:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 1 -- 172.21.15.47:0/4020431418 wait complete. 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.770+0000 7f6105d35700 10 monclient: monmap: 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 1 librados: starting msgr at 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 1 librados: starting objecter 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 1 Processor -- start 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 1 -- start start 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 1 librados: setting wanted keys 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 1 librados: calling monclient init 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 10 monclient: init 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.771+0000 7f6105d35700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient: _add_conns ranks=[3,8,4,1,6,2,5,0,7] 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61000a0a70 0x7f610009ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): picked mon.noname-g con 0x7f61000a0a70 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 0x7f61001bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): picked mon.noname-e con 0x7f61000a2870 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f61000a31f0 0x7f61001cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): picked mon.noname-c con 0x7f61000a31f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61001a82e0 con 0x7f61000a0a70 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61001ca360 con 0x7f61000a31f0 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f610009e360 con 0x7f61000a2870 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient(hunting): authenticate will time out at 1635.173082s 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 0x7f61001bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 0x7f61001bbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37914/0 (socket says 172.21.15.47:37914) 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 -- 172.21.15.47:0/3525913043 learned_addr learned my addr 172.21.15.47:0/3525913043 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): get_auth_request con 0x7f61000a2870 auth_method 0 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f61000a31f0 0x7f61001cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): get_auth_request con 0x7f61000a31f0 auth_method 0 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60ff7fe700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61000a0a70 0x7f610009ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient(hunting): handle_auth_done global_id 4418 payload 824 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f61000a31f0 msgr2=0x7f61001cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f61000a31f0 0x7f61001cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61000a0a70 msgr2=0x7f610009ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61000a0a70 0x7f610009ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 monclient: found mon.noname-e 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0009550 con 0x7f61000a2870 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60ff7fe700 10 monclient: get_auth_request con 0x7f61000a0a70 auth_method 0 2024-04-20T02:15:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60ff7fe700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f61000a0a70 0x7f610009ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60feffd700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 0x7f61001bbeb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f60f4013000 tx=0x7f60f4004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 client.?.objecter ms_handle_connect 0x7f61000a2870 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f4004330 con 0x7f61000a2870 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 monclient: dump: 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 5 monclient: authenticate success, global_id 4418 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f401f070 con 0x7f61000a2870 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient: _renew_subs 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40094a0 con 0x7f61000a2870 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f40044b0 con 0x7f61000a2870 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f60e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f60e7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f6105d35700 10 client.4418.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.772+0000 7f60fdffb700 4 set_mon_vals no callback set 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f60e7fff700 10 monclient: dump: 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f6105d35700 10 monclient: _renew_subs 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f6105d35700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61001a8fd0 con 0x7f61000a2870 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f6105d35700 1 librados: init done 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f60f4021030 con 0x7f61000a2870 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.773+0000 7f60e7fff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f60ec03a2f0 0x7f60ec03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f60f404b040 con 0x7f61000a2870 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60e7fff700 10 client.4418.objecter ms_dispatch 0x7f6100001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60e7fff700 3 client.4418.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60e7fff700 3 client.4418.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f6105d35700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f6105d35700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 0x7f61001bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60e7fff700 20 client.4418.objecter dump_active .. 0 homeless 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6100068430 con 0x7f61001a84b0 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60ff7fe700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f60ec03a2f0 0x7f60ec03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60ff7fe700 10 monclient: get_auth_request con 0x7f60ec03a2f0 auth_method 0 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 0x7f61001bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient: get_auth_request con 0x7f61001a84b0 auth_method 0 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 10 monclient(hunting): handle_auth_done global_id 4487 payload 824 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60fffff700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61000a26b0 con 0x7f61001a84b0 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60e7fff700 10 client.4418.objecter ms_handle_connect 0x7f60ec03a2f0 2024-04-20T02:15:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.774+0000 7f60ff7fe700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f60ec03a2f0 0x7f60ec03c7b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009ce0 tx=0x7f60f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 client.4418.objecter ms_handle_connect 0x7f61001a84b0 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 client.4418.objecter resend_mon_ops 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60fffff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 0x7f61001bc3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61001a81b0 tx=0x7f60f800f490 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f800e040 con 0x7f61001a84b0 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 monclient: dump: 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f60f800f040 con 0x7f61001a84b0 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 msgr2=0x7f61001bc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 0x7f61001bc3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61001a81b0 tx=0x7f60f800f490 comp rx=0 tx=0).stop 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 msgr2=0x7f61001bc3f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.775+0000 7f60e7fff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001a84b0 0x7f61001bc3f0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61001a81b0 tx=0x7f60f800f490 comp rx=0 tx=0).stop 2024-04-20T02:15:55.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f6105d35700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:15:55.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f6105d35700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 0x7f610006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f6105d35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f610006a400 con 0x7f61001bc930 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 0x7f610006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient: get_auth_request con 0x7f61001bc930 auth_method 0 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.784+0000 7f60fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60fffff700 10 monclient(hunting): handle_auth_done global_id 4496 payload 824 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60fffff700 1 -- 172.21.15.47:0/3525913043 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f610009ef20 con 0x7f61001bc930 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60fffff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 0x7f610006f030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60f801b000 tx=0x7f60f8004ad0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60e7fff700 10 client.4418.objecter ms_handle_connect 0x7f61001bc930 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60e7fff700 10 client.4418.objecter resend_mon_ops 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60f8023070 con 0x7f61001bc930 2024-04-20T02:15:55.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60e7fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.785+0000 7f60e7fff700 10 monclient: dump: 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.807 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f60f80046d0 con 0x7f61001bc930 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 msgr2=0x7f610006f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 0x7f610006f030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60f801b000 tx=0x7f60f8004ad0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 msgr2=0x7f610006f030 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60e7fff700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f61001bc930 0x7f610006f030 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60f801b000 tx=0x7f60f8004ad0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.808 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:15:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:15:55.809 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:55.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 10 librados: watch_flush enter 2024-04-20T02:15:55.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 10 librados: watch_flush exit 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 20 client.4418.objecter shutdown clearing up homeless session... 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 10 client.4418.objecter successfully canceled tick 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f60ec03a2f0 msgr2=0x7f60ec03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f60ec03a2f0 0x7f60ec03c7b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009ce0 tx=0x7f60f0008040 comp rx=0 tx=0).stop 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 10 monclient: shutdown 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 msgr2=0x7f61001bbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 0x7f61001bbeb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f60f4013000 tx=0x7f60f4004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f60fffff700 1 -- 172.21.15.47:0/3525913043 reap_dead start 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 shutdown_connections 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 --2- 172.21.15.47:0/3525913043 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f61000a2870 0x7f61001bbeb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 >> 172.21.15.47:0/3525913043 conn(0x7f61000a4080 msgr2=0x7f6100071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 shutdown_connections 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 -- 172.21.15.47:0/3525913043 wait complete. 2024-04-20T02:15:55.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.786+0000 7f6105d35700 1 librados: shutdown 2024-04-20T02:15:55.812 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:15:55.812 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-04-20T02:15:55.812 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:15:55.825 INFO:tasks.ceph.mon.h:Stopped 2024-04-20T02:15:55.825 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-04-20T02:15:55.825 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e store 2024-04-20T02:15:55.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e sync_force --yes-i-really-mean-it 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.965+0000 7f523f312700 10 monclient: get_monmap_and_config 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.965+0000 7f523f312700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.965+0000 7f523f312700 10 monclient: monmap: 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:55.966949+0000 2024-04-20T02:15:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:55.966949+0000 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:55.967 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.966+0000 7f523f312700 1 Processor -- start 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.966+0000 7f523f312700 1 -- start start 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.966+0000 7f523f312700 10 monclient: init 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient: _add_conns ranks=[8,7,10,9,2,3,4,1,0,5,6] 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 0x7f52380a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): picked mon.noname-e con 0x7f52380a28e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52380a3200 0x7f5238137a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): picked mon.noname-i con 0x7f52380a3200 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5238137fd0 0x7f5238138450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): picked mon.noname-f con 0x7f5238137fd0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523809eec0 con 0x7f5238137fd0 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f52380a0140 con 0x7f52380a3200 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f523809e890 con 0x7f52380a28e0 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523f312700 10 monclient(hunting): authenticate will time out at 1635.368260s 2024-04-20T02:15:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 0x7f52380a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 0x7f52380a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37918/0 (socket says 172.21.15.47:37918) 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 -- 172.21.15.47:0/380566251 learned_addr learned my addr 172.21.15.47:0/380566251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): get_auth_request con 0x7f52380a28e0 auth_method 0 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523c8ad700 1 --2- 172.21.15.47:0/380566251 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52380a3200 0x7f5238137a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d8af700 1 --2- 172.21.15.47:0/380566251 >> v2:172.21.15.179:3300/0 conn(0x7f5238137fd0 0x7f5238138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523c8ad700 10 monclient(hunting): get_auth_request con 0x7f52380a3200 auth_method 0 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523c8ad700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523c8ad700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523c8ad700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d8af700 10 monclient(hunting): get_auth_request con 0x7f5238137fd0 auth_method 0 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d8af700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d8af700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d8af700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient(hunting): handle_auth_done global_id 4427 payload 274 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 -- 172.21.15.47:0/380566251 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52380a3200 msgr2=0x7f5238137a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 --2- 172.21.15.47:0/380566251 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52380a3200 0x7f5238137a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 -- 172.21.15.47:0/380566251 >> v2:172.21.15.179:3300/0 conn(0x7f5238137fd0 msgr2=0x7f5238138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 --2- 172.21.15.47:0/380566251 >> v2:172.21.15.179:3300/0 conn(0x7f5238137fd0 0x7f5238138450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 monclient: found mon.noname-e 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d0ae700 1 -- 172.21.15.47:0/380566251 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380751e0 con 0x7f52380a28e0 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.967+0000 7f523d8af700 1 --2- 172.21.15.47:0/380566251 >> v2:172.21.15.179:3300/0 conn(0x7f5238137fd0 0x7f5238138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523d0ae700 1 --2- 172.21.15.47:0/380566251 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 0x7f52380a2cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f52340098f0 tx=0x7f5234009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6efec0e1f259216d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 1 -- 172.21.15.47:0/380566251 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f523400bb10 con 0x7f52380a28e0 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: dump: 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 1 -- 172.21.15.47:0/380566251 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400bc70 con 0x7f52380a28e0 2024-04-20T02:15:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 5 monclient: authenticate success, global_id 4427 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 1 -- 172.21.15.47:0/380566251 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f523400bde0 con 0x7f52380a28e0 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 4 set_mon_vals no callback set 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f522bfff700 10 monclient: dump: 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 10 monclient: shutdown 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 -- 172.21.15.47:0/380566251 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 msgr2=0x7f52380a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 --2- 172.21.15.47:0/380566251 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 0x7f52380a2cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f52340098f0 tx=0x7f5234009c00 comp rx=0 tx=0).stop 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 -- 172.21.15.47:0/380566251 shutdown_connections 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 --2- 172.21.15.47:0/380566251 >> v2:172.21.15.179:3300/0 conn(0x7f5238137fd0 0x7f5238138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 --2- 172.21.15.47:0/380566251 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f52380a28e0 0x7f52380a2cc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 --2- 172.21.15.47:0/380566251 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f52380a3200 0x7f5238137a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 -- 172.21.15.47:0/380566251 >> 172.21.15.47:0/380566251 conn(0x7f52380a7500 msgr2=0x7f52380a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 -- 172.21.15.47:0/380566251 shutdown_connections 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.968+0000 7f523f312700 1 -- 172.21.15.47:0/380566251 wait complete. 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 10 monclient: build_initial_monmap 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 10 monclient: monmap: 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 1 librados: starting msgr at 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 1 librados: starting objecter 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 1 Processor -- start 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 1 -- start start 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 1 librados: setting wanted keys 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 1 librados: calling monclient init 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.969+0000 7f523f312700 10 monclient: init 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient: _add_conns ranks=[3,1,0,8,4,2,5,6,7] 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52380a28e0 0x7f52381cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): picked mon.noname-g con 0x7f52380a28e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52380a3200 0x7f52381cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): picked mon.noname-f con 0x7f52380a3200 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 0x7f5238121930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): picked mon.noname-a con 0x7f5238137fd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238072110 con 0x7f5238137fd0 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523809e560 con 0x7f52380a3200 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5238124a10 con 0x7f52380a28e0 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523f312700 10 monclient(hunting): authenticate will time out at 1635.371462s 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 0x7f5238121930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 0x7f5238121930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43416/0 (socket says 172.21.15.47:43416) 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 -- 172.21.15.47:0/2512081179 learned_addr learned my addr 172.21.15.47:0/2512081179 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): get_auth_request con 0x7f5238137fd0 auth_method 0 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52380a28e0 0x7f52381cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52380a3200 0x7f52381cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): get_auth_request con 0x7f52380a28e0 auth_method 0 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): get_auth_request con 0x7f52380a3200 auth_method 0 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523c8ad700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient(hunting): handle_auth_done global_id 4581 payload 824 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52380a28e0 msgr2=0x7f52381cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52380a28e0 0x7f52381cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52380a3200 msgr2=0x7f52381cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52380a3200 0x7f52381cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient: _finish_hunting 0 2024-04-20T02:15:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 monclient: found mon.noname-a 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.970+0000 7f523d8af700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234009550 con 0x7f5238137fd0 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523d8af700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 0x7f5238121930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5230007e70 tx=0x7f523000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 client.?.objecter ms_handle_connect 0x7f5238137fd0 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5230003a20 con 0x7f5238137fd0 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: dump: 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: _finish_auth 0 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 5 monclient: authenticate success, global_id 4581 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523000f040 con 0x7f5238137fd0 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 10 monclient: _renew_subs 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5238124cf0 con 0x7f5238137fd0 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5230003bc0 con 0x7f5238137fd0 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f5229ffb700 10 monclient: dump: 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f522b7fe700 4 set_mon_vals no callback set 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 10 client.4581.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 10 monclient: _renew_subs 2024-04-20T02:15:55.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381ccf70 con 0x7f5238137fd0 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.971+0000 7f523f312700 1 librados: init done 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5230003d20 con 0x7f5238137fd0 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523f312700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523f312700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 0x7f5238070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f52381cd1f0 con 0x7f5238123ab0 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521c03a2f0 0x7f521c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 0x7f5238070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient: get_auth_request con 0x7f5238123ab0 auth_method 0 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f5230049b00 con 0x7f5238137fd0 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 10 client.4581.objecter ms_dispatch 0x7f5238001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 3 client.4581.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 3 client.4581.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 20 client.4581.objecter dump_active .. 0 homeless 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 10 monclient(hunting): handle_auth_done global_id 4436 payload 824 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523c8ad700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521c03a2f0 0x7f521c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52380673b0 con 0x7f5238123ab0 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523c8ad700 10 monclient: get_auth_request con 0x7f521c03a2f0 auth_method 0 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 10 client.4581.objecter ms_handle_connect 0x7f5238123ab0 2024-04-20T02:15:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f523d0ae700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 0x7f5238070370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52340098f0 tx=0x7f5234004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.972+0000 7f5229ffb700 10 client.4581.objecter resend_mon_ops 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5234020070 con 0x7f5238123ab0 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 10 monclient: dump: 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 10 client.4581.objecter ms_handle_connect 0x7f521c03a2f0 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f523c8ad700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521c03a2f0 0x7f521c03c7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f523809d480 tx=0x7f522c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5234004250 con 0x7f5238123ab0 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 msgr2=0x7f5238070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 0x7f5238070370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52340098f0 tx=0x7f5234004750 comp rx=0 tx=0).stop 2024-04-20T02:15:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 msgr2=0x7f5238070370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.973+0000 7f5229ffb700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5238123ab0 0x7f5238070370 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52340098f0 tx=0x7f5234004750 comp rx=0 tx=0).stop 2024-04-20T02:15:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523f312700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:15:56.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523f312700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 0x7f52380708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523f312700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5238068310 con 0x7f523806ace0 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 0x7f52380708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient: get_auth_request con 0x7f523806ace0 auth_method 0 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:56.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 10 monclient(hunting): handle_auth_done global_id 4445 payload 824 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.982+0000 7f523d0ae700 1 -- 172.21.15.47:0/2512081179 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f52380a3680 con 0x7f523806ace0 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 client.4581.objecter ms_handle_connect 0x7f523806ace0 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f523d0ae700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 0x7f52380708b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5234009840 tx=0x7f5234011000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 client.4581.objecter resend_mon_ops 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5234004550 con 0x7f523806ace0 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 monclient: dump: 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.003 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f523402b620 con 0x7f523806ace0 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:56.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 msgr2=0x7f52380708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 0x7f52380708b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5234009840 tx=0x7f5234011000 comp rx=0 tx=0).stop 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 msgr2=0x7f52380708b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f5229ffb700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f523806ace0 0x7f52380708b0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5234009840 tx=0x7f5234011000 comp rx=0 tx=0).stop 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:15:56.005 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f523f312700 10 librados: watch_flush enter 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f523f312700 10 librados: watch_flush exit 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f523f312700 20 client.4581.objecter shutdown clearing up homeless session... 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.983+0000 7f523f312700 10 client.4581.objecter successfully canceled tick 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521c03a2f0 msgr2=0x7f521c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521c03a2f0 0x7f521c03c7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f523809d480 tx=0x7f522c008040 comp rx=0 tx=0).stop 2024-04-20T02:15:56.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 10 monclient: shutdown 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 msgr2=0x7f5238121930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523d8af700 1 -- 172.21.15.47:0/2512081179 reap_dead start 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 0x7f5238121930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5230007e70 tx=0x7f523000d3b0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 shutdown_connections 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 --2- 172.21.15.47:0/2512081179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5238137fd0 0x7f5238121930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 >> 172.21.15.47:0/2512081179 conn(0x7f52380a7500 msgr2=0x7f52380a4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 shutdown_connections 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 -- 172.21.15.47:0/2512081179 wait complete. 2024-04-20T02:15:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:55.984+0000 7f523f312700 1 librados: shutdown 2024-04-20T02:15:56.008 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-04-20T02:15:56.009 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-04-20T02:15:56.009 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:15:56.019 INFO:tasks.ceph.mon.e:Stopped 2024-04-20T02:15:56.019 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-20T02:15:56.019 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-20T02:15:56.019 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T02:15:56.019 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:15:56.071 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T02:15:56.071 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-04-20T02:15:56.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:15:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.255+0000 7f0434dab700 10 monclient: get_monmap_and_config 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.255+0000 7f0434dab700 10 monclient: build_initial_monmap 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.255+0000 7f0434dab700 10 monclient: monmap: 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:56.256403+0000 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:56.256403+0000 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.255+0000 7f0434dab700 1 Processor -- start 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.255+0000 7f0434dab700 1 -- start start 2024-04-20T02:15:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.255+0000 7f0434dab700 10 monclient: init 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient: _add_conns ranks=[5,7,8,10,4,6,2,3,1,9,0] 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f04300716f0 0x7f043006f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): picked mon.noname-h con 0x7f04300716f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 0x7f04300a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): picked mon.noname-i con 0x7f0430071ad0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0430071f30 0x7f04300a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): picked mon.noname-e con 0x7f0430071f30 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f043009e910 con 0x7f04300716f0 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f043009ed20 con 0x7f0430071ad0 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f04300a0000 con 0x7f0430071f30 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042ed9d700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0430071f30 msgr2=0x7f04300a2e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f0434dab700 10 monclient(hunting): authenticate will time out at 1635.657760s 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042e59c700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f04300716f0 msgr2=0x7f043006f090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0430071f30 0x7f04300a2e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042e59c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f04300716f0 0x7f043006f090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 0x7f04300a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 0x7f04300a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:55952/0 (socket says 172.21.15.47:55952) 2024-04-20T02:15:56.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 1 -- 172.21.15.47:0/1414922950 learned_addr learned my addr 172.21.15.47:0/1414922950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 10 monclient(hunting): get_auth_request con 0x7f0430071ad0 auth_method 0 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.256+0000 7f042dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 10 monclient(hunting): handle_auth_done global_id 4462 payload 274 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 -- 172.21.15.47:0/1414922950 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0430071f30 msgr2=0x7f04300a2e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0430071f30 0x7f04300a2e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 -- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f04300716f0 msgr2=0x7f043006f090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f04300716f0 0x7f043006f090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:15:56.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 monclient: found mon.noname-i 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 -- 172.21.15.47:0/1414922950 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300751e0 con 0x7f0430071ad0 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042dd9b700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 0x7f04300a2950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0424009920 tx=0x7f0424009c30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=3a2f654912886115 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 1 -- 172.21.15.47:0/1414922950 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0424009e70 con 0x7f0430071ad0 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: dump: 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: _finish_auth 0 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: _check_auth_tickets 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:56.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 1 -- 172.21.15.47:0/1414922950 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042400bcd0 con 0x7f0430071ad0 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f0434dab700 5 monclient: authenticate success, global_id 4462 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f0434dab700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 1 -- 172.21.15.47:0/1414922950 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f042400be40 con 0x7f0430071ad0 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f042d59a700 10 monclient: dump: 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:56.272 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f0434dab700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.257+0000 7f0434dab700 4 set_mon_vals no callback set 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 10 monclient: shutdown 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 -- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 msgr2=0x7f04300a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 0x7f04300a2950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0424009920 tx=0x7f0424009c30 comp rx=0 tx=0).stop 2024-04-20T02:15:56.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 -- 172.21.15.47:0/1414922950 shutdown_connections 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0430071f30 0x7f04300a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f04300716f0 0x7f043006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 --2- 172.21.15.47:0/1414922950 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0430071ad0 0x7f04300a2950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 -- 172.21.15.47:0/1414922950 >> 172.21.15.47:0/1414922950 conn(0x7f04300a7570 msgr2=0x7f04300a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 -- 172.21.15.47:0/1414922950 shutdown_connections 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.258+0000 7f0434dab700 1 -- 172.21.15.47:0/1414922950 wait complete. 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 10 monclient: build_initial_monmap 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 10 monclient: monmap: 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:56.274 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 1 librados: starting msgr at 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 1 librados: starting objecter 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 1 Processor -- start 2024-04-20T02:15:56.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 1 -- start start 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 1 librados: setting wanted keys 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 1 librados: calling monclient init 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.259+0000 7f0434dab700 10 monclient: init 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient: _add_conns ranks=[3,6,4,2,5,0,7,1,8] 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04300716f0 0x7f04301cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): picked mon.noname-g con 0x7f04300716f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 0x7f04301ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): picked mon.noname-d con 0x7f0430071ad0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0430071f30 0x7f04301244a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): picked mon.noname-c con 0x7f0430071f30 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0430126ad0 con 0x7f04300716f0 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f043009f310 con 0x7f0430071f30 2024-04-20T02:15:56.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0430072480 con 0x7f0430071ad0 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f0434dab700 10 monclient(hunting): authenticate will time out at 1635.661463s 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 0x7f04301ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0430071f30 0x7f04301244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 0x7f04301ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:37526/0 (socket says 172.21.15.47:37526) 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 -- 172.21.15.47:0/4208732241 learned_addr learned my addr 172.21.15.47:0/4208732241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): get_auth_request con 0x7f0430071ad0 auth_method 0 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): get_auth_request con 0x7f0430071f30 auth_method 0 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:56.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04300716f0 0x7f04301cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042ed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): get_auth_request con 0x7f04300716f0 auth_method 0 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:56.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 -- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0430071f30 msgr2=0x7f04301244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0430071f30 0x7f04301244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 -- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04300716f0 msgr2=0x7f04301cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04300716f0 0x7f04301cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 monclient: found mon.noname-d 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.260+0000 7f042dd9b700 1 -- 172.21.15.47:0/4208732241 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009550 con 0x7f0430071ad0 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f042ed9d700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0430071f30 0x7f04301244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f042dd9b700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 0x7f04301ccdc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0424000c00 tx=0x7f0424004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 client.?.objecter ms_handle_connect 0x7f0430071ad0 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:56.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f042400bb60 con 0x7f0430071ad0 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: dump: 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:56.280 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: _finish_auth 0 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: _check_auth_tickets 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 5 monclient: authenticate success, global_id 4398 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042401f070 con 0x7f0430071ad0 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 10 monclient: _renew_subs 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f042400be40 con 0x7f0430071ad0 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0430127ff0 con 0x7f0430071ad0 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:56.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f042cd99700 4 set_mon_vals no callback set 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f041effd700 10 monclient: dump: 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:56.282 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 10 client.4398.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 10 monclient: _renew_subs 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0430125f00 con 0x7f0430071ad0 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.261+0000 7f0434dab700 1 librados: init done 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0424021030 con 0x7f0430071ad0 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f041003a2f0 0x7f041003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f042404b050 con 0x7f0430071ad0 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 10 client.4398.objecter ms_dispatch 0x7f0430001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 3 client.4398.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 3 client.4398.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f041effd700 20 client.4398.objecter dump_active .. 0 homeless 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f042e59c700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f041003a2f0 0x7f041003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f042e59c700 10 monclient: get_auth_request con 0x7f041003a2f0 auth_method 0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f0434dab700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f0434dab700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f0434dab700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.262+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043006bdc0 con 0x7f0430071ad0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.263+0000 7f041effd700 10 client.4398.objecter ms_handle_connect 0x7f041003a2f0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.263+0000 7f042e59c700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f041003a2f0 0x7f041003c7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0420009450 tx=0x7f04200093e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.268+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0424015020 con 0x7f0430071ad0 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.268+0000 7f041effd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:56.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.268+0000 7f041effd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:15:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.571+0000 7f0434dab700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:15:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.571+0000 7f0434dab700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:15:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.571+0000 7f0434dab700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:15:56.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.571+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f04300666c0 con 0x7f0430071ad0 2024-04-20T02:15:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.571+0000 7f041effd700 1 -- 172.21.15.47:0/4208732241 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f0424032070 con 0x7f0430071ad0 2024-04-20T02:15:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.572+0000 7f041effd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:15:56.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.572+0000 7f041effd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:15:56.571 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":8,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 10 librados: watch_flush enter 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 10 librados: watch_flush exit 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 20 client.4398.objecter shutdown clearing up homeless session... 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 10 client.4398.objecter successfully canceled tick 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f041003a2f0 msgr2=0x7f041003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f041003a2f0 0x7f041003c7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0420009450 tx=0x7f04200093e0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 10 monclient: shutdown 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 msgr2=0x7f04301ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 0x7f04301ccdc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0424000c00 tx=0x7f0424004750 comp rx=0 tx=0).stop 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 shutdown_connections 2024-04-20T02:15:56.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f041003a2f0 0x7f041003c7b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0430071f30 0x7f04301244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04300716f0 0x7f04301cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 --2- 172.21.15.47:0/4208732241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0430071ad0 0x7f04301ccdc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 >> 172.21.15.47:0/4208732241 conn(0x7f04300a7570 msgr2=0x7f043006f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 shutdown_connections 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 -- 172.21.15.47:0/4208732241 wait complete. 2024-04-20T02:15:56.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:56.574+0000 7f0434dab700 1 librados: shutdown 2024-04-20T02:15:59.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:15:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.733+0000 7f525dbfb700 10 monclient: get_monmap_and_config 2024-04-20T02:15:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.733+0000 7f525dbfb700 10 monclient: build_initial_monmap 2024-04-20T02:15:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.733+0000 7f525dbfb700 10 monclient: monmap: 2024-04-20T02:15:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:59.733349+0000 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:59.733349+0000 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:15:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:15:59.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 Processor -- start 2024-04-20T02:15:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 -- start start 2024-04-20T02:15:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient: init 2024-04-20T02:15:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient: _add_conns ranks=[3,10,6,0,8,5,7,4,2,1,9] 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5258066610 0x7f52580a1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): picked mon.noname-g con 0x7f5258066610 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f525809c2e0 0x7f52581357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): picked mon.noname-f con 0x7f525809c2e0 addr v2:172.21.15.179:3300/0 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 0x7f5258135cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): picked mon.noname-d con 0x7f525809c6e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580669f0 con 0x7f525809c2e0 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580a2890 con 0x7f5258066610 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f525809ccc0 con 0x7f525809c6e0 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f525dbfb700 10 monclient(hunting): authenticate will time out at 1639.134678s 2024-04-20T02:15:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 0x7f5258135cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5256ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f525809c2e0 0x7f52581357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 0x7f5258135cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:37528/0 (socket says 172.21.15.47:37528) 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 1 -- 172.21.15.47:0/2374755946 learned_addr learned my addr 172.21.15.47:0/2374755946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 10 monclient(hunting): get_auth_request con 0x7f525809c6e0 auth_method 0 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5257fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f52577fe700 1 --2- 172.21.15.47:0/2374755946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5258066610 0x7f52580a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5256ffd700 10 monclient(hunting): get_auth_request con 0x7f525809c2e0 auth_method 0 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5256ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5256ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f5256ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f52577fe700 10 monclient(hunting): get_auth_request con 0x7f5258066610 auth_method 0 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f52577fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f52577fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.734+0000 7f52577fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5256ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5256ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5256ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52577fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52577fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52577fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 10 monclient(hunting): handle_auth_done global_id 4407 payload 274 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 -- 172.21.15.47:0/2374755946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5258066610 msgr2=0x7f52580a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 --2- 172.21.15.47:0/2374755946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5258066610 0x7f52580a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 -- 172.21.15.47:0/2374755946 >> v2:172.21.15.179:3300/0 conn(0x7f525809c2e0 msgr2=0x7f52581357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 --2- 172.21.15.47:0/2374755946 >> v2:172.21.15.179:3300/0 conn(0x7f525809c2e0 0x7f52581357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 10 monclient: _finish_hunting 0 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 monclient: found mon.noname-d 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 -- 172.21.15.47:0/2374755946 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580751e0 con 0x7f525809c6e0 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5256ffd700 1 --2- 172.21.15.47:0/2374755946 >> v2:172.21.15.179:3300/0 conn(0x7f525809c2e0 0x7f52581357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f5257fff700 1 --2- 172.21.15.47:0/2374755946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 0x7f5258135cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5250009e50 tx=0x7f525000b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1a9b2a7797960ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 1 -- 172.21.15.47:0/2374755946 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f525000e070 con 0x7f525809c6e0 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: dump: 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:59.748 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: _finish_auth 0 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: _check_auth_tickets 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 1 -- 172.21.15.47:0/2374755946 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52500099b0 con 0x7f525809c6e0 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f525dbfb700 5 monclient: authenticate success, global_id 4407 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f525dbfb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 1 -- 172.21.15.47:0/2374755946 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f525000bd80 con 0x7f525809c6e0 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f525dbfb700 10 monclient: get_monmap_and_config success 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f525dbfb700 4 set_mon_vals no callback set 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.735+0000 7f52567fc700 10 monclient: dump: 2024-04-20T02:15:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 10 monclient: shutdown 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 -- 172.21.15.47:0/2374755946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 msgr2=0x7f5258135cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2374755946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 0x7f5258135cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5250009e50 tx=0x7f525000b330 comp rx=0 tx=0).stop 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 -- 172.21.15.47:0/2374755946 shutdown_connections 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2374755946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f525809c6e0 0x7f5258135cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2374755946 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5258066610 0x7f52580a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2374755946 >> v2:172.21.15.179:3300/0 conn(0x7f525809c2e0 0x7f52581357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 -- 172.21.15.47:0/2374755946 >> 172.21.15.47:0/2374755946 conn(0x7f52580a6720 msgr2=0x7f52580a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 -- 172.21.15.47:0/2374755946 shutdown_connections 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 1 -- 172.21.15.47:0/2374755946 wait complete. 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.736+0000 7f525dbfb700 10 monclient: build_initial_monmap 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 10 monclient: monmap: 2024-04-20T02:15:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:15:59.752 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 1 librados: starting msgr at 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 1 librados: starting objecter 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 1 Processor -- start 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 1 -- start start 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 1 librados: setting wanted keys 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 1 librados: calling monclient init 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.737+0000 7f525dbfb700 10 monclient: init 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient: _reopen_session rank -1 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient: _add_conns ranks=[1,4,7,3,6,5,2,0,8] 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5258066610 0x7f52581ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): picked mon.noname-f con 0x7f5258066610 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 0x7f52581b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): picked mon.noname-c con 0x7f525809c2e0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:15:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f525809c6e0 0x7f52581b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): picked mon.noname-i con 0x7f525809c6e0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580a04e0 con 0x7f5258066610 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52581b2aa0 con 0x7f525809c2e0 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f52581b2d80 con 0x7f525809c6e0 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): _renew_subs 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f525dbfb700 10 monclient(hunting): authenticate will time out at 1639.138258s 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 0x7f52581b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 0x7f52581b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43260/0 (socket says 172.21.15.47:43260) 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 -- 172.21.15.47:0/2482164050 learned_addr learned my addr 172.21.15.47:0/2482164050 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:15:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): get_auth_request con 0x7f525809c2e0 auth_method 0 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5258066610 0x7f52581ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): get_auth_request con 0x7f5258066610 auth_method 0 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f525809c6e0 0x7f52581b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): get_auth_request con 0x7f525809c6e0 auth_method 0 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f52577fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5257fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient(hunting): handle_auth_done global_id 4495 payload 824 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 -- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f525809c6e0 msgr2=0x7f52581b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f525809c6e0 0x7f52581b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 -- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5258066610 msgr2=0x7f52581ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:15:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5258066610 0x7f52581ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 monclient: found mon.noname-c 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 -- 172.21.15.47:0/2482164050 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0094a0 con 0x7f525809c2e0 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f5256ffd700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 0x7f52581b7880 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f524c00ebb0 tx=0x7f524c00eec0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f523ffff700 10 client.?.objecter ms_handle_connect 0x7f525809c2e0 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f523ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f524c00ccc0 con 0x7f525809c2e0 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.738+0000 7f523ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: dump: 2024-04-20T02:15:59.758 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:59.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: _finish_auth 0 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00ce20 con 0x7f525809c2e0 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 5 monclient: authenticate success, global_id 4495 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 10 monclient: _renew_subs 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5250009b30 con 0x7f525809c2e0 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f524c0138e0 con 0x7f525809c2e0 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f5255ffb700 4 set_mon_vals no callback set 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 10 client.4495.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f523ffff700 10 monclient: dump: 2024-04-20T02:15:59.760 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:15:59.761 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 10 monclient: _renew_subs 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581bb230 con 0x7f525809c2e0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.739+0000 7f525dbfb700 1 librados: init done 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f525dbfb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f525dbfb700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f525dbfb700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52581b32a0 con 0x7f525809c2e0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f524c013470 con 0x7f525809c2e0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f523ffff700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f524403a320 0x7f524403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f524c04a190 con 0x7f525809c2e0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f523ffff700 10 client.4495.objecter ms_dispatch 0x7f5258001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f523ffff700 3 client.4495.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:15:59.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.740+0000 7f523ffff700 3 client.4495.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.741+0000 7f52577fe700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f524403a320 0x7f524403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.741+0000 7f523ffff700 20 client.4495.objecter dump_active .. 0 homeless 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.741+0000 7f52577fe700 10 monclient: get_auth_request con 0x7f524403a320 auth_method 0 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.741+0000 7f523ffff700 10 client.4495.objecter ms_handle_connect 0x7f524403a320 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.741+0000 7f52577fe700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f524403a320 0x7f524403c7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5248009be0 tx=0x7f5248009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.745+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f52581b32a0 con 0x7f525809c2e0 2024-04-20T02:15:59.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.745+0000 7f523ffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:15:59.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:15:59.745+0000 7f523ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.051+0000 7f525dbfb700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.051+0000 7f525dbfb700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.051+0000 7f525dbfb700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.051+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f52581b32a0 con 0x7f525809c2e0 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.052+0000 7f523ffff700 1 -- 172.21.15.47:0/2482164050 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f52581b32a0 con 0x7f525809c2e0 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.052+0000 7f523ffff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.052+0000 7f523ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:00.050 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":8,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 10 librados: watch_flush enter 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 10 librados: watch_flush exit 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 20 client.4495.objecter shutdown clearing up homeless session... 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 10 client.4495.objecter successfully canceled tick 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f524403a320 msgr2=0x7f524403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f524403a320 0x7f524403c7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5248009be0 tx=0x7f5248009210 comp rx=0 tx=0).stop 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 10 monclient: shutdown 2024-04-20T02:16:00.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 msgr2=0x7f52581b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 0x7f52581b7880 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f524c00ebb0 tx=0x7f524c00eec0 comp rx=0 tx=0).stop 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 shutdown_connections 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f525809c6e0 0x7f52581b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f524403a320 0x7f524403c7e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5258066610 0x7f52581ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 --2- 172.21.15.47:0/2482164050 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f525809c2e0 0x7f52581b7880 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 >> 172.21.15.47:0/2482164050 conn(0x7f52580a6720 msgr2=0x7f52581bd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 shutdown_connections 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 -- 172.21.15.47:0/2482164050 wait complete. 2024-04-20T02:16:00.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:00.054+0000 7f525dbfb700 1 librados: shutdown 2024-04-20T02:16:03.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:16:03.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.210+0000 7f7a9b345700 10 monclient: get_monmap_and_config 2024-04-20T02:16:03.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.210+0000 7f7a9b345700 10 monclient: build_initial_monmap 2024-04-20T02:16:03.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.210+0000 7f7a9b345700 10 monclient: monmap: 2024-04-20T02:16:03.209 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:03.211199+0000 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:03.211199+0000 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:03.210 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:03.211 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:03.211 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:03.211 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:03.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.210+0000 7f7a9b345700 1 Processor -- start 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.210+0000 7f7a9b345700 1 -- start start 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.210+0000 7f7a9b345700 10 monclient: init 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient: _add_conns ranks=[0,8,7,5,4,10,3,9,1,2,6] 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7a9406cc40 addr v1:172.21.15.47:6789/0 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d020 0x7f7a9413cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): picked mon.noname-e con 0x7f7a9406d020 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 0x7f7a9413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): picked mon.noname-i con 0x7f7a9406d4a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940751e0 con 0x7f7a9406cc40 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940a4260 con 0x7f7a9406d4a0 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940a4430 con 0x7f7a9406d020 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a988e0700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d020 msgr2=0x7f7a9413cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a9b345700 10 monclient(hunting): authenticate will time out at 1642.612530s 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a990e1700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f7a9406cc40 legacy=0x7f7a9406db50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a988e0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d020 0x7f7a9413cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 0x7f7a9413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 0x7f7a9413f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:55976/0 (socket says 172.21.15.47:55976) 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 1 -- 172.21.15.47:0/2591570255 learned_addr learned my addr 172.21.15.47:0/2591570255 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): get_auth_request con 0x7f7a9406d4a0 auth_method 0 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.211+0000 7f7a998e2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 10 monclient(hunting): handle_auth_done global_id 4480 payload 274 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 1 -- 172.21.15.47:0/2591570255 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d020 msgr2=0x7f7a9413cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 1 --2- 172.21.15.47:0/2591570255 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d020 0x7f7a9413cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 1 -- 172.21.15.47:0/2591570255 >> v1:172.21.15.47:6789/0 conn(0x7f7a9406cc40 legacy=0x7f7a9406db50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 10 monclient: _finish_hunting 0 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 1 monclient: found mon.noname-i 2024-04-20T02:16:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 1 -- 172.21.15.47:0/2591570255 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940704e0 con 0x7f7a9406d4a0 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a998e2700 1 --2- 172.21.15.47:0/2591570255 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 0x7f7a9413f1c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009eb0 tx=0x7f7a8c00b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=7d2b4d2e5704e394 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 1 -- 172.21.15.47:0/2591570255 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a8c00e070 con 0x7f7a9406d4a0 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: dump: 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: _finish_auth 0 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 1 -- 172.21.15.47:0/2591570255 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c003e90 con 0x7f7a9406d4a0 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 1 -- 172.21.15.47:0/2591570255 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a8c00bdb0 con 0x7f7a9406d4a0 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a8bfff700 10 monclient: dump: 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:03.227 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a9b345700 5 monclient: authenticate success, global_id 4480 2024-04-20T02:16:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a9b345700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.212+0000 7f7a9b345700 4 set_mon_vals no callback set 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 10 monclient: shutdown 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 -- 172.21.15.47:0/2591570255 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 msgr2=0x7f7a9413f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2591570255 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 0x7f7a9413f1c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009eb0 tx=0x7f7a8c00b330 comp rx=0 tx=0).stop 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 -- 172.21.15.47:0/2591570255 shutdown_connections 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2591570255 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a9406d4a0 0x7f7a9413f1c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2591570255 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d020 0x7f7a9413cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 -- 172.21.15.47:0/2591570255 >> 172.21.15.47:0/2591570255 conn(0x7f7a940a5680 msgr2=0x7f7a94066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 -- 172.21.15.47:0/2591570255 shutdown_connections 2024-04-20T02:16:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 1 -- 172.21.15.47:0/2591570255 wait complete. 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 10 monclient: build_initial_monmap 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.213+0000 7f7a9b345700 10 monclient: monmap: 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 librados: starting msgr at 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 librados: starting objecter 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 Processor -- start 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 -- start start 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 librados: setting wanted keys 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 librados: calling monclient init 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient: init 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient: _add_conns ranks=[2,1,8,6,0,5,4,7,3] 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 0x7f7a941ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): picked mon.noname-b con 0x7f7a9406cc40 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:16:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9406d020 0x7f7a941cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): picked mon.noname-f con 0x7f7a9406d020 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d4a0 0x7f7a94123cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): picked mon.noname-e con 0x7f7a9406d4a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941273b0 con 0x7f7a9406d020 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941264c0 con 0x7f7a9406cc40 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94072920 con 0x7f7a9406d4a0 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a9b345700 10 monclient(hunting): authenticate will time out at 1642.615780s 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a998e2700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d4a0 msgr2=0x7f7a94123cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a998e2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d4a0 0x7f7a94123cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 0x7f7a941ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 0x7f7a941ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:38816/0 (socket says 172.21.15.47:38816) 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 1 -- 172.21.15.47:0/2118938200 learned_addr learned my addr 172.21.15.47:0/2118938200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 10 monclient(hunting): get_auth_request con 0x7f7a9406cc40 auth_method 0 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a990e1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.214+0000 7f7a988e0700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9406d020 0x7f7a941cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): get_auth_request con 0x7f7a9406d020 auth_method 0 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a988e0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 10 monclient(hunting): handle_auth_done global_id 4475 payload 824 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 -- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d4a0 msgr2=0x7f7a94123cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d4a0 0x7f7a94123cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 -- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9406d020 msgr2=0x7f7a941cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9406d020 0x7f7a941cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 10 monclient: _finish_hunting 0 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 monclient: found mon.noname-b 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 -- 172.21.15.47:0/2118938200 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c009b30 con 0x7f7a9406cc40 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a990e1700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 0x7f7a941ccb70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b550 tx=0x7f7a9000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 client.?.objecter ms_handle_connect 0x7f7a9406cc40 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a90005070 con 0x7f7a9406cc40 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: dump: 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: _finish_auth 0 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90005210 con 0x7f7a9406cc40 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a9b345700 5 monclient: authenticate success, global_id 4475 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a9b345700 10 monclient: _renew_subs 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a9b345700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:16:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a94072540 con 0x7f7a9406cc40 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a900057f0 con 0x7f7a9406cc40 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a89ffb700 10 monclient: dump: 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.215+0000 7f7a8b7fe700 4 set_mon_vals no callback set 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 10 client.4475.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 10 monclient: _renew_subs 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a94125880 con 0x7f7a9406cc40 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 1 librados: init done 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7a90005380 con 0x7f7a9406cc40 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a89ffb700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7a7803a2f0 0x7f7a7803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:16:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a94067f90 con 0x7f7a9406cc40 2024-04-20T02:16:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f7a90048f10 con 0x7f7a9406cc40 2024-04-20T02:16:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a89ffb700 10 client.4475.objecter ms_dispatch 0x7f7a94001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a89ffb700 3 client.4475.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.216+0000 7f7a89ffb700 3 client.4475.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.217+0000 7f7a988e0700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7a7803a2f0 0x7f7a7803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.217+0000 7f7a988e0700 10 monclient: get_auth_request con 0x7f7a7803a2f0 auth_method 0 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.217+0000 7f7a89ffb700 20 client.4475.objecter dump_active .. 0 homeless 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.217+0000 7f7a89ffb700 10 client.4475.objecter ms_handle_connect 0x7f7a7803a2f0 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.217+0000 7f7a988e0700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7a7803a2f0 0x7f7a7803c7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a94125450 tx=0x7f7a84009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.222+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7a90016020 con 0x7f7a9406cc40 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.222+0000 7f7a89ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.222+0000 7f7a89ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.524+0000 7f7a9b345700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.524+0000 7f7a9b345700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.524+0000 7f7a9b345700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.524+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7a94070150 con 0x7f7a9406cc40 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.525+0000 7f7a89ffb700 1 -- 172.21.15.47:0/2118938200 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f7a94070150 con 0x7f7a9406cc40 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.525+0000 7f7a89ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.525+0000 7f7a89ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:03.525 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":8,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":29,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 10 librados: watch_flush enter 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 10 librados: watch_flush exit 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 20 client.4475.objecter shutdown clearing up homeless session... 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 10 client.4475.objecter successfully canceled tick 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7a7803a2f0 msgr2=0x7f7a7803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7a7803a2f0 0x7f7a7803c7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a94125450 tx=0x7f7a84009190 comp rx=0 tx=0).stop 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 10 monclient: shutdown 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 msgr2=0x7f7a941ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 0x7f7a941ccb70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b550 tx=0x7f7a9000b860 comp rx=0 tx=0).stop 2024-04-20T02:16:03.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 shutdown_connections 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7a9406d4a0 0x7f7a94123cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9406cc40 0x7f7a941ccb70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7a7803a2f0 0x7f7a7803c7b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 --2- 172.21.15.47:0/2118938200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a9406d020 0x7f7a941cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 >> 172.21.15.47:0/2118938200 conn(0x7f7a940a5680 msgr2=0x7f7a94138020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 shutdown_connections 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 -- 172.21.15.47:0/2118938200 wait complete. 2024-04-20T02:16:03.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:03.527+0000 7f7a9b345700 1 librados: shutdown 2024-04-20T02:16:06.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.686+0000 7f8b7345a700 10 monclient: get_monmap_and_config 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.686+0000 7f8b7345a700 10 monclient: build_initial_monmap 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.686+0000 7f8b7345a700 10 monclient: monmap: 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:06.687348+0000 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:06.687348+0000 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:06.686 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.686+0000 7f8b7345a700 1 Processor -- start 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 -- start start 2024-04-20T02:16:06.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient: init 2024-04-20T02:16:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient: _add_conns ranks=[8,3,10,4,1,5,9,2,7,0,6] 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 0x7f8b6c0a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): picked mon.noname-e con 0x7f8b6c0a4c80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): picked mon.noname-g con 0x7f8b6c0a0d20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): picked mon.noname-f con 0x7f8b6c0a55a0 addr v2:172.21.15.179:3300/0 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c0a0000 con 0x7f8b6c0a55a0 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0a6a10 con 0x7f8b6c0a0d20 2024-04-20T02:16:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c09e440 con 0x7f8b6c0a4c80 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b7345a700 10 monclient(hunting): authenticate will time out at 1646.088606s 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b711f6700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 msgr2=0x7f8b6c0a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b711f6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 0x7f8b6c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:42022/0 (socket says 172.21.15.47:42022) 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 learned_addr learned my addr 172.21.15.47:0/989615549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a55a0 auth_method 0 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a0d20 auth_method 0 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.687+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.688+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:06.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 msgr2=0x7f8b6c0a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 0x7f8b6c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a55a0 auth_method 0 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:06.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.888+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a0d20 auth_method 0 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:06.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:06.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:06.889+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:16:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 msgr2=0x7f8b6c0a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 0x7f8b6c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:16:07.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a55a0 auth_method 0 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.289+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a0d20 auth_method 0 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:07.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:07.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:07.290+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:16:08.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 msgr2=0x7f8b6c0a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:08.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 0x7f8b6c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a55a0 auth_method 0 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:08.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.090+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 10 monclient(hunting): get_auth_request con 0x7f8b6c0a0d20 auth_method 0 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:08.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:08.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:08.091+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:16:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): tick 2024-04-20T02:16:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:16:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:16:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:16:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 msgr2=0x7f8b6c0a07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8b6c0a4c80 0x7f8b6c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 msgr2=0x7f8b6c0a5060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b6c0a0d20 0x7f8b6c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 -- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 msgr2=0x7f8b6c137a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 --2- 172.21.15.47:0/989615549 >> v2:172.21.15.179:3300/0 conn(0x7f8b6c0a55a0 0x7f8b6c137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient: _add_conns ranks=[4,1,0,6,2,9,7,5] 2024-04-20T02:16:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): picked mon.noname-c con 0x7f8b500038a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8b50006230 addr v1:172.21.15.179:6789/0 2024-04-20T02:16:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b50009f40 addr v1:172.21.15.47:6789/0 2024-04-20T02:16:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 -- 172.21.15.47:0/989615549 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6c0751e0 con 0x7f8b50009f40 2024-04-20T02:16:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 -- 172.21.15.47:0/989615549 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5000edb0 con 0x7f8b50006230 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 1 -- 172.21.15.47:0/989615549 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0a0000 con 0x7f8b500038a0 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b62ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.687+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v1:172.21.15.47:6789/0 conn(0x7f8b50009f40 legacy=0x7f8b5000c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 10 monclient(hunting): get_auth_request con 0x7f8b500038a0 auth_method 0 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.688+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.888+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v1:172.21.15.47:6789/0 conn(0x7f8b50009f40 legacy=0x7f8b5000c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:09.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 10 monclient(hunting): get_auth_request con 0x7f8b500038a0 auth_method 0 2024-04-20T02:16:09.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:09.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:09.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:09.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:09.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:09.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:09.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:09.889+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:16:10.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.289+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> v1:172.21.15.47:6789/0 conn(0x7f8b50009f40 legacy=0x7f8b5000c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:10.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 10 monclient(hunting): get_auth_request con 0x7f8b500038a0 auth_method 0 2024-04-20T02:16:10.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:16:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:16:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:16:10.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.290+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:16:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.313+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166489597 0 0) 0x7f8b5000edb0 con 0x7f8b50006230 2024-04-20T02:16:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.313+0000 7f8b63fff700 10 monclient(hunting): _negotiate 2024-04-20T02:16:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.313+0000 7f8b63fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.313+0000 7f8b63fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.313+0000 7f8b63fff700 10 monclient(hunting): my global_id is 4618 2024-04-20T02:16:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.313+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b580035b0 con 0x7f8b50006230 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3163617412 0 0) 0x7f8b580035b0 con 0x7f8b50006230 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 msgr2=0x7f8b50005cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b500038a0 0x7f8b50005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 >> v1:172.21.15.47:6789/0 conn(0x7f8b50009f40 legacy=0x7f8b5000c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: _finish_hunting 0 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:16:10.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 reap_dead start 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50010000 con 0x7f8b50006230 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: _finish_auth 0 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: _check_auth_tickets 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f8b5c00c760 con 0x7f8b50006230 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:16:10.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: dump: 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 >> v1:172.21.15.179:6789/0 conn(0x7f8b50006230 legacy=0x7f8b500086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient: _add_conns ranks=[4,6,1,2,3,8,7,0,5] 2024-04-20T02:16:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b58004250 0x7f8b58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): picked mon.c con 0x7f8b58004250 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8b58004eb0 0x7f8b58009a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): picked mon.d con 0x7f8b58004eb0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b58009f40 0x7f8b5800c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): picked mon.f con 0x7f8b58009f40 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0a0000 con 0x7f8b58009f40 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5800ca80 con 0x7f8b58004250 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5800cbf0 con 0x7f8b58004eb0 2024-04-20T02:16:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b63fff700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b58004250 0x7f8b58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b711f6700 10 monclient(hunting): get_auth_request con 0x7f8b58004250 auth_method 0 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b709f5700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8b58004eb0 0x7f8b58009a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b711f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b711f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b711f6700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b58009f40 0x7f8b5800c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b719f7700 10 monclient(hunting): get_auth_request con 0x7f8b58009f40 auth_method 0 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b719f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b719f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b719f7700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b709f5700 10 monclient(hunting): get_auth_request con 0x7f8b58004eb0 auth_method 0 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b709f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b709f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.315+0000 7f8b709f5700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b711f6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b711f6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b711f6700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b709f5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b709f5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b709f5700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient(hunting): handle_auth_done global_id 4618 payload 293 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8b58004eb0 msgr2=0x7f8b58009a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8b58004eb0 0x7f8b58009a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b58004250 msgr2=0x7f8b58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:16:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b58004250 0x7f8b58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient: _finish_hunting 0 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 monclient: found mon.f 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 -- 172.21.15.47:0/989615549 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580035b0 con 0x7f8b58009f40 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient: _finish_auth 0 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 10 monclient: _check_auth_tickets 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b7345a700 5 monclient: authenticate success, global_id 4618 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b711f6700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b58004250 0x7f8b58004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:16:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b7345a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b719f7700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b58009f40 0x7f8b5800c330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0675f0 tx=0x7f8b58005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf281726c57c363 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b64009e80 con 0x7f8b58009f40 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: dump: 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6400f070 con 0x7f8b58009f40 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b7345a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b7345a700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 1 -- 172.21.15.47:0/989615549 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b64007cc0 con 0x7f8b58009f40 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b7345a700 4 set_mon_vals no callback set 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.316+0000 7f8b63fff700 10 monclient: dump: 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 10 monclient: shutdown 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 -- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b58009f40 msgr2=0x7f8b5800c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b58009f40 0x7f8b5800c330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0675f0 tx=0x7f8b58005880 comp rx=0 tx=0).stop 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 -- 172.21.15.47:0/989615549 shutdown_connections 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b58009f40 0x7f8b5800c330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8b58004eb0 0x7f8b58009a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 --2- 172.21.15.47:0/989615549 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8b58004250 0x7f8b58004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 -- 172.21.15.47:0/989615549 >> 172.21.15.47:0/989615549 conn(0x7f8b6c0aa280 msgr2=0x7f8b6c0aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 -- 172.21.15.47:0/989615549 shutdown_connections 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.317+0000 7f8b7345a700 1 -- 172.21.15.47:0/989615549 wait complete. 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.318+0000 7f8b7345a700 10 monclient: build_initial_monmap 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.318+0000 7f8b7345a700 10 monclient: monmap: 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 librados: starting msgr at 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 librados: starting objecter 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 Processor -- start 2024-04-20T02:16:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 -- start start 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 librados: setting wanted keys 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 librados: calling monclient init 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 10 monclient: init 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 10 monclient: _add_conns ranks=[3,7,0,5,1,8,6,2,4] 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 0x7f8b6c1cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 10 monclient(hunting): picked mon.noname-g con 0x7f8b50006230 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.319+0000 7f8b7345a700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b58004250 0x7f8b6c1bd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): picked mon.noname-i con 0x7f8b58004250 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58004eb0 0x7f8b6c1bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): picked mon.noname-a con 0x7f8b58004eb0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:16:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c09ea70 con 0x7f8b58004eb0 2024-04-20T02:16:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c09dd20 con 0x7f8b50006230 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1ab960 con 0x7f8b58004250 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b7345a700 10 monclient(hunting): authenticate will time out at 1649.721064s 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b719f7700 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58004eb0 msgr2=0x7f8b6c1bdcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b719f7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58004eb0 0x7f8b6c1bdcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 0x7f8b6c1cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b58004250 0x7f8b6c1bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 0x7f8b6c1cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:51720/0 (socket says 172.21.15.47:51720) 2024-04-20T02:16:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 -- 172.21.15.47:0/4285849880 learned_addr learned my addr 172.21.15.47:0/4285849880 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): get_auth_request con 0x7f8b50006230 auth_method 0 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): get_auth_request con 0x7f8b58004250 auth_method 0 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b709f5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient(hunting): handle_auth_done global_id 4413 payload 824 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 -- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b58004250 msgr2=0x7f8b6c1bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b58004250 0x7f8b6c1bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 -- 172.21.15.47:0/4285849880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58004eb0 msgr2=0x7f8b6c1bdcc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58004eb0 0x7f8b6c1bdcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient: _finish_hunting 0 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 monclient: found mon.noname-g 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 -- 172.21.15.47:0/4285849880 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c1ad160 con 0x7f8b50006230 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b711f6700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 0x7f8b6c1cbe30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000ea0 tx=0x7f8b6800e680 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b61ffb700 10 client.?.objecter ms_handle_connect 0x7f8b50006230 2024-04-20T02:16:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.320+0000 7f8b61ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b68014070 con 0x7f8b50006230 2024-04-20T02:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: dump: 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: _finish_auth 0 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68015040 con 0x7f8b50006230 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b6800fe30 con 0x7f8b50006230 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:16:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b61ffb700 10 monclient: dump: 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b637fe700 4 set_mon_vals no callback set 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b7345a700 5 monclient: authenticate success, global_id 4413 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b7345a700 10 monclient: _renew_subs 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b7345a700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.324+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c1ac8c0 con 0x7f8b50006230 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.325+0000 7f8b7345a700 10 client.4413.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.325+0000 7f8b7345a700 10 monclient: _renew_subs 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.325+0000 7f8b7345a700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.325+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c1be200 con 0x7f8b50006230 2024-04-20T02:16:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.325+0000 7f8b7345a700 1 librados: init done 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8b68022410 con 0x7f8b50006230 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b5003b430 0x7f8b5003d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b7345a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b7345a700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b7345a700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8b6804aff0 con 0x7f8b50006230 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 10 client.4413.objecter ms_dispatch 0x7f8b6c001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 3 client.4413.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 3 client.4413.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c1acdc0 con 0x7f8b50006230 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b61ffb700 20 client.4413.objecter dump_active .. 0 homeless 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b709f5700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b5003b430 0x7f8b5003d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.326+0000 7f8b709f5700 10 monclient: get_auth_request con 0x7f8b5003b430 auth_method 0 2024-04-20T02:16:10.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.327+0000 7f8b709f5700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b5003b430 0x7f8b5003d8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c8d0 tx=0x7f8b5c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.327+0000 7f8b61ffb700 10 client.4413.objecter ms_handle_connect 0x7f8b5003b430 2024-04-20T02:16:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.333+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f8b6c1acdc0 con 0x7f8b50006230 2024-04-20T02:16:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.333+0000 7f8b61ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:10.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.333+0000 7f8b61ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b7345a700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:16:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b7345a700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:16:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b7345a700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:16:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8b6c1acdc0 con 0x7f8b50006230 2024-04-20T02:16:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b61ffb700 1 -- 172.21.15.47:0/4285849880 <== mon.3 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3157 (secure 0 0 0) 0x7f8b6c1acdc0 con 0x7f8b50006230 2024-04-20T02:16:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b61ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:16:10.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.644+0000 7f8b61ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:10.644 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":12,"quorum":[1,2,3,4,6,7],"quorum_names":["f","b","g","c","d","i"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 10 librados: watch_flush enter 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 10 librados: watch_flush exit 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 20 client.4413.objecter shutdown clearing up homeless session... 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 10 client.4413.objecter successfully canceled tick 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b5003b430 msgr2=0x7f8b5003d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b5003b430 0x7f8b5003d8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c8d0 tx=0x7f8b5c008040 comp rx=0 tx=0).stop 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 10 monclient: shutdown 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 msgr2=0x7f8b6c1cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 0x7f8b6c1cbe30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b68000ea0 tx=0x7f8b6800e680 comp rx=0 tx=0).stop 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 shutdown_connections 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b5003b430 0x7f8b5003d8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b58004eb0 0x7f8b6c1bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b50006230 0x7f8b6c1cbe30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 --2- 172.21.15.47:0/4285849880 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b58004250 0x7f8b6c1bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 >> 172.21.15.47:0/4285849880 conn(0x7f8b6c0aa280 msgr2=0x7f8b6c0a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 shutdown_connections 2024-04-20T02:16:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 -- 172.21.15.47:0/4285849880 wait complete. 2024-04-20T02:16:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.647+0000 7f8b7345a700 1 librados: shutdown 2024-04-20T02:16:10.660 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-04-20T02:16:10.660 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-04-20T02:16:10.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.798+0000 7f8df5001700 10 monclient: get_monmap_and_config 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.798+0000 7f8df5001700 10 monclient: build_initial_monmap 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.798+0000 7f8df5001700 10 monclient: monmap: 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:10.799722+0000 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:10.799722+0000 2024-04-20T02:16:10.798 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:10.799 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:10.800 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:10.800 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 1 Processor -- start 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 1 -- start start 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient: init 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient: _add_conns ranks=[0,8,1,4,6,3,2,10,9,7,5] 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8df0137d90 addr v1:172.21.15.47:6789/0 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 0x7f8df013cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient(hunting): picked mon.noname-e con 0x7f8df00a09e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8df013d1c0 addr v1:172.21.15.179:6789/0 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df00751e0 con 0x7f8df0137d90 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df009dfc0 con 0x7f8df013d1c0 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8deed9d700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f8df0137d90 legacy=0x7f8df0138170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:10.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8df009fa40 con 0x7f8df00a09e0 2024-04-20T02:16:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8dee59c700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 msgr2=0x7f8df013cc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8df5001700 10 monclient(hunting): authenticate will time out at 1650.200958s 2024-04-20T02:16:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.799+0000 7f8dee59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 0x7f8df013cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8def59e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8df013d1c0 0x7f8df013f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:36414/0 (socket says 172.21.15.47:36414) 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8def59e700 1 -- 172.21.15.47:0/1510782439 learned_addr learned my addr 172.21.15.47:0/1510782439 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878532841 0 0) 0x7f8df009dfc0 con 0x7f8df013d1c0 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): _negotiate 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): my global_id is 4627 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc0035b0 con 0x7f8df013d1c0 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2145006220 0 0) 0x7f8ddc0035b0 con 0x7f8df013d1c0 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 msgr2=0x7f8df013cc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:10.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 0x7f8df013cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 >> v1:172.21.15.47:6789/0 conn(0x7f8df0137d90 legacy=0x7f8df0138170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0138880 con 0x7f8df013d1c0 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: _finish_auth 0 2024-04-20T02:16:10.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: _check_auth_tickets 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f8de8001c60 con 0x7f8df013d1c0 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: dump: 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.817 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 >> v1:172.21.15.179:6789/0 conn(0x7f8df013d1c0 legacy=0x7f8df013f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient: _add_conns ranks=[3,8,0,7,6,1,4,2,5] 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ddc004280 0x7f8ddc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): picked mon.g con 0x7f8ddc004280 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 0x7f8ddc005010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): picked mon.e con 0x7f8df00a09e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:10.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc009a00 0x7f8ddc00bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): picked mon.a con 0x7f8ddc009a00 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df009fa40 con 0x7f8ddc009a00 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc00c5b0 con 0x7f8ddc004280 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc00c720 con 0x7f8df00a09e0 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dedd9b700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dee59c700 1 -- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 msgr2=0x7f8ddc005010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8def59e700 1 -- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc009a00 msgr2=0x7f8ddc00bdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8def59e700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc009a00 0x7f8ddc00bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.800+0000 7f8dee59c700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 0x7f8ddc005010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ddc004280 0x7f8ddc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): get_auth_request con 0x7f8ddc004280 auth_method 0 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient(hunting): handle_auth_done global_id 4627 payload 293 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 -- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 msgr2=0x7f8ddc005010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8df00a09e0 0x7f8ddc005010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 -- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc009a00 msgr2=0x7f8ddc00bdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc009a00 0x7f8ddc00bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 monclient: found mon.g 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:16:10.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 -- 172.21.15.47:0/1510782439 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0035b0 con 0x7f8ddc004280 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient: _finish_auth 0 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 10 monclient: _check_auth_tickets 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8df5001700 5 monclient: authenticate success, global_id 4627 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8df5001700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8deed9d700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ddc004280 0x7f8ddc004850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e30 tx=0x7f8ddc005880 comp rx=0 tx=0).ready entity=mon.3 client_cookie=1df4b13bdf6d4fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dd80097f0 con 0x7f8ddc004280 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: dump: 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.821 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd8003a40 con 0x7f8ddc004280 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8df5001700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:10.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 1 -- 172.21.15.47:0/1510782439 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dd8003bb0 con 0x7f8ddc004280 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8df5001700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8df5001700 4 set_mon_vals no callback set 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.801+0000 7f8dedd9b700 10 monclient: dump: 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.823 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 10 monclient: shutdown 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 1 -- 172.21.15.47:0/1510782439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ddc004280 msgr2=0x7f8ddc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 1 --2- 172.21.15.47:0/1510782439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ddc004280 0x7f8ddc004850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e30 tx=0x7f8ddc005880 comp rx=0 tx=0).stop 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8def59e700 1 -- 172.21.15.47:0/1510782439 reap_dead start 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 1 -- 172.21.15.47:0/1510782439 shutdown_connections 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 1 -- 172.21.15.47:0/1510782439 >> 172.21.15.47:0/1510782439 conn(0x7f8df00aa8b0 msgr2=0x7f8df00a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 1 -- 172.21.15.47:0/1510782439 shutdown_connections 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.802+0000 7f8df5001700 1 -- 172.21.15.47:0/1510782439 wait complete. 2024-04-20T02:16:10.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.803+0000 7f8df5001700 10 monclient: build_initial_monmap 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.803+0000 7f8df5001700 10 monclient: monmap: 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:10.825 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.803+0000 7f8df5001700 1 librados: starting msgr at 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.803+0000 7f8df5001700 1 librados: starting objecter 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.803+0000 7f8df5001700 1 Processor -- start 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 -- start start 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 librados: setting wanted keys 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 librados: calling monclient init 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient: init 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient: _add_conns ranks=[0,6,5,2,7,1,3,4,8] 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc004280 0x7f8df00a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): picked mon.noname-a con 0x7f8ddc004280 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8df009b840 0x7f8df009bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): picked mon.noname-d con 0x7f8df009b840 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8df009c180 0x7f8df012e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): picked mon.noname-h con 0x7f8df009c180 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8deed9d700 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc004280 msgr2=0x7f8df00a3ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df009ccb0 con 0x7f8ddc004280 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00a3020 con 0x7f8df009c180 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8df009e9e0 con 0x7f8df009b840 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8deed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc004280 0x7f8df00a3ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8df5001700 10 monclient(hunting): authenticate will time out at 1650.205587s 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8def59e700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8df009c180 msgr2=0x7f8df012e9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8def59e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8df009c180 0x7f8df012e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8df009b840 0x7f8df009bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8df009b840 0x7f8df009bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:42932/0 (socket says 172.21.15.47:42932) 2024-04-20T02:16:10.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 1 -- 172.21.15.47:0/3488210723 learned_addr learned my addr 172.21.15.47:0/3488210723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): get_auth_request con 0x7f8df009b840 auth_method 0 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.804+0000 7f8dee59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 10 monclient(hunting): handle_auth_done global_id 4416 payload 824 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8df009c180 msgr2=0x7f8df012e9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8df009c180 0x7f8df012e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc004280 msgr2=0x7f8df00a3ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ddc004280 0x7f8df00a3ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 10 monclient: _finish_hunting 0 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 monclient: found mon.noname-d 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:10.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df009dc20 con 0x7f8df009b840 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8dee59c700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8df009b840 0x7f8df009bc40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b560 tx=0x7f8de000b920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 client.?.objecter ms_handle_connect 0x7f8df009b840 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8de0003c40 con 0x7f8df009b840 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: dump: 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.829 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: _finish_auth 0 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 5 monclient: authenticate success, global_id 4416 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0003dc0 con 0x7f8df009b840 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 10 monclient: _renew_subs 2024-04-20T02:16:10.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df00a1d80 con 0x7f8df009b840 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8de00103a0 con 0x7f8df009b840 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8ded59a700 4 set_mon_vals no callback set 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 10 client.4416.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8de77fe700 10 monclient: dump: 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.831 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 10 monclient: _renew_subs 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df009d200 con 0x7f8df009b840 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.805+0000 7f8df5001700 1 librados: init done 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8df5001700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8df5001700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 0x7f8df012eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8df5001700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0068180 con 0x7f8df006aac0 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8de0005070 con 0x7f8df009b840 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 0x7f8df012eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient: get_auth_request con 0x7f8df006aac0 auth_method 0 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8dd003a2f0 0x7f8dd003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f8de0048f80 con 0x7f8df009b840 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 10 client.4416.objecter ms_dispatch 0x7f8df0001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 3 client.4416.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:10.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 3 client.4416.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 20 client.4416.objecter dump_active .. 0 homeless 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 10 monclient(hunting): handle_auth_done global_id 4484 payload 824 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8def59e700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8dd003a2f0 0x7f8dd003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8df00a0000 con 0x7f8df006aac0 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8def59e700 10 monclient: get_auth_request con 0x7f8dd003a2f0 auth_method 0 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 10 client.4416.objecter ms_handle_connect 0x7f8df006aac0 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8deed9d700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 0x7f8df012eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e30 tx=0x7f8dd8005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.806+0000 7f8de77fe700 10 client.4416.objecter resend_mon_ops 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dd801c040 con 0x7f8df006aac0 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 10 monclient: dump: 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.834 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.835 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 10 client.4416.objecter ms_handle_connect 0x7f8dd003a2f0 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8def59e700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8dd003a2f0 0x7f8dd003c7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8df00a7700 tx=0x7f8de80012b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8dd800bd70 con 0x7f8df006aac0 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 msgr2=0x7f8df012eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 0x7f8df012eef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e30 tx=0x7f8dd8005aa0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 msgr2=0x7f8df012eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:10.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.807+0000 7f8de77fe700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df006aac0 0x7f8df012eef0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e30 tx=0x7f8dd8005aa0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8df5001700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:16:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8df5001700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 0x7f8df01365d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8df5001700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df006a400 con 0x7f8df012f430 2024-04-20T02:16:10.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 0x7f8df01365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient: get_auth_request con 0x7f8df012f430 auth_method 0 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 10 monclient(hunting): handle_auth_done global_id 4493 payload 824 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 1 -- 172.21.15.47:0/3488210723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8df00aa0c0 con 0x7f8df012f430 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8deed9d700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 0x7f8df01365d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80038e0 tx=0x7f8dd8009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8de77fe700 10 client.4416.objecter ms_handle_connect 0x7f8df012f430 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8de77fe700 10 client.4416.objecter resend_mon_ops 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8dd800bee0 con 0x7f8df012f430 2024-04-20T02:16:10.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8de77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8de77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.815+0000 7f8de77fe700 10 monclient: dump: 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:10.840 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9320 (secure 0 0 0) 0x7f8dd80282f0 con 0x7f8df012f430 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 msgr2=0x7f8df01365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 0x7f8df01365d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80038e0 tx=0x7f8dd8009520 comp rx=0 tx=0).stop 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 msgr2=0x7f8df01365d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8de77fe700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8df012f430 0x7f8df01365d0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80038e0 tx=0x7f8dd8009520 comp rx=0 tx=0).stop 2024-04-20T02:16:10.841 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 12, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: 7 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:16:10.842 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:10.843 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:16:10.844 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:10.845 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:16:10.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.849 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:16:10.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.852 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:16:10.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.855 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.856 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:16:10.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:10.860 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:10.861 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:10.862 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:16:10.862 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 10 librados: watch_flush enter 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 10 librados: watch_flush exit 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 20 client.4416.objecter shutdown clearing up homeless session... 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 10 client.4416.objecter successfully canceled tick 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8dd003a2f0 msgr2=0x7f8dd003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8dd003a2f0 0x7f8dd003c7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8df00a7700 tx=0x7f8de80012b0 comp rx=0 tx=0).stop 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 10 monclient: shutdown 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8df009b840 msgr2=0x7f8df009bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 --2- 172.21.15.47:0/3488210723 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8df009b840 0x7f8df009bc40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b560 tx=0x7f8de000b920 comp rx=0 tx=0).stop 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8def59e700 1 -- 172.21.15.47:0/3488210723 reap_dead start 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8def59e700 1 -- 172.21.15.47:0/3488210723 reap_dead start 2024-04-20T02:16:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 shutdown_connections 2024-04-20T02:16:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 >> 172.21.15.47:0/3488210723 conn(0x7f8df00aa8b0 msgr2=0x7f8df0131d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 shutdown_connections 2024-04-20T02:16:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 -- 172.21.15.47:0/3488210723 wait complete. 2024-04-20T02:16:10.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:10.816+0000 7f8df5001700 1 librados: shutdown 2024-04-20T02:16:10.865 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:16:10.865 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-04-20T02:16:10.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:16:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.005+0000 7fb666e7a700 10 monclient: get_monmap_and_config 2024-04-20T02:16:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.005+0000 7fb666e7a700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.005+0000 7fb666e7a700 10 monclient: monmap: 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:11.006312+0000 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:11.006312+0000 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.005 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.006 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.006 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:11.006 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:11.006 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.005+0000 7fb666e7a700 1 Processor -- start 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 -- start start 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient: init 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient: _add_conns ranks=[5,7,9,8,1,2,3,6,0,4,10] 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb6600a0b90 0x7fb6600a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): picked mon.noname-h con 0x7fb6600a0b90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 0x7fb6600a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): picked mon.noname-i con 0x7fb6600a2950 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6600a3290 0x7fb66013eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): picked mon.noname-a con 0x7fb6600a3290 addr v2:172.21.15.47:3300/0 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6600a14b0 con 0x7fb6600a3290 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600a0210 con 0x7fb6600a0b90 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600673e0 con 0x7fb6600a2950 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb666e7a700 10 monclient(hunting): authenticate will time out at 1650.407641s 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb665417700 1 -- >> v2:172.21.15.47:3300/0 conn(0x7fb6600a3290 msgr2=0x7fb66013eb40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb664c16700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb6600a0b90 msgr2=0x7fb6600a0f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb664c16700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb6600a0b90 0x7fb6600a0f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb665417700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb6600a3290 0x7fb66013eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 0x7fb6600a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 0x7fb6600a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:37236/0 (socket says 172.21.15.47:37236) 2024-04-20T02:16:11.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 1 -- 172.21.15.47:0/1977236722 learned_addr learned my addr 172.21.15.47:0/1977236722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 10 monclient(hunting): get_auth_request con 0x7fb6600a2950 auth_method 0 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.006+0000 7fb65ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 10 monclient(hunting): handle_auth_done global_id 4498 payload 274 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 -- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb6600a0b90 msgr2=0x7fb6600a0f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 --2- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb6600a0b90 0x7fb6600a0f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 -- 172.21.15.47:0/1977236722 >> v2:172.21.15.47:3300/0 conn(0x7fb6600a3290 msgr2=0x7fb66013eb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 --2- 172.21.15.47:0/1977236722 >> v2:172.21.15.47:3300/0 conn(0x7fb6600a3290 0x7fb66013eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 monclient: found mon.noname-i 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 -- 172.21.15.47:0/1977236722 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600751e0 con 0x7fb6600a2950 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65ffff700 1 --2- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 0x7fb6600a2d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6480098f0 tx=0x7fb648009c00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ea3b5ef016c203c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 1 -- 172.21.15.47:0/1977236722 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb64800bb10 con 0x7fb6600a2950 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: dump: 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.020 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 1 -- 172.21.15.47:0/1977236722 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64800bc70 con 0x7fb6600a2950 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 1 -- 172.21.15.47:0/1977236722 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb64800bde0 con 0x7fb6600a2950 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb65f7fe700 10 monclient: dump: 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb666e7a700 5 monclient: authenticate success, global_id 4498 2024-04-20T02:16:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb666e7a700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.007+0000 7fb666e7a700 4 set_mon_vals no callback set 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 10 monclient: shutdown 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 -- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 msgr2=0x7fb6600a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 --2- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 0x7fb6600a2d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6480098f0 tx=0x7fb648009c00 comp rx=0 tx=0).stop 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 -- 172.21.15.47:0/1977236722 shutdown_connections 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 --2- 172.21.15.47:0/1977236722 >> v2:172.21.15.47:3300/0 conn(0x7fb6600a3290 0x7fb66013eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 --2- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb6600a0b90 0x7fb6600a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 --2- 172.21.15.47:0/1977236722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6600a2950 0x7fb6600a2d50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 -- 172.21.15.47:0/1977236722 >> 172.21.15.47:0/1977236722 conn(0x7fb6600a7570 msgr2=0x7fb6600a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 -- 172.21.15.47:0/1977236722 shutdown_connections 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.008+0000 7fb666e7a700 1 -- 172.21.15.47:0/1977236722 wait complete. 2024-04-20T02:16:11.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 10 monclient: monmap: 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.025 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 1 librados: starting msgr at 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 1 librados: starting objecter 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 1 Processor -- start 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 1 -- start start 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 1 librados: setting wanted keys 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 1 librados: calling monclient init 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.009+0000 7fb666e7a700 10 monclient: init 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient: _add_conns ranks=[1,0,4,2,5,6,3,8,7] 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6600a0b90 0x7fb6601cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): picked mon.noname-f con 0x7fb6600a0b90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:16:11.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6600a2950 0x7fb6601ba810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): picked mon.noname-a con 0x7fb6600a2950 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6600a3290 0x7fb6601bad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): picked mon.noname-c con 0x7fb6600a3290 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6601a9f30 con 0x7fb6600a2950 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6601a7e40 con 0x7fb6600a0b90 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb66009eab0 con 0x7fb6600a3290 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb666e7a700 10 monclient(hunting): authenticate will time out at 1650.411163s 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb65ffff700 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6600a2950 msgr2=0x7fb6601ba810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb65ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6600a2950 0x7fb6601ba810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6600a3290 0x7fb6601bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6600a0b90 0x7fb6601cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6600a3290 0x7fb6601bad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48506/0 (socket says 172.21.15.47:48506) 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 1 -- 172.21.15.47:0/152477358 learned_addr learned my addr 172.21.15.47:0/152477358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): get_auth_request con 0x7fb6600a3290 auth_method 0 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): get_auth_request con 0x7fb6600a0b90 auth_method 0 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient(hunting): handle_auth_done global_id 4636 payload 824 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6600a3290 msgr2=0x7fb6601bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6600a3290 0x7fb6601bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6600a2950 msgr2=0x7fb6601ba810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6600a2950 0x7fb6601ba810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 monclient: found mon.noname-f 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb648009550 con 0x7fb6600a0b90 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb665417700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6600a3290 0x7fb6601bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:16:11.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb664c16700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6600a0b90 0x7fb6601cc350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b520 tx=0x7fb65000b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb65d7fa700 10 client.?.objecter ms_handle_connect 0x7fb6600a0b90 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.010+0000 7fb65d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb650005070 con 0x7fb6600a0b90 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: dump: 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.030 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650005210 con 0x7fb6600a0b90 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 5 monclient: authenticate success, global_id 4636 2024-04-20T02:16:11.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 monclient: _renew_subs 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66009e560 con 0x7fb6600a0b90 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6500057f0 con 0x7fb6600a0b90 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65d7fa700 10 monclient: dump: 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.032 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 client.4636.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 monclient: _renew_subs 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6601aa210 con 0x7fb6600a0b90 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb65effd700 4 set_mon_vals no callback set 2024-04-20T02:16:11.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 1 librados: init done 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 0x7fb6601a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.011+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb660068310 con 0x7fb66006aac0 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 0x7fb6601a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb650005380 con 0x7fb6600a0b90 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient: get_auth_request con 0x7fb66006aac0 auth_method 0 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 10 monclient(hunting): handle_auth_done global_id 4513 payload 824 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb64003a2f0 0x7fb64003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb66009ef90 con 0x7fb66006aac0 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fb650048e70 con 0x7fb6600a0b90 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 10 client.4636.objecter ms_dispatch 0x7fb660001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 3 client.4636.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 3 client.4636.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 20 client.4636.objecter dump_active .. 0 homeless 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb665417700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb64003a2f0 0x7fb64003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 10 client.4636.objecter ms_handle_connect 0x7fb66006aac0 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 10 client.4636.objecter resend_mon_ops 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65ffff700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 0x7fb6601a66b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6600a7f50 tx=0x7fb648004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb665417700 10 monclient: get_auth_request con 0x7fb64003a2f0 auth_method 0 2024-04-20T02:16:11.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb648004570 con 0x7fb66006aac0 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.012+0000 7fb65d7fa700 10 monclient: dump: 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 10 client.4636.objecter ms_handle_connect 0x7fb64003a2f0 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb665417700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb64003a2f0 0x7fb64003c7b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb660072300 tx=0x7fb65800d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb64801f070 con 0x7fb66006aac0 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 msgr2=0x7fb6601a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 0x7fb6601a66b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6600a7f50 tx=0x7fb648004750 comp rx=0 tx=0).stop 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 msgr2=0x7fb6601a66b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.013+0000 7fb65d7fa700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb66006aac0 0x7fb6601a66b0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6600a7f50 tx=0x7fb648004750 comp rx=0 tx=0).stop 2024-04-20T02:16:11.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.020+0000 7fb666e7a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb666e7a700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 0x7fb6601a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb666e7a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb66006bb90 con 0x7fb6601a6bf0 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 0x7fb6601a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient: get_auth_request con 0x7fb6601a6bf0 auth_method 0 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 10 monclient(hunting): handle_auth_done global_id 4522 payload 824 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 1 -- 172.21.15.47:0/152477358 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb66009f270 con 0x7fb6601a6bf0 2024-04-20T02:16:11.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65ffff700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 0x7fb6601a7000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb648009840 tx=0x7fb6480092c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65d7fa700 10 client.4636.objecter ms_handle_connect 0x7fb6601a6bf0 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65d7fa700 10 client.4636.objecter resend_mon_ops 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb648004400 con 0x7fb6601a6bf0 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.021+0000 7fb65d7fa700 10 monclient: dump: 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.041 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9320 (secure 0 0 0) 0x7fb64802a300 con 0x7fb6601a6bf0 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 msgr2=0x7fb6601a7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 0x7fb6601a7000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb648009840 tx=0x7fb6480092c0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 msgr2=0x7fb6601a7000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb65d7fa700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6601a6bf0 0x7fb6601a7000 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb648009840 tx=0x7fb6480092c0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 12, 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:16:11.043 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: 7 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.044 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.045 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:16:11.046 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:16:11.047 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.048 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.051 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:16:11.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.054 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.056 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:16:11.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.062 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.064 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:16:11.065 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:16:11.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 10 librados: watch_flush enter 2024-04-20T02:16:11.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 10 librados: watch_flush exit 2024-04-20T02:16:11.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 20 client.4636.objecter shutdown clearing up homeless session... 2024-04-20T02:16:11.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 10 client.4636.objecter successfully canceled tick 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb64003a2f0 msgr2=0x7fb64003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb64003a2f0 0x7fb64003c7b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb660072300 tx=0x7fb65800d040 comp rx=0 tx=0).stop 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 10 monclient: shutdown 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6600a0b90 msgr2=0x7fb6601cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 --2- 172.21.15.47:0/152477358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6600a0b90 0x7fb6601cc350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb65000b520 tx=0x7fb65000b830 comp rx=0 tx=0).stop 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb665417700 1 -- 172.21.15.47:0/152477358 reap_dead start 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb665417700 1 -- 172.21.15.47:0/152477358 reap_dead start 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 shutdown_connections 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 >> 172.21.15.47:0/152477358 conn(0x7fb6600a7570 msgr2=0x7fb6600723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 shutdown_connections 2024-04-20T02:16:11.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 -- 172.21.15.47:0/152477358 wait complete. 2024-04-20T02:16:11.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.022+0000 7fb666e7a700 1 librados: shutdown 2024-04-20T02:16:11.069 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:16:11.069 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-04-20T02:16:11.069 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:16:11.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.208+0000 7ff9e3e47700 10 monclient: get_monmap_and_config 2024-04-20T02:16:11.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.208+0000 7ff9e3e47700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.208+0000 7ff9e3e47700 10 monclient: monmap: 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:11.209482+0000 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:11.209482+0000 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.208 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:11.209 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:11.209 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.208+0000 7ff9e3e47700 1 Processor -- start 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 1 -- start start 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient: init 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient: _add_conns ranks=[4,5,0,6,9,3,2,10,7,1,8] 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 0x7ff9dc0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): picked mon.noname-c con 0x7ff9dc0a2950 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): picked mon.noname-h con 0x7ff9dc0a3270 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff9dc138040 addr v1:172.21.15.47:6789/0 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9dc0751e0 con 0x7ff9dc138040 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc09f150 con 0x7ff9dc0a2950 2024-04-20T02:16:11.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc09f710 con 0x7ff9dc0a3270 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e3e47700 10 monclient(hunting): authenticate will time out at 1650.610784s 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e23e4700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7ff9dc138040 legacy=0x7ff9dc13ef60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e13e2700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff9dc0a3270 msgr2=0x7ff9dc137b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e13e2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc137b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 0x7ff9dc0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 0x7ff9dc0a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48526/0 (socket says 172.21.15.47:48526) 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 1 -- 172.21.15.47:0/967764668 learned_addr learned my addr 172.21.15.47:0/967764668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 10 monclient(hunting): get_auth_request con 0x7ff9dc0a2950 auth_method 0 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.209+0000 7ff9e1be3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_done global_id 4531 payload 274 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 1 -- 172.21.15.47:0/967764668 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff9dc0a3270 msgr2=0x7ff9dc137b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 1 --2- 172.21.15.47:0/967764668 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc137b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 1 -- 172.21.15.47:0/967764668 >> v1:172.21.15.47:6789/0 conn(0x7ff9dc138040 legacy=0x7ff9dc13ef60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 1 monclient: found mon.noname-c 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 1 -- 172.21.15.47:0/967764668 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0672d0 con 0x7ff9dc0a2950 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e1be3700 1 --2- 172.21.15.47:0/967764668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 0x7ff9dc0a2d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009920 tx=0x7ff9d8009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3a140f3b8d989892 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 1 -- 172.21.15.47:0/967764668 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9d8009e70 con 0x7ff9dc0a2950 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: dump: 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.223 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e3e47700 5 monclient: authenticate success, global_id 4531 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e3e47700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 1 -- 172.21.15.47:0/967764668 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d80044a0 con 0x7ff9dc0a2950 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e3e47700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e3e47700 4 set_mon_vals no callback set 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 1 -- 172.21.15.47:0/967764668 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9d800bc90 con 0x7ff9dc0a2950 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:16:11.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.210+0000 7ff9e0be1700 10 monclient: dump: 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 10 monclient: shutdown 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 -- 172.21.15.47:0/967764668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 msgr2=0x7ff9dc0a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/967764668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 0x7ff9dc0a2d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009920 tx=0x7ff9d8009c30 comp rx=0 tx=0).stop 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 -- 172.21.15.47:0/967764668 shutdown_connections 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/967764668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a2950 0x7ff9dc0a2d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/967764668 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 -- 172.21.15.47:0/967764668 >> 172.21.15.47:0/967764668 conn(0x7ff9dc0a7570 msgr2=0x7ff9dc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 -- 172.21.15.47:0/967764668 shutdown_connections 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.211+0000 7ff9e3e47700 1 -- 172.21.15.47:0/967764668 wait complete. 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 10 monclient: monmap: 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.226 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 1 librados: starting msgr at 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 1 librados: starting objecter 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 1 Processor -- start 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 1 -- start start 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 1 librados: setting wanted keys 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 1 librados: calling monclient init 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.212+0000 7ff9e3e47700 10 monclient: init 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient: _add_conns ranks=[3,4,1,6,7,8,2,0,5] 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9dc0a2950 0x7ff9dc1cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): picked mon.noname-g con 0x7ff9dc0a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc1cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): picked mon.noname-c con 0x7ff9dc0a3270 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9dc138040 0x7ff9dc1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): picked mon.noname-f con 0x7ff9dc138040 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc124f50 con 0x7ff9dc138040 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc09dd70 con 0x7ff9dc0a2950 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc09d960 con 0x7ff9dc0a3270 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 10 monclient(hunting): authenticate will time out at 1650.614142s 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc1cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc1cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48542/0 (socket says 172.21.15.47:48542) 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9dc0a2950 0x7ff9dc1cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 -- 172.21.15.47:0/660979912 learned_addr learned my addr 172.21.15.47:0/660979912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): get_auth_request con 0x7ff9dc0a3270 auth_method 0 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): get_auth_request con 0x7ff9dc0a2950 auth_method 0 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9dc138040 0x7ff9dc1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): get_auth_request con 0x7ff9dc138040 auth_method 0 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e1be3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient(hunting): handle_auth_done global_id 4540 payload 824 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9dc0a2950 msgr2=0x7ff9dc1cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9dc0a2950 0x7ff9dc1cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9dc138040 msgr2=0x7ff9dc1ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9dc138040 0x7ff9dc1ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 monclient: found mon.noname-c 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8009550 con 0x7ff9dc0a3270 2024-04-20T02:16:11.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e13e2700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc1cc810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00b5a0 tx=0x7ff9cc00b8b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 client.?.objecter ms_handle_connect 0x7ff9dc0a3270 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9cc003c40 con 0x7ff9dc0a3270 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 monclient: dump: 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.232 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc003de0 con 0x7ff9dc0a3270 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.213+0000 7ff9e3e47700 5 monclient: authenticate success, global_id 4540 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9cc011400 con 0x7ff9dc0a3270 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d27fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:16:11.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d3fff700 4 set_mon_vals no callback set 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d27fc700 10 monclient: dump: 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.234 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 10 monclient: _renew_subs 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc123a10 con 0x7ff9dc0a3270 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 10 client.4540.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 10 monclient: _renew_subs 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc124900 con 0x7ff9dc0a3270 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9e3e47700 1 librados: init done 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.214+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff9cc005070 con 0x7ff9dc0a3270 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9c003a320 0x7ff9c003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7ff9cc049780 con 0x7ff9dc0a3270 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 10 client.4540.objecter ms_dispatch 0x7ff9dc001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:11.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 3 client.4540.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 3 client.4540.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 20 client.4540.objecter dump_active .. 0 homeless 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e3e47700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e1be3700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9c003a320 0x7ff9c003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 0x7ff9dc1219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc068310 con 0x7ff9dc06aac0 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e1be3700 10 monclient: get_auth_request con 0x7ff9c003a320 auth_method 0 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 0x7ff9dc1219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient: get_auth_request con 0x7ff9dc06aac0 auth_method 0 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_done global_id 4425 payload 824 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9dc067730 con 0x7ff9dc06aac0 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 10 client.4540.objecter ms_handle_connect 0x7ff9c003a320 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e1be3700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9c003a320 0x7ff9c003c7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009920 tx=0x7ff9d8008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 10 client.4540.objecter ms_handle_connect 0x7ff9dc06aac0 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9d27fc700 10 client.4540.objecter resend_mon_ops 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.215+0000 7ff9e23e4700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 0x7ff9dc1219a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4003d90 tx=0x7ff9d400bf30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9d4010040 con 0x7ff9dc06aac0 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 10 monclient: dump: 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.237 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff9d4006850 con 0x7ff9dc06aac0 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 msgr2=0x7ff9dc1219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 0x7ff9dc1219a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4003d90 tx=0x7ff9d400bf30 comp rx=0 tx=0).stop 2024-04-20T02:16:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 msgr2=0x7ff9dc1219a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.216+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc06aac0 0x7ff9dc1219a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e3e47700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 0x7ff9dc1222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e3e47700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc06a400 con 0x7ff9dc121ee0 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 0x7ff9dc1222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient: get_auth_request con 0x7ff9dc121ee0 auth_method 0 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 10 monclient(hunting): handle_auth_done global_id 4434 payload 824 2024-04-20T02:16:11.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 1 -- 172.21.15.47:0/660979912 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff9dc124db0 con 0x7ff9dc121ee0 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9d27fc700 10 client.4540.objecter ms_handle_connect 0x7ff9dc121ee0 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9d27fc700 10 client.4540.objecter resend_mon_ops 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9e23e4700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 0x7ff9dc1222f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d010 tx=0x7ff9d4003c70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9d4022070 con 0x7ff9dc121ee0 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9d27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.224+0000 7ff9d27fc700 10 monclient: dump: 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.242 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9320 (secure 0 0 0) 0x7ff9d401de20 con 0x7ff9dc121ee0 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 msgr2=0x7ff9dc1222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 0x7ff9dc1222f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d010 tx=0x7ff9d4003c70 comp rx=0 tx=0).stop 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 msgr2=0x7ff9dc1222f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9d27fc700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9dc121ee0 0x7ff9dc1222f0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400d010 tx=0x7ff9d4003c70 comp rx=0 tx=0).stop 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 12, 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:16:11.244 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: 7 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.245 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.246 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:16:11.247 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.248 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.250 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:11.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.256 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.262 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:16:11.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:16:11.264 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:16:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 10 librados: watch_flush enter 2024-04-20T02:16:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 10 librados: watch_flush exit 2024-04-20T02:16:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 20 client.4540.objecter shutdown clearing up homeless session... 2024-04-20T02:16:11.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 10 client.4540.objecter successfully canceled tick 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9c003a320 msgr2=0x7ff9c003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9c003a320 0x7ff9c003c7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009920 tx=0x7ff9d8008030 comp rx=0 tx=0).stop 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 10 monclient: shutdown 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 msgr2=0x7ff9dc1cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e23e4700 1 -- 172.21.15.47:0/660979912 reap_dead start 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc1cc810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00b5a0 tx=0x7ff9cc00b8b0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 shutdown_connections 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 --2- 172.21.15.47:0/660979912 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff9dc0a3270 0x7ff9dc1cc810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 >> 172.21.15.47:0/660979912 conn(0x7ff9dc0a7570 msgr2=0x7ff9dc0a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 shutdown_connections 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 -- 172.21.15.47:0/660979912 wait complete. 2024-04-20T02:16:11.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.225+0000 7ff9e3e47700 1 librados: shutdown 2024-04-20T02:16:11.268 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:16:11.268 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-04-20T02:16:11.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:16:11.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.408+0000 7fc751fa7700 10 monclient: get_monmap_and_config 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.408+0000 7fc751fa7700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.408+0000 7fc751fa7700 10 monclient: monmap: 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:11.409561+0000 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:11.409561+0000 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.408 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:11.409 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:11.409 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 Processor -- start 2024-04-20T02:16:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 -- start start 2024-04-20T02:16:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient: init 2024-04-20T02:16:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient: _add_conns ranks=[7,4,1,8,10,2,0,6,3,5,9] 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c071610 0x7fc74c0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): picked mon.noname-i con 0x7fc74c071610 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 0x7fc74c0a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): picked mon.noname-c con 0x7fc74c071f30 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc74c0a2e90 addr v1:172.21.15.179:6789/0 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74c0751e0 con 0x7fc74c0a2e90 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0727b0 con 0x7fc74c071f30 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0a0880 con 0x7fc74c071610 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc751fa7700 10 monclient(hunting): authenticate will time out at 1650.810821s 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.409+0000 7fc74affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 0x7fc74c0a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c071610 0x7fc74c0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 0x7fc74c0a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48554/0 (socket says 172.21.15.47:48554) 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc74c0a2e90 0x7fc74c13b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:36428/0 (socket says 172.21.15.47:36428) 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 -- 172.21.15.47:0/3899797931 learned_addr learned my addr 172.21.15.47:0/3899797931 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): get_auth_request con 0x7fc74c071f30 auth_method 0 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): get_auth_request con 0x7fc74c071610 auth_method 0 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient(hunting): handle_auth_done global_id 4549 payload 274 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 -- 172.21.15.47:0/3899797931 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c071610 msgr2=0x7fc74c0719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 1 -- 172.21.15.47:0/3899797931 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300168263 0 0) 0x7fc74c0751e0 con 0x7fc74c0a2e90 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 --2- 172.21.15.47:0/3899797931 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c071610 0x7fc74c0719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 -- 172.21.15.47:0/3899797931 >> v1:172.21.15.179:6789/0 conn(0x7fc74c0a2e90 legacy=0x7fc74c13b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 monclient: found mon.noname-c 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 -- 172.21.15.47:0/3899797931 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c09e8f0 con 0x7fc74c071f30 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74affd700 1 --2- 172.21.15.47:0/3899797931 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 0x7fc74c0a2950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc740009920 tx=0x7fc740009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7eed4f3520231068 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 1 -- 172.21.15.47:0/3899797931 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc740011aa0 con 0x7fc74c071f30 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: dump: 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 1 -- 172.21.15.47:0/3899797931 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc740011c00 con 0x7fc74c071f30 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc751fa7700 5 monclient: authenticate success, global_id 4549 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc751fa7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc751fa7700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 1 -- 172.21.15.47:0/3899797931 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc740011d70 con 0x7fc74c071f30 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc751fa7700 4 set_mon_vals no callback set 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.410+0000 7fc74a7fc700 10 monclient: dump: 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 10 monclient: shutdown 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 -- 172.21.15.47:0/3899797931 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 msgr2=0x7fc74c0a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 --2- 172.21.15.47:0/3899797931 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 0x7fc74c0a2950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc740009920 tx=0x7fc740009c30 comp rx=0 tx=0).stop 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 -- 172.21.15.47:0/3899797931 shutdown_connections 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 --2- 172.21.15.47:0/3899797931 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c071610 0x7fc74c0719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 --2- 172.21.15.47:0/3899797931 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc74c071f30 0x7fc74c0a2950 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 -- 172.21.15.47:0/3899797931 >> 172.21.15.47:0/3899797931 conn(0x7fc74c0a7570 msgr2=0x7fc74c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 -- 172.21.15.47:0/3899797931 shutdown_connections 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.411+0000 7fc751fa7700 1 -- 172.21.15.47:0/3899797931 wait complete. 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 10 monclient: monmap: 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 1 librados: starting msgr at 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 1 librados: starting objecter 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 1 Processor -- start 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 1 -- start start 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 1 librados: setting wanted keys 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 1 librados: calling monclient init 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.412+0000 7fc751fa7700 10 monclient: init 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient: _add_conns ranks=[5,8,7,3,4,1,0,2,6] 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc74c071610 0x7fc74c136350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): picked mon.noname-h con 0x7fc74c071610 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc74c071f30 0x7fc74c136890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): picked mon.noname-e con 0x7fc74c071f30 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c0a2e90 0x7fc74c13aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): picked mon.noname-i con 0x7fc74c0a2e90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c13c610 con 0x7fc74c071610 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c09f260 con 0x7fc74c0a2e90 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0723f0 con 0x7fc74c071f30 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc751fa7700 10 monclient(hunting): authenticate will time out at 1650.814361s 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74b7fe700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc74c071610 msgr2=0x7fc74c136350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74affd700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc74c071f30 msgr2=0x7fc74c136890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74b7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc74c071610 0x7fc74c136350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74affd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc74c071f30 0x7fc74c136890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c0a2e90 0x7fc74c13aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c0a2e90 0x7fc74c13aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:37248/0 (socket says 172.21.15.47:37248) 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 -- 172.21.15.47:0/2274386636 learned_addr learned my addr 172.21.15.47:0/2274386636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): get_auth_request con 0x7fc74c0a2e90 auth_method 0 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient(hunting): handle_auth_done global_id 4516 payload 824 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc74c071f30 msgr2=0x7fc74c136890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc74c071f30 0x7fc74c136890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc74c071610 msgr2=0x7fc74c136350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc74c071610 0x7fc74c136350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 monclient: found mon.noname-i 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.413+0000 7fc74bfff700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc740009550 con 0x7fc74c0a2e90 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc74bfff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c0a2e90 0x7fc74c13aad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc74400cca0 tx=0x7fc74400a340 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 client.?.objecter ms_handle_connect 0x7fc74c0a2e90 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc74400cea0 con 0x7fc74c0a2e90 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: dump: 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74400abe0 con 0x7fc74c0a2e90 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 5 monclient: authenticate success, global_id 4516 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 10 monclient: _renew_subs 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c11ee90 con 0x7fc74c0a2e90 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc74400ad90 con 0x7fc74c0a2e90 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc749ffb700 4 set_mon_vals no callback set 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 10 client.4516.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc733fff700 10 monclient: dump: 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.436 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 10 monclient: _renew_subs 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c11d9d0 con 0x7fc74c0a2e90 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.414+0000 7fc751fa7700 1 librados: init done 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc751fa7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc751fa7700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 0x7fc74c13b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c068310 con 0x7fc74c06aac0 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc7440070d0 con 0x7fc74c0a2e90 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 0x7fc74c13b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient: get_auth_request con 0x7fc74c06aac0 auth_method 0 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc73803a320 0x7fc73803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7fc744049b10 con 0x7fc74c0a2e90 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 10 client.4516.objecter ms_dispatch 0x7fc74c001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 3 client.4516.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 3 client.4516.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_done global_id 4663 payload 824 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74b7fe700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc74c11e960 con 0x7fc74c06aac0 2024-04-20T02:16:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc733fff700 20 client.4516.objecter dump_active .. 0 homeless 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74affd700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc73803a320 0x7fc73803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.415+0000 7fc74affd700 10 monclient: get_auth_request con 0x7fc73803a320 auth_method 0 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 client.4516.objecter ms_handle_connect 0x7fc74c06aac0 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 client.4516.objecter resend_mon_ops 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc74b7fe700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 0x7fc74c13b010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009fd0 tx=0x7fc73c00eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc73c013af0 con 0x7fc74c06aac0 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 monclient: dump: 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.439 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc74affd700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc73803a320 0x7fc73803c7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc740009920 tx=0x7fc740008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 client.4516.objecter ms_handle_connect 0x7fc73803a320 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc73c013c50 con 0x7fc74c06aac0 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 msgr2=0x7fc74c13b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 0x7fc74c13b010 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009fd0 tx=0x7fc73c00eb70 comp rx=0 tx=0).stop 2024-04-20T02:16:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 msgr2=0x7fc74c13b010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.416+0000 7fc733fff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c06aac0 0x7fc74c13b010 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009fd0 tx=0x7fc73c00eb70 comp rx=0 tx=0).stop 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc751fa7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc751fa7700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 0x7fc74c13b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc751fa7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c06a400 con 0x7fc74c13b550 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 0x7fc74c13b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient: get_auth_request con 0x7fc74c13b550 auth_method 0 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 10 monclient(hunting): handle_auth_done global_id 4672 payload 824 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 1 -- 172.21.15.47:0/2274386636 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc74c09ed00 con 0x7fc74c13b550 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc74b7fe700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 0x7fc74c13b960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc73c018000 tx=0x7fc73c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc733fff700 10 client.4516.objecter ms_handle_connect 0x7fc74c13b550 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc733fff700 10 client.4516.objecter resend_mon_ops 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc73c022070 con 0x7fc74c13b550 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc733fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc733fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.424+0000 7fc733fff700 10 monclient: dump: 2024-04-20T02:16:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.445 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9322 (secure 0 0 0) 0x7fc73c005d40 con 0x7fc74c13b550 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 msgr2=0x7fc74c13b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 0x7fc74c13b960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc73c018000 tx=0x7fc73c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 msgr2=0x7fc74c13b960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc733fff700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc74c13b550 0x7fc74c13b960 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc73c018000 tx=0x7fc73c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:11.446 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 12, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: 7 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:16:11.447 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.448 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:16:11.449 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.450 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:16:11.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.455 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.456 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.457 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:16:11.458 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.459 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.460 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:11.462 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.463 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.464 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:16:11.465 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.466 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:16:11.467 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 10 librados: watch_flush enter 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 10 librados: watch_flush exit 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 20 client.4516.objecter shutdown clearing up homeless session... 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 10 client.4516.objecter successfully canceled tick 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc73803a320 msgr2=0x7fc73803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc73803a320 0x7fc73803c7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc740009920 tx=0x7fc740008030 comp rx=0 tx=0).stop 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 10 monclient: shutdown 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c0a2e90 msgr2=0x7fc74c13aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 --2- 172.21.15.47:0/2274386636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc74c0a2e90 0x7fc74c13aad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc74400cca0 tx=0x7fc74400a340 comp rx=0 tx=0).stop 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc74bfff700 1 -- 172.21.15.47:0/2274386636 reap_dead start 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc74bfff700 1 -- 172.21.15.47:0/2274386636 reap_dead start 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 shutdown_connections 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 >> 172.21.15.47:0/2274386636 conn(0x7fc74c0a7570 msgr2=0x7fc74c09c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 shutdown_connections 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 -- 172.21.15.47:0/2274386636 wait complete. 2024-04-20T02:16:11.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.425+0000 7fc751fa7700 1 librados: shutdown 2024-04-20T02:16:11.470 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:16:11.470 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-04-20T02:16:11.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.610+0000 7f19e5aae700 10 monclient: get_monmap_and_config 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.610+0000 7f19e5aae700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.610+0000 7f19e5aae700 10 monclient: monmap: 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:11.611920+0000 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:11.611920+0000 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:11.611 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.611+0000 7f19e5aae700 1 Processor -- start 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.611+0000 7f19e5aae700 1 -- start start 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.611+0000 7f19e5aae700 10 monclient: init 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient: _add_conns ranks=[0,7,8,1,10,2,4,6,5,3,9] 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f19e0071630 addr v1:172.21.15.47:6789/0 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): picked mon.noname-i con 0x7f19e0137930 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:11.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f19e013a260 0x7f19e013c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): picked mon.noname-e con 0x7f19e013a260 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e00751e0 con 0x7f19e0071630 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0072720 con 0x7f19e0137930 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19df7fe700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f19e0071630 legacy=0x7f19e0071a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00a0820 con 0x7f19e013a260 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19e5aae700 10 monclient(hunting): authenticate will time out at 1651.013432s 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19dffff700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f19e013a260 msgr2=0x7f19e013c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19dffff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f19e013a260 0x7f19e013c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0139d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:37252/0 (socket says 172.21.15.47:37252) 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 -- 172.21.15.47:0/2518958677 learned_addr learned my addr 172.21.15.47:0/2518958677 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): get_auth_request con 0x7f19e0137930 auth_method 0 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient(hunting): handle_auth_done global_id 4525 payload 274 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 -- 172.21.15.47:0/2518958677 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f19e013a260 msgr2=0x7f19e013c650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 --2- 172.21.15.47:0/2518958677 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f19e013a260 0x7f19e013c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 -- 172.21.15.47:0/2518958677 >> v1:172.21.15.47:6789/0 conn(0x7f19e0071630 legacy=0x7f19e0071a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 monclient: found mon.noname-i 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.612+0000 7f19deffd700 1 -- 172.21.15.47:0/2518958677 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e009e890 con 0x7f19e0137930 2024-04-20T02:16:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19deffd700 1 --2- 172.21.15.47:0/2518958677 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0139d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f19c800bbc0 tx=0x7f19c800bed0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=d42d6a76129e923d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 1 -- 172.21.15.47:0/2518958677 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19c8004030 con 0x7f19e0137930 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: dump: 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19e5aae700 5 monclient: authenticate success, global_id 4525 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 1 -- 172.21.15.47:0/2518958677 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c8004190 con 0x7f19e0137930 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19e5aae700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 1 -- 172.21.15.47:0/2518958677 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19c8004300 con 0x7f19e0137930 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19e5aae700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:11.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19e5aae700 4 set_mon_vals no callback set 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.613+0000 7f19de7fc700 10 monclient: dump: 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.628 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 10 monclient: shutdown 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 -- 172.21.15.47:0/2518958677 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 msgr2=0x7f19e0139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 --2- 172.21.15.47:0/2518958677 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0139d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f19c800bbc0 tx=0x7f19c800bed0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 -- 172.21.15.47:0/2518958677 shutdown_connections 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 --2- 172.21.15.47:0/2518958677 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f19e013a260 0x7f19e013c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 --2- 172.21.15.47:0/2518958677 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0139d20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 -- 172.21.15.47:0/2518958677 >> 172.21.15.47:0/2518958677 conn(0x7f19e00a7570 msgr2=0x7f19e00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 -- 172.21.15.47:0/2518958677 shutdown_connections 2024-04-20T02:16:11.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 1 -- 172.21.15.47:0/2518958677 wait complete. 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.614+0000 7f19e5aae700 10 monclient: monmap: 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.630 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 librados: starting msgr at 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 librados: starting objecter 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 Processor -- start 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 -- start start 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 librados: setting wanted keys 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 librados: calling monclient init 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient: init 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient: _add_conns ranks=[4,7,1,8,0,6,2,5,3] 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f19e0071630 0x7f19e009a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): picked mon.noname-c con 0x7f19e0071630 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:11.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): picked mon.noname-i con 0x7f19e0137930 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19e013a260 0x7f19e0135880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): picked mon.noname-f con 0x7f19e013a260 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e009b380 con 0x7f19e013a260 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19e009c600 con 0x7f19e0071630 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0137610 con 0x7f19e0137930 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.615+0000 7f19e5aae700 10 monclient(hunting): authenticate will time out at 1651.016971s 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f19e0071630 0x7f19e009a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f19e0071630 0x7f19e009a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48566/0 (socket says 172.21.15.47:48566) 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 1 -- 172.21.15.47:0/851433890 learned_addr learned my addr 172.21.15.47:0/851433890 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): get_auth_request con 0x7f19e0071630 auth_method 0 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): get_auth_request con 0x7f19e0137930 auth_method 0 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19e013a260 0x7f19e0135880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): get_auth_request con 0x7f19e013a260 auth_method 0 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19dffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient(hunting): handle_auth_done global_id 4534 payload 824 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f19e0071630 msgr2=0x7f19e009a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f19e0071630 0x7f19e009a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19e013a260 msgr2=0x7f19e0135880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19e013a260 0x7f19e0135880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 monclient: found mon.noname-i 2024-04-20T02:16:11.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c800b7f0 con 0x7f19e0137930 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19df7fe700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f19e0071630 0x7f19e009a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19deffd700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0135340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19e00a7f00 tx=0x7f19c8004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 client.?.objecter ms_handle_connect 0x7f19e0137930 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19c8020070 con 0x7f19e0137930 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: dump: 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.635 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c800de50 con 0x7f19e0137930 2024-04-20T02:16:11.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19c8010d50 con 0x7f19e0137930 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19e5aae700 5 monclient: authenticate success, global_id 4534 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19e5aae700 10 monclient: _renew_subs 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19e5aae700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e009f200 con 0x7f19e0137930 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19ddffb700 4 set_mon_vals no callback set 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19d7fff700 10 monclient: dump: 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.637 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.616+0000 7f19e5aae700 10 client.4534.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 10 monclient: _renew_subs 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0072380 con 0x7f19e0137930 2024-04-20T02:16:11.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 1 librados: init done 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 0x7f19e0135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0068310 con 0x7f19e006aac0 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f19c8003c60 con 0x7f19e0137930 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19df7fe700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 0x7f19e0135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19df7fe700 10 monclient: get_auth_request con 0x7f19e006aac0 auth_method 0 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19df7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19df7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.617+0000 7f19df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f19c003a2f0 0x7f19c003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f19c804da10 con 0x7f19e0137930 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 client.4534.objecter ms_dispatch 0x7f19e0001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 3 client.4534.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 3 client.4534.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 20 client.4534.objecter dump_active .. 0 homeless 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19df7fe700 10 monclient(hunting): handle_auth_done global_id 4431 payload 824 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19dffff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f19c003a2f0 0x7f19c003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19df7fe700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19e006f090 con 0x7f19e006aac0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19dffff700 10 monclient: get_auth_request con 0x7f19c003a2f0 auth_method 0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 client.4534.objecter ms_handle_connect 0x7f19e006aac0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 client.4534.objecter resend_mon_ops 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19df7fe700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 0x7f19e0135dc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005620 tx=0x7f19d00059e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19d000ed80 con 0x7f19e006aac0 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:16:11.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 monclient: dump: 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.641 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19d7fff700 10 client.4534.objecter ms_handle_connect 0x7f19c003a2f0 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.618+0000 7f19dffff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f19c003a2f0 0x7f19c003c7b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f19e0133ad0 tx=0x7f19d800d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f19d000e910 con 0x7f19e006aac0 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 msgr2=0x7f19e0135dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 0x7f19e0135dc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005620 tx=0x7f19d00059e0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 msgr2=0x7f19e0135dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.619+0000 7f19d7fff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e006aac0 0x7f19e0135dc0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005620 tx=0x7f19d00059e0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.626+0000 7f19e5aae700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:16:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.626+0000 7f19e5aae700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 0x7f19e01306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.626+0000 7f19e5aae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.626+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00a1190 con 0x7f19e009baa0 2024-04-20T02:16:11.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 0x7f19e01306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient: get_auth_request con 0x7f19e009baa0 auth_method 0 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 10 monclient(hunting): handle_auth_done global_id 4440 payload 824 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 1 -- 172.21.15.47:0/851433890 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f19e0072120 con 0x7f19e009baa0 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19df7fe700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 0x7f19e01306c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19d0018000 tx=0x7f19d0005a60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19d7fff700 10 client.4534.objecter ms_handle_connect 0x7f19e009baa0 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19d7fff700 10 client.4534.objecter resend_mon_ops 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19d000ebc0 con 0x7f19e009baa0 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19d7fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.627+0000 7f19d7fff700 10 monclient: dump: 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.646 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9320 (secure 0 0 0) 0x7f19d0012600 con 0x7f19e009baa0 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 msgr2=0x7f19e01306c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 0x7f19e01306c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19d0018000 tx=0x7f19d0005a60 comp rx=0 tx=0).stop 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 msgr2=0x7f19e01306c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19d7fff700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f19e009baa0 0x7f19e01306c0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19d0018000 tx=0x7f19d0005a60 comp rx=0 tx=0).stop 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:16:11.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 12, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: 7 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:16:11.648 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.649 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.650 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:16:11.651 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:11.654 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.657 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.661 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.663 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:16:11.664 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.666 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:16:11.667 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:16:11.668 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:16:11.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 10 librados: watch_flush enter 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 10 librados: watch_flush exit 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 20 client.4534.objecter shutdown clearing up homeless session... 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 10 client.4534.objecter successfully canceled tick 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f19c003a2f0 msgr2=0x7f19c003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f19c003a2f0 0x7f19c003c7b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f19e0133ad0 tx=0x7f19d800d040 comp rx=0 tx=0).stop 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 10 monclient: shutdown 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 msgr2=0x7f19e0135340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 --2- 172.21.15.47:0/851433890 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f19e0137930 0x7f19e0135340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19e00a7f00 tx=0x7f19c8004750 comp rx=0 tx=0).stop 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19dffff700 1 -- 172.21.15.47:0/851433890 reap_dead start 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19dffff700 1 -- 172.21.15.47:0/851433890 reap_dead start 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 shutdown_connections 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 >> 172.21.15.47:0/851433890 conn(0x7f19e00a7570 msgr2=0x7f19e00a66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 shutdown_connections 2024-04-20T02:16:11.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 -- 172.21.15.47:0/851433890 wait complete. 2024-04-20T02:16:11.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.628+0000 7f19e5aae700 1 librados: shutdown 2024-04-20T02:16:11.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:16:11.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-04-20T02:16:11.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.811+0000 7efeb79ed700 10 monclient: get_monmap_and_config 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.811+0000 7efeb79ed700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.811+0000 7efeb79ed700 10 monclient: monmap: 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:16:11.813060+0000 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:16:11.813060+0000 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.811 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:16:11.812 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.812+0000 7efeb79ed700 1 Processor -- start 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.812+0000 7efeb79ed700 1 -- start start 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.812+0000 7efeb79ed700 10 monclient: init 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient: _add_conns ranks=[3,6,10,4,9,5,7,0,2,1,8] 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeb0071630 0x7efeb0071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): picked mon.noname-g con 0x7efeb0071630 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 0x7efeb00a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): picked mon.noname-d con 0x7efeb0071f50 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efeb00a2e90 0x7efeb00a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): picked mon.noname-f con 0x7efeb00a2e90 addr v2:172.21.15.179:3300/0 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb0072720 con 0x7efeb00a2e90 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00a0820 con 0x7efeb0071630 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efeb009e890 con 0x7efeb0071f50 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb79ed700 10 monclient(hunting): authenticate will time out at 1651.214402s 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 0x7efeb00a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 0x7efeb00a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:42972/0 (socket says 172.21.15.47:42972) 2024-04-20T02:16:11.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 -- 172.21.15.47:0/3342762066 learned_addr learned my addr 172.21.15.47:0/3342762066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): get_auth_request con 0x7efeb0071f50 auth_method 0 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5f8a700 1 --2- 172.21.15.47:0/3342762066 >> v2:172.21.15.179:3300/0 conn(0x7efeb00a2e90 0x7efeb00a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5789700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeb0071630 0x7efeb0071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5f8a700 10 monclient(hunting): get_auth_request con 0x7efeb00a2e90 auth_method 0 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5f8a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5f8a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5f8a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5789700 10 monclient(hunting): get_auth_request con 0x7efeb0071630 auth_method 0 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5789700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5789700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5789700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient(hunting): handle_auth_done global_id 4443 payload 274 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 -- 172.21.15.47:0/3342762066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeb0071630 msgr2=0x7efeb0071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeb0071630 0x7efeb0071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 -- 172.21.15.47:0/3342762066 >> v2:172.21.15.179:3300/0 conn(0x7efeb00a2e90 msgr2=0x7efeb00a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 --2- 172.21.15.47:0/3342762066 >> v2:172.21.15.179:3300/0 conn(0x7efeb00a2e90 0x7efeb00a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 monclient: found mon.noname-d 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb4f88700 1 -- 172.21.15.47:0/3342762066 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00751e0 con 0x7efeb0071f50 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5f8a700 1 --2- 172.21.15.47:0/3342762066 >> v2:172.21.15.179:3300/0 conn(0x7efeb00a2e90 0x7efeb00a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.813+0000 7efeb5789700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeb0071630 0x7efeb0071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb4f88700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 0x7efeb00a2950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efea00098f0 tx=0x7efea0009c00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=df8be0fb329330fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 1 -- 172.21.15.47:0/3342762066 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efea000bb10 con 0x7efeb0071f50 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: dump: 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.827 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 5 monclient: authenticate success, global_id 4443 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 1 -- 172.21.15.47:0/3342762066 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea000bc70 con 0x7efeb0071f50 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 10 monclient: get_monmap_and_config success 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 4 set_mon_vals no callback set 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 1 -- 172.21.15.47:0/3342762066 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efea000bde0 con 0x7efeb0071f50 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efea7fff700 10 monclient: dump: 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.829 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 10 monclient: shutdown 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 -- 172.21.15.47:0/3342762066 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 msgr2=0x7efeb00a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 0x7efeb00a2950 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efea00098f0 tx=0x7efea0009c00 comp rx=0 tx=0).stop 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 -- 172.21.15.47:0/3342762066 shutdown_connections 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 --2- 172.21.15.47:0/3342762066 >> v2:172.21.15.179:3300/0 conn(0x7efeb00a2e90 0x7efeb00a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efeb0071630 0x7efeb0071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 --2- 172.21.15.47:0/3342762066 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efeb0071f50 0x7efeb00a2950 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.814+0000 7efeb79ed700 1 -- 172.21.15.47:0/3342762066 >> 172.21.15.47:0/3342762066 conn(0x7efeb00a7570 msgr2=0x7efeb00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.815+0000 7efeb79ed700 1 -- 172.21.15.47:0/3342762066 shutdown_connections 2024-04-20T02:16:11.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.815+0000 7efeb79ed700 1 -- 172.21.15.47:0/3342762066 wait complete. 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.815+0000 7efeb79ed700 10 monclient: build_initial_monmap 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.815+0000 7efeb79ed700 10 monclient: monmap: 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:11.831 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 1 librados: starting msgr at 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 1 librados: starting objecter 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 1 Processor -- start 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 1 -- start start 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 1 librados: setting wanted keys 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 1 librados: calling monclient init 2024-04-20T02:16:11.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.816+0000 7efeb79ed700 10 monclient: init 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient: _reopen_session rank -1 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient: _add_conns ranks=[5,7,8,6,3,1,2,4,0] 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efeb0071630 0x7efeb01b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): picked mon.noname-h con 0x7efeb0071630 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb0071f50 0x7efeb01b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): picked mon.noname-i con 0x7efeb0071f50 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7efeb00a2e90 0x7efeb01c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): picked mon.noname-e con 0x7efeb00a2e90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01a7a20 con 0x7efeb0071630 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efeb009f270 con 0x7efeb0071f50 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0067a70 con 0x7efeb00a2e90 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): _renew_subs 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb79ed700 10 monclient(hunting): authenticate will time out at 1651.218333s 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb5789700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efeb0071630 msgr2=0x7efeb01b75f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb5789700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efeb0071630 0x7efeb01b75f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb5f8a700 1 -- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7efeb00a2e90 msgr2=0x7efeb01c9e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3304/0 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb5f8a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7efeb00a2e90 0x7efeb01c9e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb0071f50 0x7efeb01b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb0071f50 0x7efeb01b7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:37274/0 (socket says 172.21.15.47:37274) 2024-04-20T02:16:11.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 -- 172.21.15.47:0/1266170071 learned_addr learned my addr 172.21.15.47:0/1266170071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): get_auth_request con 0x7efeb0071f50 auth_method 0 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient(hunting): handle_auth_done global_id 4543 payload 824 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7efeb00a2e90 msgr2=0x7efeb01c9e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7efeb00a2e90 0x7efeb01c9e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efeb0071630 msgr2=0x7efeb01b75f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efeb0071630 0x7efeb01b75f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient: _finish_hunting 0 2024-04-20T02:16:11.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 monclient: found mon.noname-i 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.817+0000 7efeb4f88700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0009550 con 0x7efeb0071f50 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb4f88700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb0071f50 0x7efeb01b7b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efea000b0e0 tx=0x7efea0004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 client.?.objecter ms_handle_connect 0x7efeb0071f50 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efea0004410 con 0x7efeb0071f50 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:16:11.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: dump: 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.837 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0004590 con 0x7efeb0071f50 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 5 monclient: authenticate success, global_id 4543 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 10 monclient: _renew_subs 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb01a7740 con 0x7efeb0071f50 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efea001b430 con 0x7efeb0071f50 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea77fe700 4 set_mon_vals no callback set 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 10 client.4543.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efea5ffb700 10 monclient: dump: 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.839 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 10 monclient: _renew_subs 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01b8070 con 0x7efeb0071f50 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.818+0000 7efeb79ed700 1 librados: init done 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb79ed700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb79ed700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 0x7efeb01ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0068310 con 0x7efeb006aac0 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7efea0021020 con 0x7efeb0071f50 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 0x7efeb01ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient: get_auth_request con 0x7efeb006aac0 auth_method 0 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe9403a2f0 0x7efe9403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7efea004af00 con 0x7efeb0071f50 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 10 client.4543.objecter ms_dispatch 0x7efeb0001550 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 3 client.4543.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 3 client.4543.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:11.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efea5ffb700 20 client.4543.objecter dump_active .. 0 homeless 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 10 monclient(hunting): handle_auth_done global_id 4552 payload 824 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5789700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efeb01b8200 con 0x7efeb006aac0 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5f8a700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe9403a2f0 0x7efe9403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.819+0000 7efeb5f8a700 10 monclient: get_auth_request con 0x7efe9403a2f0 auth_method 0 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 client.4543.objecter ms_handle_connect 0x7efeb006aac0 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 client.4543.objecter resend_mon_ops 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efeb5789700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 0x7efeb01ca3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e740 tx=0x7efeac00eb00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efeac00c070 con 0x7efeb006aac0 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 monclient: dump: 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.842 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 client.4543.objecter ms_handle_connect 0x7efe9403a2f0 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efeb5f8a700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe9403a2f0 0x7efe9403c7b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efeb01ba070 tx=0x7efea8009430 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efeac004030 con 0x7efeb006aac0 2024-04-20T02:16:11.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:16:11.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:16:11.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 msgr2=0x7efeb01ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 0x7efeb01ca3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e740 tx=0x7efeac00eb00 comp rx=0 tx=0).stop 2024-04-20T02:16:11.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 msgr2=0x7efeb01ca3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.820+0000 7efea5ffb700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb006aac0 0x7efeb01ca3b0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e740 tx=0x7efeac00eb00 comp rx=0 tx=0).stop 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb79ed700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb79ed700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 0x7efeb01ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb79ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00a1190 con 0x7efeb01a56d0 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb5789700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 0x7efeb01ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb5789700 10 monclient: get_auth_request con 0x7efeb01a56d0 auth_method 0 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb5789700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb5789700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.828+0000 7efeb5789700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efeb5789700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:11.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efeb5789700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efeb5789700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efeb5789700 10 monclient(hunting): handle_auth_done global_id 4561 payload 824 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efeb5789700 1 -- 172.21.15.47:0/1266170071 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efeb00723e0 con 0x7efeb01a56d0 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efeb5789700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 0x7efeb01ca8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e740 tx=0x7efeac0047c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efea5ffb700 10 client.4543.objecter ms_handle_connect 0x7efeb01a56d0 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efea5ffb700 10 client.4543.objecter resend_mon_ops 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efeac003e90 con 0x7efeb01a56d0 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efea5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efea5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.829+0000 7efea5ffb700 10 monclient: dump: 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:11.847 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9150 (secure 0 0 0) 0x7efeac003a20 con 0x7efeb01a56d0 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 msgr2=0x7efeb01ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 0x7efeb01ca8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e740 tx=0x7efeac0047c0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 msgr2=0x7efeb01ca8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efea5ffb700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb01a56d0 0x7efeb01ca8f0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e740 tx=0x7efeac0047c0 comp rx=0 tx=0).stop 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 12, 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:16:11.849 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: 7 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.850 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.851 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:16:11.852 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:16:11.853 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:16:11.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.858 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:16:11.861 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:16:11.862 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.865 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:16:11.866 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.867 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:16:11.868 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:16:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 10 librados: watch_flush enter 2024-04-20T02:16:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 10 librados: watch_flush exit 2024-04-20T02:16:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 20 client.4543.objecter shutdown clearing up homeless session... 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 10 client.4543.objecter successfully canceled tick 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe9403a2f0 msgr2=0x7efe9403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe9403a2f0 0x7efe9403c7b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efeb01ba070 tx=0x7efea8009430 comp rx=0 tx=0).stop 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 10 monclient: shutdown 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb0071f50 msgr2=0x7efeb01b7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 --2- 172.21.15.47:0/1266170071 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efeb0071f50 0x7efeb01b7b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efea000b0e0 tx=0x7efea0004750 comp rx=0 tx=0).stop 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb5f8a700 1 -- 172.21.15.47:0/1266170071 reap_dead start 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb5f8a700 1 -- 172.21.15.47:0/1266170071 reap_dead start 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 shutdown_connections 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 >> 172.21.15.47:0/1266170071 conn(0x7efeb00a7570 msgr2=0x7efeb01bb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 shutdown_connections 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.830+0000 7efeb79ed700 1 -- 172.21.15.47:0/1266170071 wait complete. 2024-04-20T02:16:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:16:11.831+0000 7efeb79ed700 1 librados: shutdown 2024-04-20T02:16:11.872 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:16:11.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:16:41.242 INFO:tasks.workunit.client.0.smithi179.stderr:Note: switching to '15eecde2bab9803889d929c7e423d07970d8cbed'. 2024-04-20T02:16:41.242 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.242 INFO:tasks.workunit.client.0.smithi179.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T02:16:41.242 INFO:tasks.workunit.client.0.smithi179.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: git switch -c 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr:Or undo this operation with: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: git switch - 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr: 2024-04-20T02:16:41.243 INFO:tasks.workunit.client.0.smithi179.stderr:HEAD is now at 15eecde2bab Merge branch 'wip-62819-reef' of https://github.com/k0ste/ceph into wip-yuri7-testing-2024-04-18-1351-reef 2024-04-20T02:16:41.247 DEBUG:teuthology.orchestra.run.smithi179:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T02:16:41.270 INFO:tasks.workunit.client.0.smithi179.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T02:16:41.272 INFO:tasks.workunit.client.0.smithi179.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T02:16:41.272 INFO:tasks.workunit.client.0.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T02:16:41.760 INFO:tasks.workunit.client.0.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T02:16:41.815 INFO:tasks.workunit.client.0.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T02:16:41.852 INFO:tasks.workunit.client.0.smithi179.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T02:16:41.854 INFO:tasks.workunit.client.0.smithi179.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T02:16:41.854 INFO:tasks.workunit.client.0.smithi179.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T02:16:41.890 INFO:tasks.workunit.client.0.smithi179.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T02:16:41.893 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-20T02:16:41.893 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T02:16:41.951 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-04-20T02:16:41.952 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-04-20T02:16:41.953 DEBUG:teuthology.orchestra.run.smithi179:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15eecde2bab9803889d929c7e423d07970d8cbed TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ parallel=1 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' '' = --serial ']' 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ color= 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' -t 1 ']' 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ declare -A pids 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.014 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.015 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_aio 2024-04-20T02:16:42.015 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_aio' 2024-04-20T02:16:42.015 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_aio 2024-04-20T02:16:42.015 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.015 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_aio 2024-04-20T02:16:42.015 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46798 2024-04-20T02:16:42.016 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_aio on pid 46798' 2024-04-20T02:16:42.016 INFO:tasks.workunit.client.0.smithi179.stdout:test api_aio on pid 46798 2024-04-20T02:16:42.017 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46798 2024-04-20T02:16:42.017 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-04-20T02:16:42.017 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.017 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.017 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_aio_pp 2024-04-20T02:16:42.017 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_aio_pp' 2024-04-20T02:16:42.018 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_aio_pp 2024-04-20T02:16:42.018 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.018 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_aio 2024-04-20T02:16:42.018 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_aio.log 2024-04-20T02:16:42.018 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_aio: /' 2024-04-20T02:16:42.018 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_aio_pp 2024-04-20T02:16:42.019 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46806 2024-04-20T02:16:42.019 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_aio_pp on pid 46806' 2024-04-20T02:16:42.019 INFO:tasks.workunit.client.0.smithi179.stdout:test api_aio_pp on pid 46806 2024-04-20T02:16:42.020 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46806 2024-04-20T02:16:42.020 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-04-20T02:16:42.020 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.020 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_io 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_io' 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_io 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_aio_pp 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_aio_pp: /' 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_io 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46814 2024-04-20T02:16:42.021 INFO:tasks.workunit.client.0.smithi179.stdout:test api_io on pid 46814 2024-04-20T02:16:42.022 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_io on pid 46814' 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46814 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_io_pp 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_io_pp' 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_io_pp 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_io 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_io.log 2024-04-20T02:16:42.023 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_io: /' 2024-04-20T02:16:42.024 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_io_pp 2024-04-20T02:16:42.024 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46822 2024-04-20T02:16:42.024 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_io_pp on pid 46822' 2024-04-20T02:16:42.024 INFO:tasks.workunit.client.0.smithi179.stdout:test api_io_pp on pid 46822 2024-04-20T02:16:42.025 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46822 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_asio 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_asio' 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_asio 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_io_pp 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-04-20T02:16:42.026 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_io_pp: /' 2024-04-20T02:16:42.027 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_asio 2024-04-20T02:16:42.027 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46830 2024-04-20T02:16:42.027 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_asio on pid 46830' 2024-04-20T02:16:42.027 INFO:tasks.workunit.client.0.smithi179.stdout:test api_asio on pid 46830 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46830 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_list 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_list' 2024-04-20T02:16:42.028 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_list 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_asio 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_asio: /' 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_asio.log 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_list 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46838 2024-04-20T02:16:42.029 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_list on pid 46838' 2024-04-20T02:16:42.030 INFO:tasks.workunit.client.0.smithi179.stdout:test api_list on pid 46838 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46838 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_lock 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_lock' 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_lock 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.031 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_list 2024-04-20T02:16:42.032 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_list.log 2024-04-20T02:16:42.032 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_lock 2024-04-20T02:16:42.032 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46846 2024-04-20T02:16:42.033 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_lock on pid 46846' 2024-04-20T02:16:42.033 INFO:tasks.workunit.client.0.smithi179.stdout:test api_lock on pid 46846 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46846 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_lock_pp 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_lock_pp' 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_lock_pp 2024-04-20T02:16:42.034 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_lock 2024-04-20T02:16:42.045 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.045 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_lock: /' 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stdout:test api_lock_pp on pid 46854 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_lock_pp 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46854 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_lock_pp on pid 46854' 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46854 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.051 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.052 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_lock.log 2024-04-20T02:16:42.052 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_list: /' 2024-04-20T02:16:42.053 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-04-20T02:16:42.053 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_misc 2024-04-20T02:16:42.058 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_lock_pp 2024-04-20T02:16:42.058 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_lock_pp: /' 2024-04-20T02:16:42.058 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-04-20T02:16:42.058 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_misc' 2024-04-20T02:16:42.060 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.062 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_misc 2024-04-20T02:16:42.063 INFO:tasks.workunit.client.0.smithi179.stdout:test api_misc on pid 46889 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_misc 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46889 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_misc on pid 46889' 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46889 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.064 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-04-20T02:16:42.069 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_misc 2024-04-20T02:16:42.069 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_misc.log 2024-04-20T02:16:42.069 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_misc: /' 2024-04-20T02:16:42.069 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_misc_pp 2024-04-20T02:16:42.070 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_misc_pp' 2024-04-20T02:16:42.073 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.074 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_misc_pp 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [----------] Global test environment set-up. 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [----------] 12 tests from AsioRados 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.075+0000 7f18a2f128c0 10 monclient: get_monmap_and_config 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.075+0000 7f18a2f128c0 10 monclient: build_initial_monmap 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.076+0000 7f18a2f128c0 10 monclient: monmap: 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: epoch 0 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: last_changed 0.000000 2024-04-20T02:16:42.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: created 0.000000 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: min_mon_release 0 (unknown) 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: election_strategy: 1 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.076+0000 7f18a2f128c0 1 Processor -- start 2024-04-20T02:16:42.077 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.076+0000 7f18a2f128c0 1 -- start start 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.076+0000 7f18a2f128c0 10 monclient: init 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient: _reopen_session rank -1 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient: _add_conns ranks=[5,4,2,1,6,3,0,8,7] 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x55fad8c17820 0x55fad8c16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): picked mon.noname-h con 0x55fad8c17820 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55fad8c1ded0 0x55fad8d49300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): picked mon.noname-c con 0x55fad8c1ded0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 0x55fad8d49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): picked mon.noname-b con 0x55fad8c1e2b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fad8b3c6d0 con 0x55fad8c1e2b0 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:42.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x55fad8b3c370 con 0x55fad8c1ded0 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x55fad8b33680 con 0x55fad8c17820 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): _renew_subs 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a2f128c0 10 monclient(hunting): authenticate will time out at 1665.484343s 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0218700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x55fad8c17820 msgr2=0x55fad8c16ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0218700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x55fad8c17820 0x55fad8c16ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 0x55fad8d49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 0x55fad8d49840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.179:40224/0 (socket says 172.21.15.179:40224) 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 1 -- 172.21.15.179:0/1606406540 learned_addr learned my addr 172.21.15.179:0/1606406540 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 10 monclient(hunting): get_auth_request con 0x55fad8c1e2b0 auth_method 0 2024-04-20T02:16:42.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.077+0000 7f18a0a19700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f189fa17700 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55fad8c1ded0 0x55fad8d49300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 10 monclient(hunting): handle_auth_done global_id 4511 payload 274 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 1 -- 172.21.15.179:0/1606406540 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x55fad8c17820 msgr2=0x55fad8c16ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x55fad8c17820 0x55fad8c16ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 1 -- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55fad8c1ded0 msgr2=0x55fad8d49300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stdout: atest api_misc_pp on pid 46953 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_misc_pp 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46953 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_misc_pp on pid 46953' 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46953 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.081 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout:pi_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55fad8c1ded0 0x55fad8d49300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 10 monclient: _finish_hunting 0 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 1 monclient: found mon.noname-b 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.078+0000 7f18a0a19700 1 -- 172.21.15.179:0/1606406540 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894005de0 con 0x55fad8c1e2b0 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.079+0000 7f18a0a19700 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 0x55fad8d49840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f189400ac00 tx=0x7f189400d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5634026d890100 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 1 -- 172.21.15.179:0/1606406540 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1894008b00 con 0x55fad8c1e2b0 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:16:42.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: dump: 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: epoch 1 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: min_mon_release 18 (reef) 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: election_strategy: 3 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:42.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: _finish_auth 0 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: _check_auth_tickets 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 1 -- 172.21.15.179:0/1606406540 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894008e50 con 0x55fad8c1e2b0 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f18a2f128c0 5 monclient: authenticate success, global_id 4511 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f18a2f128c0 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:42.084 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f18a2f128c0 10 monclient: get_monmap_and_config success 2024-04-20T02:16:42.085 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_tier_pp 2024-04-20T02:16:42.085 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_tier_pp' 2024-04-20T02:16:42.085 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.085 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-04-20T02:16:42.085 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_misc_pp 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f18a2f128c0 4 set_mon_vals no callback set 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 1 -- 172.21.15.179:0/1606406540 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f189401c8b0 con 0x55fad8c1e2b0 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.081+0000 7f189ea15700 10 monclient: dump: 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: epoch 1 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.086 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: min_mon_release 18 (reef) 2024-04-20T02:16:42.087 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: election_strategy: 3 2024-04-20T02:16:42.087 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:42.087 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:42.087 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:42.087 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:42.088 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:42.088 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:42.088 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 10 monclient: shutdown 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 -- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 msgr2=0x55fad8d49840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:42.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 0x55fad8d49840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f189400ac00 tx=0x7f189400d000 comp rx=0 tx=0).stop 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 -- 172.21.15.179:0/1606406540 shutdown_connections 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55fad8c1e2b0 0x55fad8d49840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x55fad8c17820 0x55fad8c16ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/1606406540 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55fad8c1ded0 0x55fad8d49300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.082+0000 7f18a2f128c0 1 -- 172.21.15.179:0/1606406540 >> 172.21.15.179:0/1606406540 conn(0x55fad8bed520 msgr2=0x55fad8c13f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 1 -- 172.21.15.179:0/1606406540 shutdown_connections 2024-04-20T02:16:42.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 1 -- 172.21.15.179:0/1606406540 wait complete. 2024-04-20T02:16:42.091 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 10 monclient: build_initial_monmap 2024-04-20T02:16:42.091 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 10 monclient: monmap: 2024-04-20T02:16:42.091 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: epoch 0 2024-04-20T02:16:42.091 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:16:42.091 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: last_changed 0.000000 2024-04-20T02:16:42.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: created 0.000000 2024-04-20T02:16:42.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: min_mon_release 0 (unknown) 2024-04-20T02:16:42.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: election_strategy: 1 2024-04-20T02:16:42.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:16:42.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:16:42.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:16:42.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:16:42.094 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 1 librados: starting msgr at 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 1 librados: starting objecter 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.083+0000 7f18a2f128c0 1 Processor -- start 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.084+0000 7f18a2f128c0 1 -- start start 2024-04-20T02:16:42.095 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.084+0000 7f18a2f128c0 1 librados: setting wanted keys 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.084+0000 7f18a2f128c0 1 librados: calling monclient init 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.084+0000 7f18a2f128c0 10 monclient: init 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient: _reopen_session rank -1 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient: _add_conns ranks=[3,7,6,8,5,2,1,0,4] 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55fad8c17820 0x55fad8caf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): picked mon.noname-g con 0x55fad8c17820 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x55fad8c1ded0 0x55fad8caf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): picked mon.noname-i con 0x55fad8c1ded0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x55fad8c1e2b0 0x55fad8d4d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): picked mon.noname-d con 0x55fad8c1e2b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:16:42.096 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fad8bf3260 con 0x55fad8c17820 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x55fad8bf2270 con 0x55fad8c1e2b0 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): start opening mon connection 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x55fad8bf28f0 con 0x55fad8c1ded0 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): _renew_subs 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a2f128c0 10 monclient(hunting): authenticate will time out at 1665.492233s 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55fad8c17820 0x55fad8caf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x55fad8c1e2b0 0x55fad8d4d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55fad8c17820 0x55fad8caf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:47926/0 (socket says 172.21.15.179:47926) 2024-04-20T02:16:42.097 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x55fad8c1e2b0 0x55fad8d4d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.179:40086/0 (socket says 172.21.15.179:40086) 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 1 -- 172.21.15.179:0/3028036439 learned_addr learned my addr 172.21.15.179:0/3028036439 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 10 monclient(hunting): get_auth_request con 0x55fad8c17820 auth_method 0 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0218700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): get_auth_request con 0x55fad8c1e2b0 auth_method 0 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f189fa17700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x55fad8c1ded0 0x55fad8caf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:16:42.098 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f18a0a19700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f189fa17700 10 monclient(hunting): get_auth_request con 0x55fad8c1ded0 auth_method 0 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f189fa17700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f189fa17700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.085+0000 7f189fa17700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 10 monclient(hunting): handle_auth_done global_id 4461 payload 824 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 -- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x55fad8c1ded0 msgr2=0x55fad8caf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x55fad8c1ded0 0x55fad8caf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 -- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55fad8c17820 msgr2=0x55fad8caf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x55fad8c17820 0x55fad8caf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:42.099 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 10 monclient: _finish_hunting 0 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 monclient: found mon.noname-d 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fad8b3eb40 con 0x55fad8c1e2b0 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f18a0a19700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x55fad8c1e2b0 0x55fad8d4d0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55fad8bee920 tx=0x7f1894002fb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 client.?.objecter ms_handle_connect 0x55fad8c1e2b0 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 client.?.objecter resend_mon_ops 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f189401cac0 con 0x55fad8c1e2b0 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: dump: 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: epoch 1 2024-04-20T02:16:42.100 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: min_mon_release 18 (reef) 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: election_strategy: 3 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:42.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: _finish_auth 0 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: _check_auth_tickets 2024-04-20T02:16:42.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189401cc80 con 0x55fad8c1e2b0 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f189401cdf0 con 0x55fad8c1e2b0 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.086+0000 7f189da13700 10 monclient: dump: 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: epoch 1 2024-04-20T02:16:42.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: min_mon_release 18 (reef) 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: election_strategy: 3 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:16:42.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 5 monclient: authenticate success, global_id 4461 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 10 monclient: _renew_subs 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fad8b65a40 con 0x55fad8c1e2b0 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 10 client.4461.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:42.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 10 monclient: _renew_subs 2024-04-20T02:16:42.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:42.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fad8b4b8a0 con 0x55fad8c1e2b0 2024-04-20T02:16:42.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 1 librados: init done 2024-04-20T02:16:42.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.087+0000 7f18a2f128c0 10 librados: wait_for_osdmap waiting 2024-04-20T02:16:42.107 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_misc_pp: /' 2024-04-20T02:16:42.107 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_tier_pp 2024-04-20T02:16:42.107 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_tier_pp 2024-04-20T02:16:42.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da137test api_tier_pp on pid 46980 2024-04-20T02:16:42.108 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46980 2024-04-20T02:16:42.108 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_tier_pp on pid 46980' 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46980 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_pool 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_pool' 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_pool 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.109 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_tier_pp 2024-04-20T02:16:42.110 INFO:tasks.workunit.client.0.smithi179.stdout:test api_pool on pid 46995 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_pool 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46995 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_pool on pid 46995' 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=46995 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_snapshots 2024-04-20T02:16:42.111 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_snapshots' 2024-04-20T02:16:42.112 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_snapshots 2024-04-20T02:16:42.112 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.112 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_pool 2024-04-20T02:16:42.112 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_pool.log 2024-04-20T02:16:42.112 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_pool: /' 2024-04-20T02:16:42.112 INFO:tasks.workunit.client.0.smithi179.stdout:test api_snapshots on pid 47004 2024-04-20T02:16:42.113 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_snapshots 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47004 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_snapshots on pid 47004' 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47004 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_snapshots_pp 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_snapshots: /' 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_tier_pp: /' 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_snapshots 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_snapshots_pp' 2024-04-20T02:16:42.114 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_snapshots_pp 2024-04-20T02:16:42.115 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-04-20T02:16:42.115 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.115 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_snapshots_pp 2024-04-20T02:16:42.115 INFO:tasks.workunit.client.0.smithi179.stdout:test api_snapshots_pp on pid 47034 2024-04-20T02:16:42.116 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47034 2024-04-20T02:16:42.116 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_snapshots_pp on pid 47034' 2024-04-20T02:16:42.116 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47034 2024-04-20T02:16:42.116 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.116 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.116 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-04-20T02:16:42.117 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_stat 2024-04-20T02:16:42.117 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-04-20T02:16:42.117 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_stat' 2024-04-20T02:16:42.117 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_stat 2024-04-20T02:16:42.119 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.119 INFO:tasks.workunit.client.0.smithi179.stdout:test api_stat on pid 47043 2024-04-20T02:16:42.119 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_stat 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47043 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_stat on pid 47043' 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47043 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_stat_pp 2024-04-20T02:16:42.120 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-04-20T02:16:42.121 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_stat 2024-04-20T02:16:42.121 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_stat_pp' 2024-04-20T02:16:42.121 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_stat.log 2024-04-20T02:16:42.122 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_stat: /' 2024-04-20T02:16:42.125 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_snapshots_pp 2024-04-20T02:16:42.125 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_stat_pp 2024-04-20T02:16:42.125 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.128 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_stat_pp 2024-04-20T02:16:42.128 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47061 2024-04-20T02:16:42.128 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_stat_pp on pid 47061' 2024-04-20T02:16:42.128 INFO:tasks.workunit.client.0.smithi179.stdout:test api_stat_pp on pid 47061 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47061 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_watch_notify 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_watch_notify' 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_watch_notify 2024-04-20T02:16:42.129 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.130 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-04-20T02:16:42.130 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_stat_pp 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stdout:test api_watch_notify on pid 47077 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_watch_notify 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47077 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_watch_notify on pid 47077' 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47077 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.135 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.136 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-04-20T02:16:42.136 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_watch_notify: /' 2024-04-20T02:16:42.149 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_stat_pp: /' 2024-04-20T02:16:42.149 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-04-20T02:16:42.149 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_watch_notify_pp 2024-04-20T02:16:42.150 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_watch_notify 2024-04-20T02:16:42.150 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-04-20T02:16:42.151 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_watch_notify_pp' 2024-04-20T02:16:42.151 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_watch_notify_pp 2024-04-20T02:16:42.151 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.152 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_watch_notify_pp 2024-04-20T02:16:42.153 INFO:tasks.workunit.client.0.smithi179.stdout:test api_watch_notify_pp on pid 47116 2024-04-20T02:16:42.153 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47116 2024-04-20T02:16:42.153 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_watch_notify_pp on pid 47116' 2024-04-20T02:16:42.153 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47116 2024-04-20T02:16:42.154 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.154 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.154 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-04-20T02:16:42.154 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-04-20T02:16:42.155 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-04-20T02:16:42.155 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-04-20T02:16:42.159 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_cmd 2024-04-20T02:16:42.159 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_cmd' 2024-04-20T02:16:42.159 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.161 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_cmd 2024-04-20T02:16:42.162 INFO:tasks.workunit.client.0.smithi179.stdout:test api_cmd on pid 47137 2024-04-20T02:16:42.162 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_cmd 2024-04-20T02:16:42.163 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47137 2024-04-20T02:16:42.163 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_cmd on pid 47137' 2024-04-20T02:16:42.163 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47137 2024-04-20T02:16:42.163 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.163 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.171 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-04-20T02:16:42.172 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_cmd_pp 2024-04-20T02:16:42.172 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_cmd_pp' 2024-04-20T02:16:42.172 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.173 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_cmd_pp 2024-04-20T02:16:42.175 INFO:tasks.workunit.client.0.smithi179.stdout:test api_cmd_pp on pid 47173 2024-04-20T02:16:42.175 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_cmd_pp 2024-04-20T02:16:42.175 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47173 2024-04-20T02:16:42.175 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_cmd_pp on pid 47173' 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_cmd.log 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47173 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_cmd 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_cmd_pp 2024-04-20T02:16:42.176 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_service 2024-04-20T02:16:42.177 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_service' 2024-04-20T02:16:42.177 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-04-20T02:16:42.178 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.179 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_service 2024-04-20T02:16:42.180 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_service 2024-04-20T02:16:42.180 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47192 2024-04-20T02:16:42.180 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_service on pid 47192' 2024-04-20T02:16:42.181 INFO:tasks.workunit.client.0.smithi179.stdout:test api_service on pid 47192 2024-04-20T02:16:42.181 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47192 2024-04-20T02:16:42.181 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_service_pp 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_service_pp' 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_service_pp 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_service.log 2024-04-20T02:16:42.182 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_service: /' 2024-04-20T02:16:42.183 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.183 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_cmd: /' 2024-04-20T02:16:42.191 INFO:tasks.workunit.client.0.smithi179.stdout:test api_service_pp on pid 47205 2024-04-20T02:16:42.192 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_service_pp 2024-04-20T02:16:42.192 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47205 2024-04-20T02:16:42.192 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_service_pp on pid 47205' 2024-04-20T02:16:42.192 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47205 2024-04-20T02:16:42.192 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_c_write_operations 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_c_write_operations' 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_c_write_operations 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_service 2024-04-20T02:16:42.193 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-04-20T02:16:42.200 INFO:tasks.workunit.client.0.smithi179.stdout:test api_c_write_operations on pid 47232 2024-04-20T02:16:42.200 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_service_pp 2024-04-20T02:16:42.200 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_service_pp: /' 2024-04-20T02:16:42.200 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.200 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_c_write_operations 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47232 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_c_write_operations on pid 47232' 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47232 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_c_read_operations 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_c_read_operations' 2024-04-20T02:16:42.201 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_c_read_operations 2024-04-20T02:16:42.204 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_c_write_operations 2024-04-20T02:16:42.205 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-04-20T02:16:42.205 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-04-20T02:16:42.209 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stdout:test api_c_read_operations on pid 47253 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_c_read_operations 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47253 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_c_read_operations on pid 47253' 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47253 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.210 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-04-20T02:16:42.217 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_c_read_operations 2024-04-20T02:16:42.218 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s api_cls_remote_reads 2024-04-20T02:16:42.218 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' api_cls_remote_reads' 2024-04-20T02:16:42.218 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo api_cls_remote_reads 2024-04-20T02:16:42.218 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-04-20T02:16:42.219 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-04-20T02:16:42.220 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.221 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=api_cls_remote_reads 2024-04-20T02:16:42.222 INFO:tasks.workunit.client.0.smithi179.stdout:test api_cls_remote_reads on pid 47286 2024-04-20T02:16:42.222 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47286 2024-04-20T02:16:42.224 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test api_cls_remote_reads on pid 47286' 2024-04-20T02:16:42.224 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47286 2024-04-20T02:16:42.224 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.224 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.224 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-04-20T02:16:42.225 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-04-20T02:16:42.225 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-04-20T02:16:42.225 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-04-20T02:16:42.237 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s list_parallel 2024-04-20T02:16:42.237 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' list_parallel' 2024-04-20T02:16:42.238 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo list_parallel 2024-04-20T02:16:42.239 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.241 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=list_parallel 2024-04-20T02:16:42.241 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47313 2024-04-20T02:16:42.242 INFO:tasks.workunit.client.0.smithi179.stdout:test list_parallel on pid 47313 2024-04-20T02:16:42.242 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test list_parallel on pid 47313' 2024-04-20T02:16:42.243 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47313 2024-04-20T02:16:42.243 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.243 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.245 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s open_pools_parallel 2024-04-20T02:16:42.245 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' open_pools_parallel' 2024-04-20T02:16:42.247 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-04-20T02:16:42.247 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.247 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo open_pools_parallel 2024-04-20T02:16:42.247 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_list_parallel.log 2024-04-20T02:16:42.247 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ list_parallel: /' 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stdout:test open_pools_parallel on pid 47338 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=open_pools_parallel 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47338 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test open_pools_parallel on pid 47338' 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47338 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-04-20T02:16:42.248 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.249 INFO:tasks.workunit.client.0.smithi179.stderr:++ printf %25s delete_pools_parallel 2024-04-20T02:16:42.249 INFO:tasks.workunit.client.0.smithi179.stderr:+ r=' delete_pools_parallel' 2024-04-20T02:16:42.249 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_list_parallel 2024-04-20T02:16:42.249 INFO:tasks.workunit.client.0.smithi179.stderr:++ echo delete_pools_parallel 2024-04-20T02:16:42.249 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_open_pools_parallel 2024-04-20T02:16:42.249 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-04-20T02:16:42.250 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stdout:test delete_pools_parallel on pid 47364 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:++ awk '{print $1}' 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ ff=delete_pools_parallel 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47364 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ echo 'test delete_pools_parallel on pid 47364' 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ pids[$f]=47364 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ ret=0 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ '[' 1 -eq 1 ']' 2024-04-20T02:16:42.256 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:16:42.257 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47004 2024-04-20T02:16:42.257 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47004 2024-04-20T02:16:42.257 INFO:tasks.workunit.client.0.smithi179.stderr:+ ceph_test_rados_delete_pools_parallel 2024-04-20T02:16:42.257 INFO:tasks.workunit.client.0.smithi179.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-04-20T02:16:42.257 INFO:tasks.workunit.client.0.smithi179.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-04-20T02:16:43.030 INFO:tasks.workunit.client.0.smithi179.stdout:00 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f189401c650 con 0x55fad8c1e2b0 2024-04-20T02:16:43.030 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da13700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1888038850 0x7f188803ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:43.030 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4346+0+0 (secure 0 0 0) 0x7f189404bdc0 con 0x55fad8c1e2b0 2024-04-20T02:16:43.030 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da13700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_map(16..16 src has 1..16) v4 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da13700 3 client.4461.objecter handle_osd_map got epochs [16,16] > 0 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da13700 3 client.4461.objecter handle_osd_map decoding full epoch 16 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f189da13700 20 client.4461.objecter dump_active .. 0 homeless 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f18a2f128c0 10 librados: wait_for_osdmap done waiting 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.088+0000 7f18a2f128c0 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x55fad8b99c60 con 0x55fad8c1e2b0 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f189e214700 4 set_mon_vals no callback set 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f189401c900 con 0x55fad8c1e2b0 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f189da13700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f189e214700 10 client.4461.objecter _get_latest_version latest 16, have it 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f18a2f128c0 10 client.4461.objecter create_pool name=ceph_test_rados_api_asio 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f18a2f128c0 10 client.4461.objecter pool_op_submit 1 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f18a2f128c0 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.089+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x55fad8baabe0 con 0x55fad8c1e2b0 2024-04-20T02:16:43.031 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.094+0000 7f18a0218700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1888038850 0x7f188803ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:43.032 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.094+0000 7f18a0218700 10 monclient: get_auth_request con 0x7f1888038850 auth_method 0 2024-04-20T02:16:43.032 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.094+0000 7f18a0218700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1888038850 0x7f188803ad10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18900026c0 tx=0x7f1890003b80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:43.032 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:42.094+0000 7f189da13700 10 client.4461.objecter ms_handle_connect 0x7f1888038850 2024-04-20T02:16:43.036 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 7 ==== pool_op_reply(tid 1 (0) Success v17) v1 ==== 43+0+0 (secure 0 0 0) 0x7f189403e090 con 0x55fad8c1e2b0 2024-04-20T02:16:43.036 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 pool_op_reply(tid 1 (0) Success v17) v1 2024-04-20T02:16:43.036 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 client.4461.objecter handle_pool_op_reply pool_op_reply(tid 1 (0) Success v17) v1 2024-04-20T02:16:43.036 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 client.4461.objecter have request 1 at 0x55fad8cb1b80 Op: create 2024-04-20T02:16:43.036 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 20 client.4461.objecter waiting for client to reach epoch 17 before calling back 2024-04-20T02:16:43.036 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 client.4461.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 monclient: _renew_subs 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7f1888002ca0 con 0x55fad8c1e2b0 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.031+0000 7f189da13700 10 client.4461.objecter done 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.036+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== mon.6 v2:172.21.15.47:3303/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 8363+0+0 (secure 0 0 0) 0x7f1894041020 con 0x55fad8c1e2b0 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f189da13700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_map(17..17 src has 1..17) v4 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f189da13700 3 client.4461.objecter handle_osd_map got epochs [17,17] > 16 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f189da13700 3 client.4461.objecter handle_osd_map decoding incremental epoch 17 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f189da13700 20 client.4461.objecter dump_active .. 0 homeless 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 10 librados: set snap read head -> 1 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 10 librados: writefull oid=exist nspace= 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d4d5e0 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 17 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:43.037 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:43.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x55fad8d57e80 0x55fad8d5a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:16:43.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:43.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [writefull 0~5 in=5b] tid 2 osd.1 2024-04-20T02:16:43.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:43.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 2 2024-04-20T02:16:43.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 15 client.4461.objecter _send_op 2 to 5.1 on osd.1 2024-04-20T02:16:44.190 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:2 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x55fad8d5a7b0 con 0x55fad8d57e80 2024-04-20T02:16:44.190 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.190 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.037+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.190 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.038+0000 7f189fa17700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x55fad8d57e80 0x55fad8d5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:16:44.190 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.038+0000 7f189fa17700 10 monclient: get_auth_request con 0x55fad8d57e80 auth_method 0 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.039+0000 7f189fa17700 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x55fad8d57e80 0x55fad8d5a270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:43.040+0000 7f189da13700 10 client.4461.objecter ms_handle_connect 0x55fad8d57e80 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 1 ==== osd_map(18..18 src has 1..18) v4 ==== 7441+0+0 (crc 0 0 0) 0x7f189800c880 con 0x55fad8d57e80 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_map(18..18 src has 1..18) v4 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 3 client.4461.objecter handle_osd_map got epochs [18,18] > 17 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 3 client.4461.objecter handle_osd_map decoding incremental epoch 18 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 10 client.4461.objecter checking op 2 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 20 client.4461.objecter dump_active .. 0 homeless 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.106+0000 7f189da13700 20 client.4461.objecter 2 5.d3a59201 osd.1 exist [writefull 0~5 in=5b] 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 2 ==== osd_op_reply(2 exist [writefull 0~5] v18'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.191 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(2 exist [writefull 0~5] v18'1 uv1 ondisk = 0) v8 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 2 ondisk uv 1 in 5.1 attempt 0 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 2 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 15 client.4461.objecter _finish_op 2 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 2 2024-04-20T02:16:44.192 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.198 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-04-20T02:16:44.198 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 librados: Objecter returned from writefull r=0 2024-04-20T02:16:44.198 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d4d5e0 2024-04-20T02:16:44.198 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.198 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [read 0~256] tid 3 osd.1 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 3 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 15 client.4461.objecter _send_op 3 to 5.1 on osd.1 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:3 5.1 5:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d505e0 con 0x55fad8d57e80 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d509c0 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base noexist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target noexist @5 -> pgid 5.a7bbaa1 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.a7bbaa1 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid noexist '@5' '@5' [read 0~256] tid 4 osd.1 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 4 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 15 client.4461.objecter _send_op 4 to 5.1 on osd.1 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:4 5.1 5:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d50eb0 con 0x55fad8d57e80 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.191+0000 7f18a2f128c0 5 client.4461.objecter 2 in flight 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(3 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 3 ondisk uv 1 in 5.1 attempt 0 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 5 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 3 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 15 client.4461.objecter _finish_op 3 2024-04-20T02:16:44.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (9 ms) 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 3 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 4 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 4 ondisk uv 0 in 5.1 attempt 0 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 4 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 15 client.4461.objecter _finish_op 4 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 4 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d509c0 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [read 0~256] tid 5 osd.1 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 5 2024-04-20T02:16:44.202 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 15 client.4461.objecter _send_op 5 to 5.1 on osd.1 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:5 5.1 5:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d4d5e0 con 0x55fad8d57e80 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5ab60 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base noexist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target noexist @5 -> pgid 5.a7bbaa1 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.a7bbaa1 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid noexist '@5' '@5' [read 0~256] tid 6 osd.1 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.210 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 6 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 15 client.4461.objecter _send_op 6 to 5.1 on osd.1 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:6 5.1 5:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b050 con 0x55fad8d57e80 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.199+0000 7f18a2f128c0 5 client.4461.objecter 2 in flight 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(5 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 5 ondisk uv 1 in 5.1 attempt 0 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 5 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 5 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 15 client.4461.objecter _finish_op 5 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 5 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.204+0000 7f189fa17700 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.211 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 6 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 6 ondisk uv 0 in 5.1 attempt 0 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 10 client.4461.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 6 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 15 client.4461.objecter _finish_op 6 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 6 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (11 ms) 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5ab60 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.212 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [read 0~256] tid 7 osd.1 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 7 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 15 client.4461.objecter _send_op 7 to 5.1 on osd.1 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:7 5.1 5:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d509c0 con 0x55fad8d57e80 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d9ba90 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base noexist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target noexist @5 -> pgid 5.a7bbaa1 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.a7bbaa1 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.213 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid noexist '@5' '@5' [read 0~256] tid 8 osd.1 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 8 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 15 client.4461.objecter _send_op 8 to 5.1 on osd.1 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:8 5.1 5:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d9bf80 con 0x55fad8d57e80 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.211+0000 7f18a2f128c0 5 client.4461.objecter 2 in flight 2024-04-20T02:16:44.214 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(7 exist [read 0~5 out=5b] v0'0 uv1 ondisk = 0) v8 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 7 ondisk uv 1 in 5.1 attempt 0 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 5 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 7 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 15 client.4461.objecter _finish_op 7 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 7 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 8 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 8 ondisk uv 0 in 5.1 attempt 0 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-04-20T02:16:44.215 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 8 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 15 client.4461.objecter _finish_op 8 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 8 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (2 ms) 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 librados: queue_aio_write 0x55fad8b3c160 completion 0x55fad8bf3260 write_seq 1 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5ab60 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.216 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [write 0~5 in=5b] tid 9 osd.1 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 9 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 15 client.4461.objecter _send_op 9 to 5.1 on osd.1 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:9 5.1 5:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5c410 con 0x55fad8d57e80 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.212+0000 7f18a2f128c0 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 9 ==== osd_op_reply(9 exist [write 0~5] v18'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(9 exist [write 0~5] v18'2 uv2 ondisk = 0) v8 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 9 ondisk uv 2 in 5.1 attempt 0 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 9 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 15 client.4461.objecter _finish_op 9 2024-04-20T02:16:44.217 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.218 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 9 2024-04-20T02:16:44.218 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.218 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f189fa17700 20 librados: complete_aio_write 0x55fad8bf3260 2024-04-20T02:16:44.218 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (2 ms) 2024-04-20T02:16:44.218 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-04-20T02:16:44.218 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-04-20T02:16:44.224 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 20 librados: queue_aio_write 0x55fad8b3c160 completion 0x55fad8d5c7c0 write_seq 2 2024-04-20T02:16:44.224 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5ab60 2024-04-20T02:16:44.224 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.224 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [write 0~5 in=5b] tid 10 osd.1 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 10 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 15 client.4461.objecter _send_op 10 to 5.1 on osd.1 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.214+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:10 5.1 5:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5c410 con 0x55fad8d57e80 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.215+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.215+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.215+0000 7f18a2f128c0 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 10 ==== osd_op_reply(10 exist [write 0~5] v18'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(10 exist [write 0~5] v18'3 uv3 ondisk = 0) v8 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 10 ondisk uv 3 in 5.1 attempt 0 2024-04-20T02:16:44.225 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 10 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 15 client.4461.objecter _finish_op 10 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 10 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f189fa17700 20 librados: complete_aio_write 0x55fad8d5c7c0 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (11 ms) 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=head 2024-04-20T02:16:44.226 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 librados: queue_aio_write 0x55fad8b3c160 completion 0x55fad8d514c0 write_seq 3 2024-04-20T02:16:44.227 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5ab60 2024-04-20T02:16:44.227 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.227 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.227 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.227 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.227 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [write 0~5 in=5b] tid 11 osd.1 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 11 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 15 client.4461.objecter _send_op 11 to 5.1 on osd.1 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:11 5.1 5:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d7eb80 con 0x55fad8d57e80 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.225+0000 7f18a2f128c0 20 librados: aio_write exist 0~5 snapc=0=[] snap_seq=1 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 11 ==== osd_op_reply(11 exist [write 0~5] v18'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(11 exist [write 0~5] v18'4 uv4 ondisk = 0) v8 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 11 ondisk uv 4 in 5.1 attempt 0 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 11 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 15 client.4461.objecter _finish_op 11 2024-04-20T02:16:44.228 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 11 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f189fa17700 20 librados: complete_aio_write 0x55fad8d514c0 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3 ms) 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5b400 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.229 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [read 0~0] tid 12 osd.1 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 12 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 15 client.4461.objecter _send_op 12 to 5.1 on osd.1 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:12 5.1 5:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b8f0 con 0x55fad8d57e80 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.230 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5bdb0 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base noexist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target noexist @5 -> pgid 5.a7bbaa1 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.a7bbaa1 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid noexist '@5' '@5' [read 0~0] tid 13 osd.1 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 13 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 15 client.4461.objecter _send_op 13 to 5.1 on osd.1 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:13 5.1 5:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d7ef90 con 0x55fad8d57e80 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.228+0000 7f18a2f128c0 5 client.4461.objecter 2 in flight 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.231 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(12 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 12 ondisk uv 4 in 5.1 attempt 0 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 5 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 12 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 15 client.4461.objecter _finish_op 12 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 12 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 13 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 13 ondisk uv 0 in 5.1 attempt 0 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 13 2024-04-20T02:16:44.232 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 15 client.4461.objecter _finish_op 13 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 13 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5bdb0 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [read 0~0] tid 14 osd.1 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 14 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 15 client.4461.objecter _send_op 14 to 5.1 on osd.1 2024-04-20T02:16:44.233 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:14 5.1 5:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b400 con 0x55fad8d57e80 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d7ef90 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base noexist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target noexist @5 -> pgid 5.a7bbaa1 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.a7bbaa1 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid noexist '@5' '@5' [read 0~0] tid 15 osd.1 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 15 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 15 client.4461.objecter _send_op 15 to 5.1 on osd.1 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:15 5.1 5:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d7f480 con 0x55fad8d57e80 2024-04-20T02:16:44.234 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.229+0000 7f18a2f128c0 5 client.4461.objecter 2 in flight 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(14 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 14 ondisk uv 4 in 5.1 attempt 0 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 5 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 14 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 15 client.4461.objecter _finish_op 14 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 14 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 15 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 15 ondisk uv 0 in 5.1 attempt 0 2024-04-20T02:16:44.235 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 15 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 15 client.4461.objecter _finish_op 15 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 15 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (2 ms) 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5bdb0 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [read 0~0] tid 16 osd.1 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.236 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 16 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 15 client.4461.objecter _send_op 16 to 5.1 on osd.1 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:16 5.1 5:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b400 con 0x55fad8d57e80 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d7e510 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base noexist @5 precalc_pgid 0 pgid 0.0 is_read 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target noexist @5 -> pgid 5.a7bbaa1 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.a7bbaa1 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid noexist '@5' '@5' [read 0~0] tid 17 osd.1 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 17 2024-04-20T02:16:44.237 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 15 client.4461.objecter _send_op 17 to 5.1 on osd.1 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:17 5.1 5:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d7ea00 con 0x55fad8d57e80 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.230+0000 7f18a2f128c0 5 client.4461.objecter 2 in flight 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(16 exist [read 0~5 out=5b] v0'0 uv4 ondisk = 0) v8 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.238 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 16 ondisk uv 4 in 5.1 attempt 0 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 5 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 16 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 15 client.4461.objecter _finish_op 16 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 5 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 16 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 17 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 17 ondisk uv 0 in 5.1 attempt 0 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2024-04-20T02:16:44.239 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 17 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 15 client.4461.objecter _finish_op 17 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 17 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 20 librados: queue_aio_write 0x55fad8b3c160 completion 0x55fad8d5b940 write_seq 4 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5bdb0 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [writefull 0~5 in=5b] tid 18 osd.1 2024-04-20T02:16:44.240 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 18 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 15 client.4461.objecter _send_op 18 to 5.1 on osd.1 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:18 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b400 con 0x55fad8d57e80 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.231+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 18 ==== osd_op_reply(18 exist [writefull 0~5] v18'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(18 exist [writefull 0~5] v18'5 uv5 ondisk = 0) v8 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 18 ondisk uv 5 in 5.1 attempt 0 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 18 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 15 client.4461.objecter _finish_op 18 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (4 ms) 2024-04-20T02:16:44.241 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 18 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f189fa17700 20 librados: complete_aio_write 0x55fad8d5b940 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 20 librados: queue_aio_write 0x55fad8b3c160 completion 0x55fad8d5ba50 write_seq 5 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5bdb0 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.242 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [writefull 0~5 in=5b] tid 19 osd.1 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 19 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 15 client.4461.objecter _send_op 19 to 5.1 on osd.1 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:19 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b400 con 0x55fad8d57e80 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.235+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 19 ==== osd_op_reply(19 exist [writefull 0~5] v18'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(19 exist [writefull 0~5] v18'6 uv6 ondisk = 0) v8 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 19 ondisk uv 6 in 5.1 attempt 0 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 19 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 15 client.4461.objecter _finish_op 19 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 19 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f189fa17700 20 librados: complete_aio_write 0x55fad8d5ba50 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (4 ms) 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 20 librados: queue_aio_write 0x55fad8b3c160 completion 0x55fad8d5b940 write_seq 6 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 10 client.4461.objecter _op_submit op 0x55fad8d5bdb0 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 20 client.4461.objecter _calc_target epoch 18 base exist @5 precalc_pgid 0 pgid 0.0 is_write 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 20 client.4461.objecter _calc_target target exist @5 -> pgid 5.d3a59201 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 10 client.4461.objecter _calc_target raw pgid 5.d3a59201 -> actual 5.1 acting [1,4] primary 1 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 20 client.4461.objecter _get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 10 client.4461.objecter _op_submit oid exist '@5' '@5' [writefull 0~5 in=5b] tid 20 osd.1 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 20 client.4461.objecter get_session s=0x55fad8d4dad0 osd=1 3 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 15 client.4461.objecter _session_op_assign 1 20 2024-04-20T02:16:44.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 15 client.4461.objecter _send_op 20 to 5.1 on osd.1 2024-04-20T02:16:44.253 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- osd_op(unknown.0.0:20 5.1 5:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x55fad8d5b400 con 0x55fad8d57e80 2024-04-20T02:16:44.253 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.240+0000 7f18a2f128c0 5 client.4461.objecter 1 in flight 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 1 -- 172.21.15.179:0/3028036439 <== osd.1 v2:172.21.15.47:6805/42310 20 ==== osd_op_reply(20 exist [writefull 0~5] v18'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f189800ca40 con 0x55fad8d57e80 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 10 client.4461.objecter ms_dispatch 0x55fad8b90ef0 osd_op_reply(20 exist [writefull 0~5] v18'7 uv7 ondisk = 0) v8 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 10 client.4461.objecter in handle_osd_op_reply 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 7 client.4461.objecter handle_osd_op_reply 20 ondisk uv 7 in 5.1 attempt 0 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 10 client.4461.objecter op 0 rval 0 len 0 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 15 client.4461.objecter handle_osd_op_reply completed tid 20 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 15 client.4461.objecter _finish_op 20 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 4 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 15 client.4461.objecter _session_op_remove 1 20 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 5 client.4461.objecter 0 in flight 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f189fa17700 20 librados: complete_aio_write 0x55fad8d5b940 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (15 ms) 2024-04-20T02:16:44.254 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 10 librados: watch_flush enter 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 10 librados: watch_flush exit 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 10 client.4461.objecter close_session for osd.1 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x55fad8d57e80 msgr2=0x55fad8d5a270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x55fad8d57e80 0x55fad8d5a270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 20 client.4461.objecter put_session s=0x55fad8d4dad0 osd=1 1 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 20 client.4461.objecter shutdown clearing up homeless session... 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.254+0000 7f18a2f128c0 10 client.4461.objecter successfully canceled tick 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1888038850 msgr2=0x7f188803ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1888038850 0x7f188803ad10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18900026c0 tx=0x7f1890003b80 comp rx=0 tx=0).stop 2024-04-20T02:16:44.255 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 10 monclient: shutdown 2024-04-20T02:16:44.258 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:16:44.258 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x55fad8c1e2b0 msgr2=0x55fad8d4d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:16:44.258 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 1 --2- 172.21.15.179:0/3028036439 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x55fad8c1e2b0 0x55fad8d4d0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55fad8bee920 tx=0x7f1894002fb0 comp rx=0 tx=0).stop 2024-04-20T02:16:44.258 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a0a19700 1 -- 172.21.15.179:0/3028036439 reap_dead start 2024-04-20T02:16:44.258 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 shutdown_connections 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.255+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 >> 172.21.15.179:0/3028036439 conn(0x55fad8bed520 msgr2=0x55fad8c13f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.256+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 shutdown_connections 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.256+0000 7f18a2f128c0 1 -- 172.21.15.179:0/3028036439 wait complete. 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.256+0000 7f18a2f128c0 1 librados: shutdown 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [----------] 12 tests from AsioRados (64 ms total) 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [----------] Global test environment tear-down 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (2181 ms total) 2024-04-20T02:16:44.259 INFO:tasks.workunit.client.0.smithi179.stdout: api_asio: [ PASSED ] 12 tests. 2024-04-20T02:16:44.574 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] Global test environment set-up. 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] 7 tests from LibRadosList 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.ListObjects (199 ms) 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (26 ms) 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset foo1,foo2,foo3 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo1 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo2 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo3 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset foo1,foo4,foo5 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo4 2024-04-20T02:16:44.575 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo5 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo1 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset foo6,foo7 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo7 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo6 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns1:foo4 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns1:foo5 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns2:foo7 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns2:foo6 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns1:foo1 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: :foo1 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: :foo2 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: :foo3 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (95 ms) 2024-04-20T02:16:44.576 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 1 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 10 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 13 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 7 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 14 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 0 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 15 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 11 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 5 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 8 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 6 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 3 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 4 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 12 0 2024-04-20T02:16:44.577 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 9 0 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 2 0 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (85 ms) 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: x cursor=MIN 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=1 cursor=14:02547ec2:::1:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=10 cursor=14:52ea6a34:::10:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=13 cursor=14:566253c9:::13:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=7 cursor=14:5c6b0b28:::7:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=14 cursor=14:62a1935d:::14:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=0 cursor=14:6cac518f:::0:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=15 cursor=14:863748b0:::15:head 2024-04-20T02:16:44.578 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=11 cursor=14:89d3ae78:::11:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=5 cursor=14:b29083e3:::5:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=8 cursor=14:bd63b0f1:::8:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=6 cursor=14:c4fdafeb:::6:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=3 cursor=14:cfc208b3:::3:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=4 cursor=14:d83876eb:::4:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=12 cursor=14:de5d7c5f:::12:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=9 cursor=14:e960b815:::9:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > oid=2 cursor=14:f905c69b:::2:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: FIRST> seek to MIN oid=1 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=1 cursor=14:02547ec2:::1:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=10 cursor=14:52ea6a34:::10:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-04-20T02:16:44.579 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-04-20T02:16:44.580 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=13 cursor=14:566253c9:::13:head 2024-04-20T02:16:44.580 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:566253c9:::13:head 2024-04-20T02:16:44.580 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-04-20T02:16:44.580 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=7 cursor=14:5c6b0b28:::7:head 2024-04-20T02:16:44.580 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-04-20T02:16:44.580 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-04-20T02:16:44.645 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=14 cursor=14:62a1935d:::14:head 2024-04-20T02:16:44.645 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-04-20T02:16:44.645 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-04-20T02:16:44.645 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=0 cursor=14:6cac518f:::0:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=15 cursor=14:863748b0:::15:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:863748b0:::15:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=11 cursor=14:89d3ae78:::11:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=5 cursor=14:b29083e3:::5:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=8 cursor=14:bd63b0f1:::8:head 2024-04-20T02:16:44.646 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=6 cursor=14:c4fdafeb:::6:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=3 cursor=14:cfc208b3:::3:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=4 cursor=14:d83876eb:::4:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=12 cursor=14:de5d7c5f:::12:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=9 cursor=14:e960b815:::9:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:e960b815:::9:head 2024-04-20T02:16:44.647 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : oid=2 cursor=14:f905c69b:::2:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:52ea6a34:::10:head expected=14:52ea6a34:::10:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=10 expected=10 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:02547ec2:::1:head expected=14:02547ec2:::1:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=1 expected=1 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:f905c69b:::2:head expected=14:f905c69b:::2:head 2024-04-20T02:16:44.648 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=2 expected=2 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:e960b815:::9:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:e960b815:::9:head expected=14:e960b815:::9:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=9 expected=9 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:b29083e3:::5:head expected=14:b29083e3:::5:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=5 expected=5 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:89d3ae78:::11:head expected=14:89d3ae78:::11:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=11 expected=11 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:863748b0:::15:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:863748b0:::15:head expected=14:863748b0:::15:head 2024-04-20T02:16:44.649 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-04-20T02:16:44.650 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=15 expected=15 2024-04-20T02:16:44.650 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-04-20T02:16:44.650 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:6cac518f:::0:head expected=14:6cac518f:::0:head 2024-04-20T02:16:44.650 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-04-20T02:16:45.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : e api_c_read_operations: Running main() from gmock_main.cc 2024-04-20T02:16:45.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-04-20T02:16:45.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (187 ms) 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (22 ms) 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (6 ms) 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (35 ms) 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-04-20T02:16:45.052 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (10 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (11 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (12 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (12 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (7 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (6 ms) 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-04-20T02:16:45.053 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (12 ms) 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (13 ms) 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (13 ms) 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (365 ms total) 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2822 ms total) 2024-04-20T02:16:45.054 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (154 ms) 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (20 ms) 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2689 ms) 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (2863 ms total) 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: 2024-04-20T02:16:45.060 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-04-20T02:16:45.061 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (2863 ms total) 2024-04-20T02:16:45.061 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-04-20T02:16:45.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-04-20T02:16:45.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-04-20T02:16:45.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-04-20T02:16:45.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-04-20T02:16:45.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (2790 ms) 2024-04-20T02:16:45.089 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (2790 ms total) 2024-04-20T02:16:45.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: 2024-04-20T02:16:45.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-04-20T02:16:45.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (2791 ms total) 2024-04-20T02:16:45.090 INFO:tasks.workunit.client.0.smithi179.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-04-20T02:16:45.168 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: Running main() from gmock_main.cc 2024-04-20T02:16:45.168 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-04-20T02:16:45.168 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [----------] Global test environment set-up. 2024-04-20T02:16:45.168 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-04-20T02:16:45.168 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (83 ms) 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (39 ms) 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2747 ms) 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.029011+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi179-47118-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.030415+0000 mon.g [INF] from='client.? 172.21.15.179:0/948009943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi179-47037-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.030778+0000 mon.g [INF] from='client.? 172.21.15.179:0/1619193873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi179-46819-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.031640+0000 mon.g [INF] from='client.? 172.21.15.179:0/1912490769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi179-47255-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.031674+0000 mon.d [INF] from='client.? 172.21.15.179:0/2089643454' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi179-47118-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi179-47118-1"}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.031961+0000 mon.g [INF] from='client.? 172.21.15.179:0/523578141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi179-47079-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.032032+0000 mon.d [INF] from='client.? 172.21.15.179:0/2480636474' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi179-46843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.032251+0000 mon.g [INF] from='client.? 172.21.15.179:0/2449842373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi179-46811-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.169 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.032514+0000 mon.g [INF] from='client.? 172.21.15.179:0/2227641354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi179-47009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.032742+0000 mon.g [INF] from='client.? 172.21.15.179:0/2945529565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47179-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.034826+0000 mon.g [INF] from='client.? 172.21.15.179:0/1487961644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi179-46803-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035384+0000 mon.i [INF] from='client.? 172.21.15.179:0/1549253788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035582+0000 mon.i [INF] from='client.? 172.21.15.179:0/3906250069' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi179-46962-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035656+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi179-47037-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035769+0000 mon.i [INF] from='client.? 172.21.15.179:0/661763098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035835+0000 mon.d [INF] from='client.? 172.21.15.179:0/4009237645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi179-46856-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035936+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi179-46819-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.035948+0000 mon.i [INF] from='client.? 172.21.15.179:0/4213653230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi179-46851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.170 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.036130+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi179-47255-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.036335+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi179-47079-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.036534+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi179-46811-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.036736+0000 mon.f [INF] from='client.? 172.21.15.179:0/1445268761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi179-47045-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.036937+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi179-47009-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.037138+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47179-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.037328+0000 mon.f [INF] from='client.? 172.21.15.179:0/2898709920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.171 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.039045+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.039223+0000 mon.f [INF] from='client.? 172.21.15.179:0/4281745995' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47239-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.039404+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi179-46803-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.039625+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi179-47118-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi179-47118-1"}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.040149+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi179-46962-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.040315+0000 mon.f [INF] from='client.? 172.21.15.179:0/947729632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi179-47069-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.040533+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi179-46843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.040724+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.040910+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi179-46856-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:43.041104+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi179-46851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.033756+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi179-47037-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.033868+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi179-46819-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.033950+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi179-47255-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034031+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi179-47079-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034117+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi179-46811-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034198+0000 mon.f [INF] from='client.? 172.21.15.179:0/1445268761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi179-47045-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034269+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi179-47009-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034342+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47179-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034410+0000 mon.f [INF] from='client.? 172.21.15.179:0/2898709920' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47289-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034463+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47169-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034535+0000 mon.f [INF] from='client.? 172.21.15.179:0/4281745995' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47239-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.034597+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi179-46803-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.066770+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi179-46962-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.066920+0000 mon.f [INF] from='client.? 172.21.15.179:0/947729632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi179-47069-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.066998+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi179-46843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.067101+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.067185+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi179-46856-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.067276+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi179-46851-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:45.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.759697+0000 mon.d [INF] from='client.? 172.21.15.179:0/2480636474' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi179-46843-1","var":"pg_num","val":"11"}]: dispatch 2024-04-20T02:16:45.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:44.759985+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi179-46843-1","var":"pg_num","val":"11"}]: dispatch 2024-04-20T02:16:45.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.035393+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi179-47118-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi179-47118-1"}]': finished 2024-04-20T02:16:45.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.035648+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi179-46843-1","var":"pg_num","val":"11"}]': finished 2024-04-20T02:16:46.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.050792+0000 mon.d [INF] from='client.? 172.21.15.179:0/2480636474' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi179-46843-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-20T02:16:46.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.053974+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi179-46843-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-20T02:16:46.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.096548+0000 mon.f [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T02:16:46.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.159175+0000 mon.g [INF] from='client.? 172.21.15.179:0/425839531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi179-47045-7"}]: dispatch 2024-04-20T02:16:46.184 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.163614+0000 mon.b [INF] from='client.? 172.21.15.179:0/3846766156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi179-47069-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.164868+0000 mon.c [INF] from='client.? 172.21.15.179:0/1858774032' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.186095+0000 client.admin [INF] onexx 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.217289+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.263673+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi179-47069-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.263917+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi179-47045-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.264334+0000 mon.b [INF] from='client.? 172.21.15.179:0/3846766156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi179-47069-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.264507+0000 mon.g [INF] from='client.? 172.21.15.179:0/425839531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi179-47045-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.301577+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi179-47069-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.302315+0000 mon.b [INF] from='client.? 172.21.15.179:0/3846766156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi179-47069-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.320219+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi179-47045-7"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.320795+0000 mon.g [INF] from='client.? 172.21.15.179:0/425839531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi179-47045-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.334302+0000 mon.d [INF] from='client.? 172.21.15.179:0/2867560070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi179-46819-16"}]: dispatch 2024-04-20T02:16:46.185 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.339056+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi179-47069-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.339701+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi179-46819-16"}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.339971+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi179-47045-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.340390+0000 mon.d [INF] from='client.? 172.21.15.179:0/2867560070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi179-46819-16"}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.340666+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi179-46819-16"}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.341235+0000 mon.d [INF] from='client.? 172.21.15.179:0/2867560070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi179-46819-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:45.341514+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi179-46819-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.039279+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi179-46843-1","var":"pgp_num","val":"11"}]': finished 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.039585+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi179-47069-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.039663+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi179-47045-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.039741+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi179-46819-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.041021+0000 mon.b [INF] from='client.? 172.21.15.179:0/3846766156' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi179-47069-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi179-47069-7"}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.041048+0000 mon.g [INF] from='client.? 172.21.15.179:0/1028229978' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.041437+0000 mon.g [INF] from='client.? 172.21.15.179:0/425839531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi179-47045-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi179-47045-7"}]: dispatch 2024-04-20T02:16:46.186 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.042093+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi179-47069-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi179-47069-7"}]: dispatch 2024-04-20T02:16:47.907 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.042383+0000 mon.d [INF] from='client.? 172.21.15.179:0/933223709' entity='client.admin' cmd=[{"prefix": "osd pool application list_parallel: process_1_[47381]: starting. 2024-04-20T02:16:47.907 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_1_[47381]: creating pool ceph_test_rados_list_parallel.smithi179-47332 2024-04-20T02:16:47.907 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_1_[47381]: created object 0... 2024-04-20T02:16:47.907 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_1_[47381]: created object 25... 2024-04-20T02:16:47.907 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_1_[47381]: created object 49... 2024-04-20T02:16:47.907 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_1_[47381]: finishing. 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_1_[47381]: shutting down. 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_2_[47382]: starting. 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_2_[47382]: listing objects. 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_2_[47382]: listed object 0... 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_2_[47382]: listed object 25... 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_2_[47382]: saw 50 objects 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_2_[47382]: shutting down. 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: starting. 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: creating pool ceph_test_rados_list_parallel.smithi179-47332 2024-04-20T02:16:47.908 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: created object 0... 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: created object 25... 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: created object 49... 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: finishing. 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_3_[47984]: shutting down. 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_4_[47985]: starting. 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_4_[47985]: listing objects. 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_4_[47985]: listed object 0... 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_4_[47985]: listed object 25... 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_4_[47985]: saw 44 objects 2024-04-20T02:16:47.909 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_4_[47985]: shutting down. 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_5_[47986]: starting. 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_5_[47986]: removed 25 objects... 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_5_[47986]: removed half of the objects 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_5_[47986]: removed 50 objects... 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_5_[47986]: removed 50 objects 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_5_[47986]: shutting down. 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: starting. 2024-04-20T02:16:47.910 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: creating pool ceph_test_rados_list_parallel.smithi179-47332 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: created object 0... 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: created object 25... 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: created object 49... 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: finishing. 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_6_[48123]: shutting down. 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: starting. 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: listing objects. 2024-04-20T02:16:47.911 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: listed object 0... 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: listed object 25... 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: listed object 50... 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: saw 53 objects 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_7_[48124]: shutting down. 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_8_[48126]: starting. 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_8_[48126]: added 25 objects... 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_8_[48126]: added half of the objects 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_8_[48126]: added 50 objects... 2024-04-20T02:16:47.912 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_8_[48126]: added 50 objects 2024-04-20T02:16:47.913 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_8_[48126]: shutting down. 2024-04-20T02:16:47.913 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:47.913 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.060 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ********************** open_pools_parallel: process_1_[47383]: starting. 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_1_[47383]: creating pool ceph_test_rados_open_pools_parallel.smithi179-47345 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_1_[47383]: created object 0... 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_1_[47383]: created object 25... 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_1_[47383]: created object 49... 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_1_[47383]: finishing. 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_1_[47383]: shutting down. 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_2_[47384]: starting. 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_2_[47384]: rados_pool_create. 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_2_[47384]: rados_ioctx_create. 2024-04-20T02:16:48.061 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_2_[47384]: shutting down. 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: ******************************* 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: starting. 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: creating pool ceph_test_rados_open_pools_parallel.smithi179-47345 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: created object 0... 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: created object 25... 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: created object 49... 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: finishing. 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_3_[47662]: shutting down. 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: ******************************* 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_4_[47663]: starting. 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_4_[47663]: rados_pool_create. 2024-04-20T02:16:48.062 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_4_[47663]: rados_ioctx_create. 2024-04-20T02:16:48.063 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: process_4_[47663]: shutting down. 2024-04-20T02:16:48.063 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: ******************************* 2024-04-20T02:16:48.063 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: ******************************* 2024-04-20T02:16:48.063 INFO:tasks.workunit.client.0.smithi179.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout:********* 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: starting. 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: creating pool ceph_test_rados_list_parallel.smithi179-47332 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: created object 0... 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: created object 25... 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: created object 49... 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: finishing. 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_9_[48370]: shutting down. 2024-04-20T02:16:48.246 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: starting. 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: listing objects. 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: listed object 0... 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: listed object 25... 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: listed object 50... 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: listed object 75... 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: saw 99 objects 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_10_[48371]: shutting down. 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.247 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_13_[48374]: starting. 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_13_[48374]: removed 25 objects... 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_13_[48374]: removed half of the objects 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_13_[48374]: removed 50 objects... 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_13_[48374]: removed 50 objects 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_13_[48374]: shutting down. 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_11_[48372]: starting. 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_11_[48372]: added 25 objects... 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_11_[48372]: added half of the objects 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_11_[48372]: added 50 objects... 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_11_[48372]: added 50 objects 2024-04-20T02:16:48.248 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_11_[48372]: shutting down. 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_12_[48373]: starting. 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_12_[48373]: added 25 objects... 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_12_[48373]: added half of the objects 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_12_[48373]: added 50 objects... 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_12_[48373]: added 50 objects 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_12_[48373]: shutting down. 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: starting. 2024-04-20T02:16:48.249 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: creating pool ceph_test_rados_list_parallel.smithi179-47332 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: created object 0... 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: created object 25... 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: created object 49... 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: finishing. 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_14_[48455]: shutting down. 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: starting. 2024-04-20T02:16:48.250 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listing objects. 2024-04-20T02:16:48.251 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listed object 0... 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listed object 25... 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listed object 50... 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listed object 75... 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listed object 100... 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: listed object 125... 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: saw 150 objects 2024-04-20T02:16:49.093 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_15_[48456]: shutting down. 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_16_[48457]: starting. 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_16_[48457]: added 25 objects... 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_16_[48457]: added half of the objects 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_16_[48457]: added 50 objects... 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_16_[48457]: added 50 objects 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: process_16_[48457]: shutting down. 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.094 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******************************* 2024-04-20T02:16:49.095 INFO:tasks.workunit.client.0.smithi179.stdout: list_parallel: ******* SUCCESS ********** 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: starting. 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: creating pool ceph_test_rados_delete_pools_parallel.smithi179-47365 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: created object 0... 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: created object 25... 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: created object 49... 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: finishing. 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_1_[47386]: shutting down. 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_2_[47387]: starting. 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_2_[47387]: deleting pool ceph_test_rados_delete_pools_parallel.smithi179-47365 2024-04-20T02:16:49.108 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_2_[47387]: shutting down. 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: ******************************* 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: starting. 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: creating pool ceph_test_rados_delete_pools_parallel.smithi179-47365 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: created object 0... 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: created object 25... 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: created object 49... 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: finishing. 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_3_[47685]: shutting down. 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: ******************************* 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_5_[47689]: starting. 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_5_[47689]: listing objects. 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_5_[47689]: listed object 0... 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_5_[47689]: listed object 25... 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_5_[47689]: saw 50 objects 2024-04-20T02:16:49.109 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_5_[47689]: shutting down. 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: ******************************* 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_4_[47686]: starting. 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_4_[47686]: deleting pool ceph_test_rados_delete_pools_parallel.smithi179-47365 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: process_4_[47686]: shutting down. 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: ******************************* 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: ******************************* 2024-04-20T02:16:49.110 INFO:tasks.workunit.client.0.smithi179.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-04-20T02:16:49.198 INFO:tasks.workunit.client.0.smithi179.stdout: enable","pool": "PoolQuotaPP_smithi179-46811-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.042689+0000 mon.d [INF] from='client.? 172.21.15.179:0/2867560070' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi179-46819-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi179-46819-16"}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.042693+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.042986+0000 mon.i [INF] from='client.? 172.21.15.179:0/346348655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi179-46984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.043274+0000 mon.i [INF] from='client.? 172.21.15.179:0/3667029520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi179-46827-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.043282+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi179-47045-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi179-47045-7"}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.043696+0000 mon.c [INF] from='client.? 172.21.15.179:0/3427481200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47239-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.043982+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi179-46811-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.044491+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi179-46819-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi179-46819-16"}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.044833+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi179-46984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.044971+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47239-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:46.045135+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi179-46827-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:47.197893+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-04-20T02:16:49.199 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.046453+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi179-47069-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi179-47069-7"}]': finished 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.046519+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi179-47045-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi179-47045-7"}]': finished 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.046591+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi179-46819-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi179-46819-16"}]': finished 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.046782+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi179-47118-1"}]': finished 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.046880+0000 mon.f [INF] from='client.? 172.21.15.179:0/1679883860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi179-46851-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.048304+0000 mon.f [INF] from='client.? 172.21.15.179:0/1679883860' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi179-46851-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi179-46851-10"}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.049134+0000 mon.d [INF] from='client.? 172.21.15.179:0/2089643454' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi179-47118-1"}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.049314+0000 mon.b [INF] from='client.? 172.21.15.179:0/3210566699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-46984-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.055209+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi179-47118-1"}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.055479+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-46984-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.058511+0000 mon.d [INF] from='client.? 172.21.15.179:0/933223709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi179-46811-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.059972+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi179-46811-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.113730+0000 mon.c [INF] from='client.? 172.21.15.179:0/3031494901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi179-46827-22"}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.115464+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi179-46827-22"}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.122942+0000 mon.c [INF] from='client.? 172.21.15.179:0/3031494901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi179-46827-22"}]: dispatch 2024-04-20T02:16:49.200 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.139215+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi179-46827-22"}]: dispatch 2024-04-20T02:16:49.201 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.140529+0000 mon.c [INF] from='client.? 172.21.15.179:0/3031494901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi179-46827-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:48.158011+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi179-46827-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.080392+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi179-47118-1"}]': finished 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.080523+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-46984-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.080702+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi179-46811-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.081026+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi179-46827-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.085049+0000 mon.c [INF] from='client.? 172.21.15.179:0/3031494901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi179-46827-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi179-46827-22"}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.085323+0000 mon.c [INF] from='client.? 172.21.15.179:0/2584453021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47239-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.086283+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi179-46827-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi179-46827-22"}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.086897+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47239-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.088578+0000 mon.c [INF] from='client.? 172.21.15.179:0/2845272679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi179-46803-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:50.050 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.089229+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi179-46803-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:50.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.089363+0000 mon.g [INF] from='client.? 172.21.15.179:0/3055817566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:50.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.090275+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi179-47000-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-20T02:16:50.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.095948+0000 mon.d [INF] from='client.? 172.21.15.179:0/933223709' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi179-46811-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-20T02:16:50.051 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.126520+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi179-46811-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.200366+0000 mon. api_stat_pp: Running main() from gmock_main.cc 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: seed 47069 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (133 ms) 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-04-20T02:16:52.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (14 ms) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (16 ms) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (11 ms) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (175 ms total) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1081 ms) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-04-20T02:16:52.082 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (14 ms) 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (15 ms) 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1110 ms total) 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (9933 ms total) 2024-04-20T02:16:52.083 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: Running main() from gmock_main.cc 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [----------] Global test environment set-up. 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (194 ms) 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-04-20T02:16:52.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: no timeout :/ 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: no timeout :/ 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: no timeout :/ 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: no timeout :/ 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: no timeout :/ 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (43 ms) 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (15 ms) 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.Checksum (6 ms) 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (11 ms) 2024-04-20T02:16:52.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (9 ms) 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (10 ms) 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.TruncTest (9 ms) 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (10 ms) 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (8 ms) 2024-04-20T02:16:52.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.RmXattr (14 ms) 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIo.XattrIter (5 ms) 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [----------] 14 tests from LibRadosIo (336 ms total) 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1104 ms) 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (4 ms) 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-04-20T02:16:52.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (4 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (12 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (8 ms) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1153 ms total) 2024-04-20T02:16:52.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: 2024-04-20T02:16:52.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [----------] Global test environment tear-down 2024-04-20T02:16:52.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [==========] 24 tests from 2 test suites ran. (10070 ms total) 2024-04-20T02:16:52.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_io: [ PASSED ] 24 tests. 2024-04-20T02:16:52.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: Running main() from gmock_main.cc 2024-04-20T02:16:52.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-04-20T02:16:52.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [----------] Global test environment set-up. 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStat.Stat (152 ms) 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (18 ms) 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStat.StatNS (16 ms) 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (7 ms) 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [----------] 5 tests from LibRadosStat (193 ms total) 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: 2024-04-20T02:16:52.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1064 ms) 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (229 ms) 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (1 ms) 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1295 ms total) 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [----------] Global test environment tear-down 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (9956 ms total) 2024-04-20T02:16:52.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_stat: [ PASSED ] 9 tests. 2024-04-20T02:16:52.146 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: Running main() from gmock_main.cc 2024-04-20T02:16:52.146 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [----------] Global test environment set-up. 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: seed 46827 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (28 ms) 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (56 ms) 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (7 ms) 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (5 ms) 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (3 ms) 2024-04-20T02:16:52.147 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (6 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (6 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (5 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (4 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (9 ms) 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-04-20T02:16:52.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (17 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (15 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (9 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (3 ms) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (188 ms total) 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-04-20T02:16:52.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1046 ms) 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (15 ms) 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (2 ms) 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-04-20T02:16:52.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-04-20T02:16:53.057 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ]c [INF] from='client.? 172.21.15.179:0/1858774032' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.200469+0000 client.admin [INF] threexx 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: got: 2024-04-20T02:16:49.200676+0000 mon.f [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7994 ms) 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (10863 ms total) 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [----------] Global test environment tear-down 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (10863 ms total) 2024-04-20T02:16:53.058 INFO:tasks.workunit.client.0.smithi179.stdout: api_cmd: [ PASSED ] 4 tests. 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [----------] Global test environment set-up. 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [----------] 4 tests from LibRadosService 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5265 ms) 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-04-20T02:16:54.705 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ OK ] LibRadosService.RegisterLate (20 ms) 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: cluster: 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: id: 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: health: HEALTH_WARN 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 9 pool(s) do not have an application enabled 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: services: 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: mon: 9 daemons, quorum f,b,g,c,d,i (age 40s), out of quorum: a, h, e 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: mgr: x(active, since 70s) 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: osd: 6 osds: 6 up (since 67s), 6 in (since 74s) 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: foo: 13 portals active (1 hosts, 3 zones) 2024-04-20T02:16:54.706 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: data: 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: pools: 18 pools, 449 pgs 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: objects: 34 objects, 25 MiB 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: usage: 161 MiB used, 540 GiB / 540 GiB avail 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: pgs: 49.666% pgs unknown 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 3.786% pgs not active 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 223 unknown 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 209 active+clean 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 17 creating+activating 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.707 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: io: 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: client: 6.0 MiB/s wr, 0 op/s rd, 10 op/s wr 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: cluster: 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: id: 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: health: HEALTH_WARN 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 6 pool(s) do not have an application enabled 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: services: 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: mon: 9 daemons, quorum f,b,g,c,d,i (age 42s), out of quorum: a, h, e 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: mgr: x(active, since 72s) 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: osd: 6 osds: 6 up (since 69s), 6 in (since 76s) 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: foo: 13 portals active (1 hosts, 3 zones) 2024-04-20T02:16:54.708 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: data: 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: pools: 20 pools, 561 pgs 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: objects: 61 objects, 145 MiB 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: usage: 179 MiB used, 540 GiB / 540 GiB avail 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: pgs: 17.825% pgs unknown 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 20.856% pgs not active 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 344 active+clean 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 117 creating+peering 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 100 unknown 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.709 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: io: 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: client: 12 MiB/s rd, 37 MiB/s wr, 254 op/s rd, 290 op/s wr 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: cluster: 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: id: 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: health: HEALTH_WARN 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 1 cache pools are missing hit_sets 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 8 pool(s) do not have an application enabled 2024-04-20T02:16:54.710 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:16:54.711 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: services: 2024-04-20T02:16:54.711 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: mon: 9 daemons, quorum f,b,g,c,d,i (age 44s), out of quorum: a, h, e 2024-04-20T02:16:54.711 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: mgr: x(active, since 74s) 2024-04-20T02:16:55.009 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (5 ms) 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (9 ms) 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (8 ms) 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-04-20T02:16:55.010 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (12 ms) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (5 ms) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (3 ms) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1134 ms total) 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [----------] Global test environment tear-down 2024-04-20T02:16:55.011 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (12963 ms total) 2024-04-20T02:16:55.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-04-20T02:16:55.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-04-20T02:16:55.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1053 ms) 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (11 ms) 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1064 ms total) 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (65 ms) 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-04-20T02:16:55.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3906 ms) 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (6 ms) 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (5 ms) 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817210847664 notify_id 124554051585 notifier_gid 4972 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817210847664 notify_id 124554051586 notifier_gid 4972 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (8 ms) 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817210847664 notify_id 124554051591 notifier_gid 4972 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (7 ms) 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-04-20T02:16:55.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209993664 notify_id 124554051586 notifier_gid 4972 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (6 ms) 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 124554051587 notifier_gid 4972 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (6 ms) 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 124554051587 notifier_gid 4972 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (7 ms) 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 124554051586 notifier_gid 4972 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (17 ms) 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-04-20T02:16:55.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: trying... 2024-04-20T02:16:56.022 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 124554051592 notifier_ api_lock: Running main() from gmock_main.cc 2024-04-20T02:16:56.022 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-04-20T02:16:56.022 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [----------] Global test environment set-up. 2024-04-20T02:16:56.022 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (127 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.LockShared (19 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1008 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1005 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (6 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.Unlock (4 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (6 ms) 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-04-20T02:16:56.023 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [----------] 8 tests from LibRadosLock (2178 ms total) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1043 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (15 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1010 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1007 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (8 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (16 ms) 2024-04-20T02:16:56.024 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (4 ms) 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3109 ms total) 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [----------] Global test environment tear-down 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (13975 ms total) 2024-04-20T02:16:56.025 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock: [ PASSED ] 16 tests. 2024-04-20T02:16:59.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-04-20T02:16:59.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-04-20T02:16:59.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-04-20T02:16:59.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-04-20T02:16:59.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: seed 46856 2024-04-20T02:16:59.012 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (128 ms) 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (42 ms) 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1052 ms) 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1006 ms) 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (5 ms) 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-04-20T02:16:59.013 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (14 ms) 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (2 ms) 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2253 ms total) 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (977 ms) 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-04-20T02:16:59.014 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (6 ms) 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1018 ms) 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1044 ms) 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (5 ms) 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (10 ms) 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-04-20T02:16:59.015 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (5 ms) 2024-04-20T02:16:59.016 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3069 ms total) 2024-04-20T02:16:59.016 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: 2024-04-20T02:16:59.016 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-04-20T02:16:59.016 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (16936 ms total) 2024-04-20T02:16:59.016 INFO:tasks.workunit.client.0.smithi179.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-04-20T02:17:03.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-04-20T02:17:03.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-04-20T02:17:03.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify_test_cb 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (204 ms) 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94132565180784 err -107 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1012 ms) 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94132565182992 err -107 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1007 ms) 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 85899345922 cookie 94132565182992 2024-04-20T02:17:03.106 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (10 ms) 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 85899345920 cookie 94132565198896 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (6 ms) 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 85899345921 cookie 94132565218032 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 85899345920 cookie 94132565218032 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 85899345920 cookie 94132565228512 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (8 ms) 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 85899345923 cookie 94132565228512 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 90194313220 cookie 94132565228512 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-04-20T02:17:03.107 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94132565228512 err -107 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_cb from 4548 notify_id 120259084294 cookie 94132565228512 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5012 ms) 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94132565228512 err -107 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11274 ms total) 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: watch_notify_test_cb 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1063 ms) 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1063 ms total) 2024-04-20T02:17:03.108 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: 2024-04-20T02:17:03.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-04-20T02:17:03.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (20936 ms total) 2024-04-20T02:17:03.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-04-20T02:17:03.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (2825 ms) 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3014 ms) 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3024 ms) 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2922 ms) 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3006 ms) 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2993 ms) 2024-04-20T02:17:03.118 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-04-20T02:17:03.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3104 ms) 2024-04-20T02:17:03.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (20889 ms total) 2024-04-20T02:17:03.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: 2024-04-20T02:17:03.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-04-20T02:17:03.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (20889 ms total) 2024-04-20T02:17:03.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout:gid 4972 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: timed out 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: flushing 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: flushed 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3006 ms) 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: trying... 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 137438953475 notifier_gid 4972 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: timed out 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: flushing 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: flushed 2024-04-20T02:17:11.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3005 ms) 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: List watches 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify2 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 150323855364 notifier_gid 4972 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify2 done 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: watch_check 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: unwatch2 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: flushing 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: done 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3034 ms) 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: List watches 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify2 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: handle_notify cookie 94817209996224 notify_id 163208757253 notifier_gid 4972 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: notify2 done 2024-04-20T02:17:11.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: watch_check 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: unwatch2 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: flushing 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: done 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3113 ms) 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16191 ms total) 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (28931 ms total) 2024-04-20T02:17:11.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [----------] Global test environment set-up. 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5126 ms) 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (47 ms) 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-04-20T02:17:12.560 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20033 ms) 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: attempt 0 of 20 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: failed to deregister: 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: { 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "epoch": 7, 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "modified": "2024-04-20T02:17:08.310334+0000", 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "services": { 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "laundry": { 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "daemons": { 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "summary": "", 2024-04-20T02:17:12.561 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "close-test-pid47211": { 2024-04-20T02:17:12.562 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "start_epoch": 7, 2024-04-20T02:17:12.562 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "start_stamp": "2024-04-20T02:17:07.503215+0000", 2024-04-20T02:17:12.562 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "gid": 9607, 2024-04-20T02:17:12.562 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "addr": "172.21.15.179:0/1908081898", 2024-04-20T02:17:12.562 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "metadata": { 2024-04-20T02:17:12.562 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "arch": "x86_64", 2024-04-20T02:17:12.565 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "ceph_release": "reef", 2024-04-20T02:17:12.565 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "ceph_version": "ceph version 18.2.2-1252-g15eecde2 (15eecde2bab9803889d929c7e423d07970d8cbed) reef (stable)", 2024-04-20T02:17:12.565 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "ceph_version_short": "18.2.2-1252-g15eecde2", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "distro": "rhel", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "distro_description": "Red Hat Enterprise Linux 8.6 (Ootpa)", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "distro_version": "8.6", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "foo": "bar", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "hostname": "smithi179", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "kernel_description": "#1 SMP Thu Nov 16 10:29:04 EST 2023", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "kernel_version": "4.18.0-513.9.1.el8_9.x86_64", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "mem_swap_kb": "0", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "mem_total_kb": "32525128", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "os": "Linux", 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "this": "that" 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: }, 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "task_status": {} 2024-04-20T02:17:12.566 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: }, 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "pid47198": { 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "start_epoch": 6, 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "start_stamp": "2024-04-20T02:16:54.753721+0000", 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "gid": 5293, 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "addr": "172.21.15.179:0/3686685488", 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "metadata": { 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "arch": "x86_64", 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "ceph_release": "reef", 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "ceph_version": "ceph version 18.2.2-1252-g15eecde2 (15eecde2bab9803889d929c7e423d07970d8cbed) reef (stable)", 2024-04-20T02:17:12.567 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "ceph_version_short": "18.2.2-1252-g15eecde2", 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50 osd: 6 osds: 6 up (since 71s), 6 in (since 78s) 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: data: 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: pools: 21 pools, 541 pgs 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: objects: 61 objects, 145 MiB 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: usage: 179 MiB used, 540 GiB / 540 GiB avail 2024-04-20T02:17:14.811 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: pgs: 35.490% pgs unknown 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 5.545% pgs not active 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 319 active+clean 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 192 unknown 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 30 creating+peering 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: io: 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: client: 12 MiB/s rd, 31 MiB/s wr, 254 op/s rd, 278 op/s wr 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ OK ] LibRadosService.StatusFormat (7220 ms) 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ RUN ] LibRadosService.Status 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ OK ] LibRadosService.Status (20083 ms) 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [----------] 4 tests from LibRadosService (32588 ms total) 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [----------] Global test environment tear-down 2024-04-20T02:17:14.812 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [==========] 4 tests from 1 test suite ran. (32588 ms total) 2024-04-20T02:17:14.813 INFO:tasks.workunit.client.0.smithi179.stdout: api_service: [ PASSED ] 4 tests. 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout:GHz", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "distro": "rhel", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "distro_description": "Red Hat Enterprise Linux 8.6 (Ootpa)", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "distro_version": "8.6", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "foo": "bar", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "hostname": "smithi179", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "kernel_description": "#1 SMP Thu Nov 16 10:29:04 EST 2023", 2024-04-20T02:17:17.940 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "kernel_version": "4.18.0-513.9.1.el8_9.x86_64", 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "mem_swap_kb": "0", 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "mem_total_kb": "32525128", 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "os": "Linux", 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "this": "that" 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: }, 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: "task_status": {} 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: } 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: } 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: } 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: } 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: } 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: 2024-04-20T02:17:17.941 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: attempt 1 of 20 2024-04-20T02:17:17.942 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10513 ms) 2024-04-20T02:17:17.942 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (35719 ms total) 2024-04-20T02:17:17.942 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: 2024-04-20T02:17:17.942 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [----------] Global test environment tear-down 2024-04-20T02:17:17.942 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (35719 ms total) 2024-04-20T02:17:17.942 INFO:tasks.workunit.client.0.smithi179.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-04-20T02:17:23.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: Running main() from gmock_main.cc 2024-04-20T02:17:23.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-04-20T02:17:23.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [----------] Global test environment set-up. 2024-04-20T02:17:23.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-04-20T02:17:23.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-04-20T02:17:23.148 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2927 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3054 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (2998 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2935 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3010 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (8088 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (4907 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5043 ms) 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-04-20T02:17:23.149 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8063 ms) 2024-04-20T02:17:23.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [----------] 9 tests from LibRadosPools (41025 ms total) 2024-04-20T02:17:23.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: 2024-04-20T02:17:23.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [----------] Global test environment tear-down 2024-04-20T02:17:23.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (41025 ms total) 2024-04-20T02:17:23.150 INFO:tasks.workunit.client.0.smithi179.stdout: api_pool: [ PASSED ] 9 tests. 2024-04-20T02:17:32.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-04-20T02:17:32.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] Global test environment set-up. 2024-04-20T02:17:32.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-04-20T02:17:32.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-04-20T02:17:32.078 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: 2024-04-20T02:16:45.190+0000 7fc37e2c38c0 -1 failed for service _ceph-mon._tcp 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: 2024-04-20T02:16:45.190+0000 7fc37e2c38c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (3095 ms) 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2011 ms) 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5106 ms total) 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-04-20T02:17:32.079 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: started 0x55ef4f0f3740 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: started 0x55ef4e4b2110 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: started 2 aios 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: waiting 0x55ef4f0f3740 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: waiting 0x55ef4e4b2110 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: done. 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (5909 ms) 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (5909 ms total) 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.Exec (26 ms) 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-04-20T02:17:32.080 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (12 ms) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4977 ms) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (31421 ms) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] 8 tests from LibRadosMisc (36438 ms total) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [----------] Global test environment tear-down 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (49964 ms total) 2024-04-20T02:17:32.081 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc: [ PASSED ] 12 tests. 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: Running main() from gmock_main.cc 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] Global test environment set-up. 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2004 ms) 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-04-20T02:17:38.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2016 ms) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2042 ms) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2010 ms) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8072 ms total) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4037 ms) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4259 ms) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4950 ms) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13247 ms total) 2024-04-20T02:17:38.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2919 ms) 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2037 ms) 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1968 ms) 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2065 ms) 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8989 ms total) 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-04-20T02:17:38.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3788 ms) 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4005 ms) 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (7793 ms total) 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [----------] Global test environment tear-down 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (55943 ms total) 2024-04-20T02:17:38.071 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-04-20T02:17:38.072 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:17:38.072 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47232 2024-04-20T02:17:38.072 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47232 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46830 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46830 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47116 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47116 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47034 2024-04-20T02:17:38.073 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47034 2024-04-20T02:17:41.873 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:17:41.873 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-04-20T02:17:41.873 INFO:tasks.ceph.mon.h:Restarting daemon 2024-04-20T02:17:41.873 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-04-20T02:17:41.876 INFO:tasks.ceph.mon.h:Started 2024-04-20T02:17:41.877 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-04-20T02:17:41.877 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-04-20T02:17:41.877 INFO:tasks.ceph.mon.e:Restarting daemon 2024-04-20T02:17:41.877 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-04-20T02:17:41.880 INFO:tasks.ceph.mon.e:Started 2024-04-20T02:17:41.880 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-20T02:17:41.880 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-20T02:17:41.880 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T02:17:41.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T02:17:41.883 INFO:tasks.ceph.mon.a:Started 2024-04-20T02:17:41.883 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:17:41.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:17:41.981 INFO:tasks.ceph.mon.h.smithi179.stderr:2024-04-20T02:17:41.981+0000 7fdbdba0fcc0 -1 mon.h@-1(???).log v0 log_external_backlog rewinding external_log_to from 21 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:17:41.995 INFO:tasks.ceph.mon.e.smithi047.stderr:2024-04-20T02:17:41.995+0000 7fb747db8cc0 -1 mon.e@-1(???).log v0 log_external_backlog rewinding external_log_to from 21 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:17:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.036+0000 7f12860e1700 10 monclient: get_monmap_and_config 2024-04-20T02:17:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.036+0000 7f12860e1700 10 monclient: build_initial_monmap 2024-04-20T02:17:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.036+0000 7f12860e1700 10 monclient: monmap: 2024-04-20T02:17:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:42.036 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:42.037874+0000 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:42.037874+0000 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:42.037 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.037+0000 7f12860e1700 1 Processor -- start 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.037+0000 7f12860e1700 1 -- start start 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.037+0000 7f12860e1700 10 monclient: init 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient: _add_conns ranks=[2,8,6,7,1,0,4,10,5,9,3] 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12800a0730 0x7f12800a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): picked mon.noname-b con 0x7f12800a0730 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:42.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 0x7f12800a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): picked mon.noname-e con 0x7f12800a1050 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f128009ffd0 0x7f12800a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): picked mon.noname-d con 0x7f128009ffd0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128009f000 con 0x7f12800a0730 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f128009ed20 con 0x7f128009ffd0 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f128009f2e0 con 0x7f12800a1050 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f12860e1700 10 monclient(hunting): authenticate will time out at 1741.439397s 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12800a0730 0x7f12800a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12800a0730 0x7f12800a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:33994/0 (socket says 172.21.15.47:33994) 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 1 -- 172.21.15.47:0/4178096980 learned_addr learned my addr 172.21.15.47:0/4178096980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:42.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): get_auth_request con 0x7f12800a0730 auth_method 0 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f128009ffd0 0x7f12800a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): get_auth_request con 0x7f128009ffd0 auth_method 0 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 0x7f12800a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:42.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 10 monclient(hunting): get_auth_request con 0x7f12800a1050 auth_method 0 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 1 -- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 msgr2=0x7f12800a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 1 -- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 msgr2=0x7f12800a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 0x7f12800a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient(hunting): handle_auth_done global_id 22371 payload 274 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127effd700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 0x7f12800a1d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 -- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 msgr2=0x7f12800a1d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 0x7f12800a1d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 -- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12800a0730 msgr2=0x7f12800a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:42.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12800a0730 0x7f12800a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 monclient: found mon.noname-d 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.038+0000 7f127ffff700 1 -- 172.21.15.47:0/4178096980 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800751e0 con 0x7f128009ffd0 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127ffff700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f128009ffd0 0x7f12800a2270 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1278009dc0 tx=0x7f127800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=55e6ec1e7f953b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 1 -- 172.21.15.47:0/4178096980 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f127800e070 con 0x7f128009ffd0 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: dump: 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.050 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:42.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f12860e1700 5 monclient: authenticate success, global_id 22371 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 1 -- 172.21.15.47:0/4178096980 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12780098f0 con 0x7f128009ffd0 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f12860e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f12860e1700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f12860e1700 4 set_mon_vals no callback set 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 1 -- 172.21.15.47:0/4178096980 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12780128a0 con 0x7f128009ffd0 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.039+0000 7f127e7fc700 10 monclient: dump: 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:42.052 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 10 monclient: shutdown 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 -- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f128009ffd0 msgr2=0x7f12800a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f128009ffd0 0x7f12800a2270 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f1278009dc0 tx=0x7f127800b330 comp rx=0 tx=0).stop 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 -- 172.21.15.47:0/4178096980 shutdown_connections 2024-04-20T02:17:42.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f128009ffd0 0x7f12800a2270 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12800a0730 0x7f12800a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 --2- 172.21.15.47:0/4178096980 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12800a1050 0x7f12800a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.040+0000 7f12860e1700 1 -- 172.21.15.47:0/4178096980 >> 172.21.15.47:0/4178096980 conn(0x7f128006cbf0 msgr2=0x7f128006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.041+0000 7f12860e1700 1 -- 172.21.15.47:0/4178096980 shutdown_connections 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.041+0000 7f12860e1700 1 -- 172.21.15.47:0/4178096980 wait complete. 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.041+0000 7f12860e1700 10 monclient: build_initial_monmap 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.041+0000 7f12860e1700 10 monclient: monmap: 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:42.054 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.041+0000 7f12860e1700 1 librados: starting msgr at 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.041+0000 7f12860e1700 1 librados: starting objecter 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 Processor -- start 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 -- start start 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 librados: setting wanted keys 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 librados: calling monclient init 2024-04-20T02:17:42.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient: init 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient: _add_conns ranks=[7,3,4,8,1,0,6,5,2] 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f128009ffd0 0x7f12801cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): picked mon.noname-i con 0x7f128009ffd0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12800a0730 0x7f1280137c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): picked mon.noname-g con 0x7f12800a0730 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 0x7f12801381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): picked mon.noname-c con 0x7f12800a1050 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1280124db0 con 0x7f12800a0730 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1280071fc0 con 0x7f12800a1050 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f128009e410 con 0x7f128009ffd0 2024-04-20T02:17:42.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.042+0000 7f12860e1700 10 monclient(hunting): authenticate will time out at 1741.443962s 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 0x7f12801381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f128009ffd0 0x7f12801cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 0x7f12801381d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:59520/0 (socket says 172.21.15.47:59520) 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12800a0730 0x7f1280137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 -- 172.21.15.47:0/957258119 learned_addr learned my addr 172.21.15.47:0/957258119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): get_auth_request con 0x7f12800a1050 auth_method 0 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): get_auth_request con 0x7f128009ffd0 auth_method 0 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:42.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): get_auth_request con 0x7f12800a0730 auth_method 0 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient(hunting): handle_auth_done global_id 22945 payload 824 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 -- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f128009ffd0 msgr2=0x7f12801cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:42.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f128009ffd0 0x7f12801cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 -- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12800a0730 msgr2=0x7f1280137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12800a0730 0x7f1280137c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 monclient: found mon.noname-c 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 -- 172.21.15.47:0/957258119 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009a70 con 0x7f12800a1050 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127f7fe700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f128009ffd0 0x7f12801cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127effd700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12800a0730 0x7f1280137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f127ffff700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 0x7f12801381d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f1278014040 tx=0x7f1278004ad0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f1267fff700 10 client.?.objecter ms_handle_connect 0x7f12800a1050 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f1267fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.043+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1278004840 con 0x7f12800a1050 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:42.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: dump: 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:42.060 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: _finish_auth 0 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: _check_auth_tickets 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 5 monclient: authenticate success, global_id 22945 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278012430 con 0x7f12800a1050 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 10 monclient: _renew_subs 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1280123280 con 0x7f12800a1050 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1278012a10 con 0x7f12800a1050 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:42.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 10 client.22945.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f1267fff700 10 monclient: dump: 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:42.062 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 10 monclient: _renew_subs 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1280125650 con 0x7f12800a1050 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f12860e1700 1 librados: init done 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.044+0000 7f127dffb700 4 set_mon_vals no callback set 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f12780125a0 con 0x7f12800a1050 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f126c03a2f0 0x7f126c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7256+0+0 (secure 0 0 0) 0x7f127804c990 con 0x7f12800a1050 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 10 client.22945.objecter ms_dispatch 0x7f1280001550 osd_map(75..75 src has 1..75) v4 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 3 client.22945.objecter handle_osd_map got epochs [75,75] > 0 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 3 client.22945.objecter handle_osd_map decoding full epoch 75 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f127f7fe700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f126c03a2f0 0x7f126c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f127f7fe700 10 monclient: get_auth_request con 0x7f126c03a2f0 auth_method 0 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.045+0000 7f1267fff700 20 client.22945.objecter dump_active .. 0 homeless 2024-04-20T02:17:42.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.046+0000 7f1267fff700 10 client.22945.objecter ms_handle_connect 0x7f126c03a2f0 2024-04-20T02:17:42.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.046+0000 7f127f7fe700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f126c03a2f0 0x7f126c03c7b0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f1270006d50 tx=0x7f12700092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:42.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.049+0000 7f12860e1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:42.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.049+0000 7f12860e1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:42.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.049+0000 7f12860e1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:42.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.049+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1280066610 con 0x7f12800a1050 2024-04-20T02:17:42.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.059+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f1278012b70 con 0x7f12800a1050 2024-04-20T02:17:42.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.059+0000 7f1267fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:42.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.059+0000 7f1267fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:42.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.385+0000 7f12860e1700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:17:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.385+0000 7f12860e1700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.385+0000 7f12860e1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:42.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:42.385+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1280138710 con 0x7f12800a1050 2024-04-20T02:17:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:45.043+0000 7f1266ffd700 10 monclient: tick 2024-04-20T02:17:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:45.043+0000 7f1266ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:45.043+0000 7f1266ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:45.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:45.043+0000 7f1266ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:46.043+0000 7f1266ffd700 10 monclient: tick 2024-04-20T02:17:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:46.043+0000 7f1266ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:46.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:46.043+0000 7f1266ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:47.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.043+0000 7f1266ffd700 10 monclient: tick 2024-04-20T02:17:47.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.043+0000 7f1266ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:47.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.043+0000 7f1266ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:47.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.045+0000 7f127cff9700 10 client.22945.objecter tick 2024-04-20T02:17:47.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.202+0000 7f1267fff700 1 -- 172.21.15.47:0/957258119 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3163 (secure 0 0 0) 0x7f1278011390 con 0x7f12800a1050 2024-04-20T02:17:47.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.202+0000 7f1267fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:47.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.202+0000 7f1267fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:47.202 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":14,"quorum":[1,2,3,4,5,6,7],"quorum_names":["f","b","g","c","h","d","i"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.205+0000 7f12860e1700 10 librados: watch_flush enter 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.205+0000 7f12860e1700 10 librados: watch_flush exit 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.205+0000 7f12860e1700 20 client.22945.objecter shutdown clearing up homeless session... 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.205+0000 7f12860e1700 10 client.22945.objecter successfully canceled tick 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f126c03a2f0 msgr2=0x7f126c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f126c03a2f0 0x7f126c03c7b0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f1270006d50 tx=0x7f12700092c0 comp rx=0 tx=0).stop 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 10 monclient: shutdown 2024-04-20T02:17:47.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 msgr2=0x7f12801381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 0x7f12801381d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f1278014040 tx=0x7f1278004ad0 comp rx=0 tx=0).stop 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 shutdown_connections 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f126c03a2f0 0x7f126c03c7b0 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12800a1050 0x7f12801381d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f128009ffd0 0x7f12801cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 --2- 172.21.15.47:0/957258119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12800a0730 0x7f1280137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 >> 172.21.15.47:0/957258119 conn(0x7f128006cbf0 msgr2=0x7f12800a5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 shutdown_connections 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 -- 172.21.15.47:0/957258119 wait complete. 2024-04-20T02:17:47.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:47.206+0000 7f12860e1700 1 librados: shutdown 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout:ntry=0 expected=0 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:62a1935d:::14:head expected=14:62a1935d:::14:head 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=14 expected=14 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:566253c9:::13:head 2024-04-20T02:17:48.374 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:566253c9:::13:head expected=14:566253c9:::13:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=13 expected=13 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:de5d7c5f:::12:head expected=14:de5d7c5f:::12:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=12 expected=12 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:d83876eb:::4:head expected=14:d83876eb:::4:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=4 expected=4 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:cfc208b3:::3:head expected=14:cfc208b3:::3:head 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=3 expected=3 2024-04-20T02:17:48.375 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:c4fdafeb:::6:head expected=14:c4fdafeb:::6:head 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=6 expected=6 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:bd63b0f1:::8:head expected=14:bd63b0f1:::8:head 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=8 expected=8 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : cursor()=14:5c6b0b28:::7:head expected=14:5c6b0b28:::7:head 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: : entry=7 expected=7 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (235 ms) 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-04-20T02:17:48.376 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (41754 ms) 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: split 0/5 -> MIN 14:33333333::::head 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: split 1/5 -> 14:33333333::::head 14:66666666::::head 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: split 2/5 -> 14:66666666::::head 14:99999999::::head 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: split 3/5 -> 14:99999999::::head 14:cccccccc::::head 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: split 4/5 -> 14:cccccccc::::head MAX 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (12648 ms) 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] 7 tests from LibRadosList (55042 ms total) 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1036 ms) 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset foo1,foo2,foo3 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo1 2024-04-20T02:17:48.377 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo2 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo3 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset foo1,foo4,foo5 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo4 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo5 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo1 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset foo6,foo7 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo7 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: foo6 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns1:foo4 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns1:foo5 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns2:foo7 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns2:foo6 2024-04-20T02:17:48.378 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: ns1:foo1 2024-04-20T02:17:48.379 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: :foo1 2024-04-20T02:17:48.379 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: :foo2 2024-04-20T02:17:48.379 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: :foo3 2024-04-20T02:17:48.379 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (45 ms) 2024-04-20T02:17:50.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:17:50.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.366+0000 7f0f09609700 10 monclient: get_monmap_and_config 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.366+0000 7f0f09609700 10 monclient: build_initial_monmap 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.366+0000 7f0f09609700 10 monclient: monmap: 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:50.367482+0000 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:50.367482+0000 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:50.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:50.367 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.366+0000 7f0f09609700 1 Processor -- start 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 -- start start 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient: init 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient: _add_conns ranks=[5,2,7,3,8,10,1,9,4,0,6] 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f040a0ad0 0x7f0f040a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): picked mon.noname-h con 0x7f0f040a0ad0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:50.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f040a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): picked mon.noname-b con 0x7f0f040a2800 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f040a31c0 0x7f0f0413e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): picked mon.noname-i con 0x7f0f040a31c0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0409ee40 con 0x7f0f040a2800 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040a3620 con 0x7f0f040a0ad0 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0409ddb0 con 0x7f0f040a31c0 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f09609700 10 monclient(hunting): authenticate will time out at 1749.768682s 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f040a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f040a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34924/0 (socket says 172.21.15.47:34924) 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 1 -- 172.21.15.47:0/3586504588 learned_addr learned my addr 172.21.15.47:0/3586504588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:50.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): get_auth_request con 0x7f0f040a2800 auth_method 0 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.367+0000 7f0f027fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f037fe700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f040a31c0 0x7f0f0413e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f02ffd700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f040a0ad0 0x7f0f040a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f037fe700 10 monclient(hunting): get_auth_request con 0x7f0f040a31c0 auth_method 0 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f037fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f037fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f037fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f02ffd700 10 monclient(hunting): get_auth_request con 0x7f0f040a0ad0 auth_method 0 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f02ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:50.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f02ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f02ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 10 monclient(hunting): handle_auth_done global_id 23024 payload 274 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 -- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f040a31c0 msgr2=0x7f0f0413e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f040a31c0 0x7f0f0413e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 -- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f040a0ad0 msgr2=0x7f0f040a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f040a0ad0 0x7f0f040a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 10 monclient: _finish_hunting 0 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 monclient: found mon.noname-b 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 -- 172.21.15.47:0/3586504588 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f040751e0 con 0x7f0f040a2800 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f027fc700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f040a2c80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009920 tx=0x7f0ef8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac15f280f984cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 1 -- 172.21.15.47:0/3586504588 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ef8009e70 con 0x7f0f040a2800 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:50.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: dump: 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:50.382 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: _finish_auth 0 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 1 -- 172.21.15.47:0/3586504588 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef800bcd0 con 0x7f0f040a2800 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f09609700 5 monclient: authenticate success, global_id 23024 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f09609700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f09609700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 1 -- 172.21.15.47:0/3586504588 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ef800be40 con 0x7f0f040a2800 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f09609700 4 set_mon_vals no callback set 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:50.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.368+0000 7f0f01ffb700 10 monclient: dump: 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:50.384 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 10 monclient: shutdown 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 -- 172.21.15.47:0/3586504588 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 msgr2=0x7f0f040a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f040a2c80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009920 tx=0x7f0ef8009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 -- 172.21.15.47:0/3586504588 shutdown_connections 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f040a31c0 0x7f0f0413e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f040a0ad0 0x7f0f040a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 --2- 172.21.15.47:0/3586504588 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f040a2c80 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 -- 172.21.15.47:0/3586504588 >> 172.21.15.47:0/3586504588 conn(0x7f0f040a7420 msgr2=0x7f0f040a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 -- 172.21.15.47:0/3586504588 shutdown_connections 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 1 -- 172.21.15.47:0/3586504588 wait complete. 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 10 monclient: build_initial_monmap 2024-04-20T02:17:50.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.369+0000 7f0f09609700 10 monclient: monmap: 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:50.386 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 librados: starting msgr at 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 librados: starting objecter 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 Processor -- start 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 -- start start 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 librados: setting wanted keys 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 librados: calling monclient init 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 10 monclient: init 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 10 monclient: _add_conns ranks=[8,2,6,4,1,7,3,0,5] 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 0x7f0f041c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 10 monclient(hunting): picked mon.noname-e con 0x7f0f040a0ad0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f041bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.370+0000 7f0f09609700 10 monclient(hunting): picked mon.noname-b con 0x7f0f040a2800 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:50.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f040a31c0 0x7f0f041cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 10 monclient(hunting): picked mon.noname-d con 0x7f0f040a31c0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041aa880 con 0x7f0f040a2800 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041a7f70 con 0x7f0f040a31c0 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041a9ee0 con 0x7f0f040a0ad0 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f09609700 10 monclient(hunting): authenticate will time out at 1749.772046s 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f041bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 0x7f0f041c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f040a31c0 0x7f0f041cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f041bcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:34932/0 (socket says 172.21.15.47:34932) 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 10 monclient(hunting): get_auth_request con 0x7f0f040a0ad0 auth_method 0 2024-04-20T02:17:50.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 -- 172.21.15.47:0/253889987 learned_addr learned my addr 172.21.15.47:0/253889987 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): get_auth_request con 0x7f0f040a2800 auth_method 0 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): get_auth_request con 0x7f0f040a31c0 auth_method 0 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 1 -- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 msgr2=0x7f0f041c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 1 -- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 msgr2=0x7f0f041c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 0x7f0f041c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f02ffd700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 0x7f0f041c90e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient(hunting): handle_auth_done global_id 23033 payload 824 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 -- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 msgr2=0x7f0f041c90e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 0x7f0f041c90e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 -- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f040a31c0 msgr2=0x7f0f041cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f040a31c0 0x7f0f041cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient: _finish_hunting 0 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 monclient: found mon.noname-b 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 -- 172.21.15.47:0/253889987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8009550 con 0x7f0f040a2800 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f037fe700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f040a31c0 0x7f0f041cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0f027fc700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f041bcda0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8013000 tx=0x7f0ef8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 client.?.objecter ms_handle_connect 0x7f0f040a2800 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ef800bb60 con 0x7f0f040a2800 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 monclient: dump: 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.371+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef800be40 con 0x7f0f040a2800 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 5 monclient: authenticate success, global_id 23033 2024-04-20T02:17:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 monclient: _renew_subs 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f041aa1c0 con 0x7f0f040a2800 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0eeb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ef8024470 con 0x7f0f040a2800 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0eeb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0eeb7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0eeb7fe700 10 monclient: dump: 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f017fa700 4 set_mon_vals no callback set 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 client.23033.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 monclient: _renew_subs 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f041a8f80 con 0x7f0f040a2800 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 1 librados: init done 2024-04-20T02:17:50.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0ef8021070 con 0x7f0f040a2800 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.372+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f041a9110 con 0x7f0f040a2800 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0ef003a2f0 0x7f0ef003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7972+0+0 (secure 0 0 0) 0x7f0ef804c200 con 0x7f0f040a2800 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 10 client.23033.objecter ms_dispatch 0x7f0f04001550 osd_map(80..80 src has 1..80) v4 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 3 client.23033.objecter handle_osd_map got epochs [80,80] > 0 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 3 client.23033.objecter handle_osd_map decoding full epoch 80 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 20 client.23033.objecter dump_active .. 0 homeless 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0f02ffd700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0ef003a2f0 0x7f0ef003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0f02ffd700 10 monclient: get_auth_request con 0x7f0ef003a2f0 auth_method 0 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0eeb7fe700 10 client.23033.objecter ms_handle_connect 0x7f0ef003a2f0 2024-04-20T02:17:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.373+0000 7f0f02ffd700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0ef003a2f0 0x7f0ef003c7b0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4006af0 tx=0x7f0ef4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.378+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0ef801ada0 con 0x7f0f040a2800 2024-04-20T02:17:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.378+0000 7f0eeb7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.378+0000 7f0eeb7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.681+0000 7f0f09609700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.681+0000 7f0f09609700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.681+0000 7f0f09609700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.681+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0f041c9ad0 con 0x7f0f040a2800 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.682+0000 7f0eeb7fe700 1 -- 172.21.15.47:0/253889987 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3163 (secure 0 0 0) 0x7f0ef8029c70 con 0x7f0f040a2800 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.682+0000 7f0eeb7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:50.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.682+0000 7f0eeb7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:50.682 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":14,"quorum":[1,2,3,4,5,6,7],"quorum_names":["f","b","g","c","h","d","i"],"quorum_leader_name":"f","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.683+0000 7f0f09609700 10 librados: watch_flush enter 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.683+0000 7f0f09609700 10 librados: watch_flush exit 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.683+0000 7f0f09609700 20 client.23033.objecter shutdown clearing up homeless session... 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.683+0000 7f0f09609700 10 client.23033.objecter successfully canceled tick 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0ef003a2f0 msgr2=0x7f0ef003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0ef003a2f0 0x7f0ef003c7b0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4006af0 tx=0x7f0ef4008040 comp rx=0 tx=0).stop 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 10 monclient: shutdown 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 msgr2=0x7f0f041bcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:50.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f041bcda0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8013000 tx=0x7f0ef8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 shutdown_connections 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0ef003a2f0 0x7f0ef003c7b0 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f040a0ad0 0x7f0f041c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f040a31c0 0x7f0f041cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 --2- 172.21.15.47:0/253889987 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0f040a2800 0x7f0f041bcda0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 >> 172.21.15.47:0/253889987 conn(0x7f0f040a7420 msgr2=0x7f0f040a5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 shutdown_connections 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 -- 172.21.15.47:0/253889987 wait complete. 2024-04-20T02:17:50.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:50.684+0000 7f0f09609700 1 librados: shutdown 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosListEC.Lis api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-04-20T02:17:53.101 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: seed 46962 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (380 ms) 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (6 ms) 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (7 ms) 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (5 ms) 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-04-20T02:17:53.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (1 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (1 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (24 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (7 ms) 2024-04-20T02:17:53.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: osd_max_attr_size = 0 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (2118 ms) 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (237 ms) 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: done waiting, doing copies 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: waiting for final deep scrubs... 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: done waiting 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60681 ms) 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (6 ms) 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-04-20T02:17:53.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (6 ms) 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4524 ms) 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-04-20T02:17:53.105 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-04-20T02:17:53.697 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:17:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.840+0000 7fd4918b6700 10 monclient: get_monmap_and_config 2024-04-20T02:17:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.840+0000 7fd4918b6700 10 monclient: build_initial_monmap 2024-04-20T02:17:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.840+0000 7fd4918b6700 10 monclient: monmap: 2024-04-20T02:17:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:53.842103+0000 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:53.842103+0000 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:53.841 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.841+0000 7fd4918b6700 1 Processor -- start 2024-04-20T02:17:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.841+0000 7fd4918b6700 1 -- start start 2024-04-20T02:17:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.841+0000 7fd4918b6700 10 monclient: init 2024-04-20T02:17:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient: _add_conns ranks=[5,9,6,8,7,1,4,10,2,0,3] 2024-04-20T02:17:53.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd48c0a4c80 0x7fd48c0a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): picked mon.noname-h con 0x7fd48c0a4c80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd48c0a0d20 0x7fd48c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): picked mon.noname-a con 0x7fd48c0a0d20 addr v2:172.21.15.47:3300/0 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd48c0a55a0 0x7fd48c137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): picked mon.noname-d con 0x7fd48c0a55a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0a0000 con 0x7fd48c0a0d20 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0a6a10 con 0x7fd48c0a4c80 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c09e440 con 0x7fd48c0a55a0 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd4918b6700 10 monclient(hunting): authenticate will time out at 1753.243263s 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd48c0a0d20 0x7fd48c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd48c0a0d20 0x7fd48c0a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47594/0 (socket says 172.21.15.47:47594) 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 -- 172.21.15.47:0/2372499153 learned_addr learned my addr 172.21.15.47:0/2372499153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): get_auth_request con 0x7fd48c0a0d20 auth_method 0 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd48c0a55a0 0x7fd48c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): get_auth_request con 0x7fd48c0a55a0 auth_method 0 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:53.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48affd700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd48c0a4c80 0x7fd48c0a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48affd700 10 monclient(hunting): get_auth_request con 0x7fd48c0a4c80 auth_method 0 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_done global_id 34128 payload 274 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 -- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd48c0a55a0 msgr2=0x7fd48c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd48c0a55a0 0x7fd48c137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 -- 172.21.15.47:0/2372499153 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd48c0a4c80 msgr2=0x7fd48c0a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd48c0a4c80 0x7fd48c0a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 monclient: found mon.noname-a 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:53.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 -- 172.21.15.47:0/2372499153 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0751e0 con 0x7fd48c0a0d20 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48b7fe700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd48c0a55a0 0x7fd48c137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48affd700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd48c0a4c80 0x7fd48c0a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.842+0000 7fd48a7fc700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c0a5060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384ae8fb1ae08698 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 1 -- 172.21.15.47:0/2372499153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd474011aa0 con 0x7fd48c0a0d20 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: dump: 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:53.855 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: _finish_auth 0 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 5 monclient: authenticate success, global_id 34128 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 1 -- 172.21.15.47:0/2372499153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474011c00 con 0x7fd48c0a0d20 2024-04-20T02:17:53.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 4 set_mon_vals no callback set 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 1 -- 172.21.15.47:0/2372499153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd474011d70 con 0x7fd48c0a0d20 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd489ffb700 10 monclient: dump: 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:53.857 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 10 monclient: shutdown 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 -- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 msgr2=0x7fd48c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c0a5060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474009c00 comp rx=0 tx=0).stop 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 -- 172.21.15.47:0/2372499153 shutdown_connections 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd48c0a55a0 0x7fd48c137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd48c0a4c80 0x7fd48c0a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2372499153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c0a5060 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 -- 172.21.15.47:0/2372499153 >> 172.21.15.47:0/2372499153 conn(0x7fd48c0aa280 msgr2=0x7fd48c0aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:53.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 -- 172.21.15.47:0/2372499153 shutdown_connections 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.843+0000 7fd4918b6700 1 -- 172.21.15.47:0/2372499153 wait complete. 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.844+0000 7fd4918b6700 10 monclient: build_initial_monmap 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.844+0000 7fd4918b6700 10 monclient: monmap: 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:53.859 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.844+0000 7fd4918b6700 1 librados: starting msgr at 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.844+0000 7fd4918b6700 1 librados: starting objecter 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.844+0000 7fd4918b6700 1 Processor -- start 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 -- start start 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 librados: setting wanted keys 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 librados: calling monclient init 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient: init 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient: _add_conns ranks=[0,8,7,1,5,4,3,6,2] 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c1cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): picked mon.noname-a con 0x7fd48c0a0d20 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:53.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd48c0a4c80 0x7fd48c1ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): picked mon.noname-e con 0x7fd48c0a4c80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd48c0a55a0 0x7fd48c124550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): picked mon.noname-i con 0x7fd48c0a55a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c09ea70 con 0x7fd48c0a0d20 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c09dd20 con 0x7fd48c0a55a0 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c126d60 con 0x7fd48c0a4c80 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd4918b6700 10 monclient(hunting): authenticate will time out at 1753.246763s 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c1cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c1cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47610/0 (socket says 172.21.15.47:47610) 2024-04-20T02:17:53.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 1 -- 172.21.15.47:0/2531558989 learned_addr learned my addr 172.21.15.47:0/2531558989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): get_auth_request con 0x7fd48c0a0d20 auth_method 0 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48a7fc700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd48c0a4c80 0x7fd48c1ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48a7fc700 10 monclient(hunting): get_auth_request con 0x7fd48c0a4c80 auth_method 0 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.845+0000 7fd48affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd48c0a55a0 0x7fd48c124550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): get_auth_request con 0x7fd48c0a55a0 auth_method 0 2024-04-20T02:17:53.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 10 monclient(hunting): handle_auth_done global_id 34137 payload 824 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 -- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd48c0a4c80 msgr2=0x7fd48c1ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd48c0a4c80 0x7fd48c1ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 -- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd48c0a55a0 msgr2=0x7fd48c124550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd48c0a55a0 0x7fd48c124550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 10 monclient: _finish_hunting 0 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 monclient: found mon.noname-a 2024-04-20T02:17:53.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 -- 172.21.15.47:0/2531558989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484009a70 con 0x7fd48c0a0d20 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd48affd700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c1cc740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e810 tx=0x7fd47c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 client.?.objecter ms_handle_connect 0x7fd48c0a0d20 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd47c013880 con 0x7fd48c0a0d20 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: dump: 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:53.864 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: _finish_auth 0 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: _check_auth_tickets 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 5 monclient: authenticate success, global_id 34137 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c013a00 con 0x7fd48c0a0d20 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:53.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 10 monclient: _renew_subs 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd47c012370 con 0x7fd48c0a0d20 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd474009550 con 0x7fd48c0a0d20 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4897fa700 4 set_mon_vals no callback set 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd483fff700 10 monclient: dump: 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:53.866 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 10 client.34137.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 10 monclient: _renew_subs 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c127ac0 con 0x7fd48c0a0d20 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.846+0000 7fd4918b6700 1 librados: init done 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd4918b6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:53.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd4918b6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd4918b6700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c137f10 con 0x7fd48c0a0d20 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd47c013410 con 0x7fd48c0a0d20 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd483fff700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd46c03a2f0 0x7fd46c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7878+0+0 (secure 0 0 0) 0x7fd47c04a860 con 0x7fd48c0a0d20 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd483fff700 10 client.34137.objecter ms_dispatch 0x7fd48c001550 osd_map(83..83 src has 1..83) v4 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd483fff700 3 client.34137.objecter handle_osd_map got epochs [83,83] > 0 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.847+0000 7fd483fff700 3 client.34137.objecter handle_osd_map decoding full epoch 83 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.848+0000 7fd483fff700 20 client.34137.objecter dump_active .. 0 homeless 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.848+0000 7fd48a7fc700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd46c03a2f0 0x7fd46c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.848+0000 7fd48a7fc700 10 monclient: get_auth_request con 0x7fd46c03a2f0 auth_method 0 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.848+0000 7fd483fff700 10 client.34137.objecter ms_handle_connect 0x7fd46c03a2f0 2024-04-20T02:17:53.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.848+0000 7fd48a7fc700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd46c03a2f0 0x7fd46c03c7b0 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:53.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.853+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd47c0136c0 con 0x7fd48c0a0d20 2024-04-20T02:17:53.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.853+0000 7fd483fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:53.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:53.853+0000 7fd483fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6tObjectsStart 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 1 0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 10 0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 13 0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 7 0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 14 0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 0 0 2024-04-20T02:17:54.102 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 15 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 11 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 5 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 8 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 6 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 3 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 4 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 12 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 9 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 2 0 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (110 ms) 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] 3 tests from LibRadosListEC (1191 ms total) 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 2024-04-20T02:17:54.103 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3020 ms) 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] 1 test from LibRadosListNP (3020 ms total) 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [----------] Global test environment tear-down 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [==========] 11 tests from 3 test suites ran. (72035 ms total) 2024-04-20T02:17:54.104 INFO:tasks.workunit.client.0.smithi179.stdout: api_list: [ PASSED ] 11 tests. 2024-04-20T02:17:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd4918b6700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:17:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd4918b6700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd4918b6700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd48c127ce0 con 0x7fd48c0a0d20 2024-04-20T02:17:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd483fff700 1 -- 172.21.15.47:0/2531558989 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fd47c023300 con 0x7fd48c0a0d20 2024-04-20T02:17:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd483fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:54.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.180+0000 7fd483fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:54.180 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":18,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 10 librados: watch_flush enter 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 10 librados: watch_flush exit 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 20 client.34137.objecter shutdown clearing up homeless session... 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 10 client.34137.objecter successfully canceled tick 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd46c03a2f0 msgr2=0x7fd46c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd46c03a2f0 0x7fd46c03c7b0 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098f0 tx=0x7fd474008030 comp rx=0 tx=0).stop 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 10 monclient: shutdown 2024-04-20T02:17:54.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 msgr2=0x7fd48c1cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c1cc740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e810 tx=0x7fd47c00eb20 comp rx=0 tx=0).stop 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 shutdown_connections 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd46c03a2f0 0x7fd46c03c7b0 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd48c0a55a0 0x7fd48c124550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd48c0a0d20 0x7fd48c1cc740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 --2- 172.21.15.47:0/2531558989 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd48c0a4c80 0x7fd48c1ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 >> 172.21.15.47:0/2531558989 conn(0x7fd48c0aa280 msgr2=0x7fd48c0a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 shutdown_connections 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 -- 172.21.15.47:0/2531558989 wait complete. 2024-04-20T02:17:54.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.183+0000 7fd4918b6700 1 librados: shutdown 2024-04-20T02:17:54.196 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:17:54.196 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:17:54.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:17:54.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.334+0000 7f70048c1700 10 monclient: get_monmap_and_config 2024-04-20T02:17:54.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.334+0000 7f70048c1700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.334+0000 7f70048c1700 10 monclient: monmap: 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:54.335278+0000 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:54.335278+0000 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.334 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:54.335 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:54.335 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.334+0000 7f70048c1700 1 Processor -- start 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.334+0000 7f70048c1700 1 -- start start 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.334+0000 7f70048c1700 10 monclient: init 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient: _add_conns ranks=[7,3,4,9,8,0,2,6,10,5,1] 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f70000a2950 0x7f70000a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): picked mon.noname-i con 0x7f70000a2950 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000067b90 0x7f70000a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): picked mon.noname-g con 0x7f7000067b90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:54.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 0x7f7000137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): picked mon.noname-c con 0x7f7000137b00 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7000072810 con 0x7f7000067b90 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70000a37b0 con 0x7f7000137b00 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f700009f710 con 0x7f70000a2950 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f70048c1700 10 monclient(hunting): authenticate will time out at 1753.736582s 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 0x7f7000137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 0x7f7000137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:49830/0 (socket says 172.21.15.47:49830) 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 1 -- 172.21.15.47:0/1053736246 learned_addr learned my addr 172.21.15.47:0/1053736246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): get_auth_request con 0x7f7000137b00 auth_method 0 2024-04-20T02:17:54.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffe59c700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f70000a2950 0x7f70000a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffdd9b700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000067b90 0x7f70000a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffe59c700 10 monclient(hunting): get_auth_request con 0x7f70000a2950 auth_method 0 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffe59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffe59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffe59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffdd9b700 10 monclient(hunting): get_auth_request con 0x7f7000067b90 auth_method 0 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.335+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_done global_id 22954 payload 274 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 -- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f70000a2950 msgr2=0x7f70000a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f70000a2950 0x7f70000a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 -- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000067b90 msgr2=0x7f70000a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000067b90 0x7f70000a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 monclient: found mon.noname-c 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:54.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 -- 172.21.15.47:0/1053736246 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000751e0 con 0x7f7000137b00 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffed9d700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 0x7f7000137f60 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009920 tx=0x7f6ff8009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=5427a7ca1e543b4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 1 -- 172.21.15.47:0/1053736246 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff8011aa0 con 0x7f7000137b00 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: dump: 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.350 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 1 -- 172.21.15.47:0/1053736246 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011c00 con 0x7f7000137b00 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 5 monclient: authenticate success, global_id 22954 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:54.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 1 -- 172.21.15.47:0/1053736246 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff8011d70 con 0x7f7000137b00 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 4 set_mon_vals no callback set 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f6ffd59a700 10 monclient: dump: 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.352 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 10 monclient: shutdown 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 1 -- 172.21.15.47:0/1053736246 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 msgr2=0x7f7000137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.336+0000 7f70048c1700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 0x7f7000137f60 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009920 tx=0x7f6ff8009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 -- 172.21.15.47:0/1053736246 shutdown_connections 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7000137b00 0x7f7000137f60 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f70000a2950 0x7f70000a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 --2- 172.21.15.47:0/1053736246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000067b90 0x7f70000a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 -- 172.21.15.47:0/1053736246 >> 172.21.15.47:0/1053736246 conn(0x7f70000a7570 msgr2=0x7f70000a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 -- 172.21.15.47:0/1053736246 shutdown_connections 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 1 -- 172.21.15.47:0/1053736246 wait complete. 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.337+0000 7f70048c1700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient: monmap: 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.354 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 librados: starting msgr at 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 librados: starting objecter 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 Processor -- start 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 -- start start 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 librados: setting wanted keys 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 librados: calling monclient init 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient: init 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient: _add_conns ranks=[7,6,3,2,4,8,1,0,5] 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7000067b90 0x7f70001cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): picked mon.noname-i con 0x7f7000067b90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f70000a2950 0x7f7000136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): picked mon.noname-d con 0x7f70000a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000137b00 0x7f7000136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): picked mon.noname-g con 0x7f7000137b00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70001264a0 con 0x7f7000137b00 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f700009ddb0 con 0x7f70000a2950 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f700009d960 con 0x7f7000067b90 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.338+0000 7f70048c1700 10 monclient(hunting): authenticate will time out at 1753.739945s 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f70000a2950 0x7f7000136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7000067b90 0x7f70001cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f70000a2950 0x7f7000136a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50076/0 (socket says 172.21.15.47:50076) 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 1 -- 172.21.15.47:0/239147162 learned_addr learned my addr 172.21.15.47:0/239147162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): get_auth_request con 0x7f70000a2950 auth_method 0 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000137b00 0x7f7000136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): get_auth_request con 0x7f7000067b90 auth_method 0 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): get_auth_request con 0x7f7000137b00 auth_method 0 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient(hunting): handle_auth_done global_id 20806 payload 824 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f70000a2950 msgr2=0x7f7000136a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f70000a2950 0x7f7000136a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000137b00 msgr2=0x7f7000136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7000137b00 0x7f7000136f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 monclient: found mon.noname-i 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8009550 con 0x7f7000067b90 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffdd9b700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f70000a2950 0x7f7000136a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ffe59c700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7000067b90 0x7f70001cc320 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009db0 tx=0x7f6ff000c3a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ff6ffd700 10 client.?.objecter ms_handle_connect 0x7f7000067b90 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ff6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff0009c20 con 0x7f7000067b90 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.339+0000 7f6ff6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: dump: 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.359 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff000f040 con 0x7f7000067b90 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 5 monclient: authenticate success, global_id 20806 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 monclient: _renew_subs 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7000124040 con 0x7f7000067b90 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ff00148f0 con 0x7f7000067b90 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ffcd99700 4 set_mon_vals no callback set 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 client.20806.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 10 monclient: dump: 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 monclient: _renew_subs 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:17:54.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7000125e50 con 0x7f7000067b90 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 1 librados: init done 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 0x7f70001224e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000068310 con 0x7f700006aac0 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.340+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6ff0014480 con 0x7f7000067b90 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 0x7f70001224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient: get_auth_request con 0x7f700006aac0 auth_method 0 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fe003a2f0 0x7f6fe003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7788+0+0 (secure 0 0 0) 0x7f6ff004b3e0 con 0x7f7000067b90 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 10 client.20806.objecter ms_dispatch 0x7f7000001550 osd_map(84..84 src has 1..84) v4 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 3 client.20806.objecter handle_osd_map got epochs [84,84] > 0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 3 client.20806.objecter handle_osd_map decoding full epoch 84 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_done global_id 34146 payload 824 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7000067560 con 0x7f700006aac0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffed9d700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fe003a2f0 0x7f6fe003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffed9d700 10 monclient: get_auth_request con 0x7f6fe003a2f0 auth_method 0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 20 client.20806.objecter dump_active .. 0 homeless 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 10 client.20806.objecter ms_handle_connect 0x7f700006aac0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 10 client.20806.objecter resend_mon_ops 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ffdd9b700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 0x7f70001224e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009ee0 tx=0x7f6fe800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fe800fd80 con 0x7f700006aac0 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.341+0000 7f6ff6ffd700 10 monclient: dump: 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.365 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 10 client.20806.objecter ms_handle_connect 0x7f6fe003a2f0 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ffed9d700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fe003a2f0 0x7f6fe003c7b0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009840 tx=0x7f6ff8008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6fe8014440 con 0x7f700006aac0 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 msgr2=0x7f70001224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 0x7f70001224e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009ee0 tx=0x7f6fe800f520 comp rx=0 tx=0).stop 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 msgr2=0x7f70001224e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.342+0000 7f6ff6ffd700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f700006aac0 0x7f70001224e0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009ee0 tx=0x7f6fe800f520 comp rx=0 tx=0).stop 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.349+0000 7f70048c1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.349+0000 7f70048c1700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 0x7f7000122e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.349+0000 7f70048c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.349+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700006a400 con 0x7f7000122a20 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 0x7f7000122e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient: get_auth_request con 0x7f7000122a20 auth_method 0 2024-04-20T02:17:54.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 10 monclient(hunting): handle_auth_done global_id 34155 payload 824 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 1 -- 172.21.15.47:0/239147162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f700009ee70 con 0x7f7000122a20 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 10 client.20806.objecter ms_handle_connect 0x7f7000122a20 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 10 client.20806.objecter resend_mon_ops 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ffdd9b700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 0x7f7000122e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006b20 tx=0x7f6fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fe802b070 con 0x7f7000122a20 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 10 monclient: dump: 2024-04-20T02:17:54.369 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.370 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.350+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f6fe8023040 con 0x7f7000122a20 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ff6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ff6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 msgr2=0x7f7000122e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ff6ffd700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 0x7f7000122e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006b20 tx=0x7f6fe8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ff6ffd700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 msgr2=0x7f7000122e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ff6ffd700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7000122a20 0x7f7000122e30 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8006b20 tx=0x7f6fe8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:54.371 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.372 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:54.373 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:54.374 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.375 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.376 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.379 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.385 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:54.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.388 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:54.391 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 10 librados: watch_flush enter 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 10 librados: watch_flush exit 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 20 client.20806.objecter shutdown clearing up homeless session... 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 10 client.20806.objecter successfully canceled tick 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fe003a2f0 msgr2=0x7f6fe003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fe003a2f0 0x7f6fe003c7b0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009840 tx=0x7f6ff8008030 comp rx=0 tx=0).stop 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 10 monclient: shutdown 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7000067b90 msgr2=0x7f70001cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 --2- 172.21.15.47:0/239147162 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7000067b90 0x7f70001cc320 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009db0 tx=0x7f6ff000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ffed9d700 1 -- 172.21.15.47:0/239147162 reap_dead start 2024-04-20T02:17:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f6ffed9d700 1 -- 172.21.15.47:0/239147162 reap_dead start 2024-04-20T02:17:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 shutdown_connections 2024-04-20T02:17:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 >> 172.21.15.47:0/239147162 conn(0x7f70000a7570 msgr2=0x7f70000a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 shutdown_connections 2024-04-20T02:17:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 -- 172.21.15.47:0/239147162 wait complete. 2024-04-20T02:17:54.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.351+0000 7f70048c1700 1 librados: shutdown 2024-04-20T02:17:54.394 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:54.394 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:17:54.394 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.533+0000 7f840f9f5700 10 monclient: get_monmap_and_config 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.533+0000 7f840f9f5700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.533+0000 7f840f9f5700 10 monclient: monmap: 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:54.534632+0000 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:54.534632+0000 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.533 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 Processor -- start 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 -- start start 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient: init 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient: _add_conns ranks=[9,5,6,4,3,0,2,1,8,7,10] 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84080a3960 0x7f8408137b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): picked mon.noname-a con 0x7f84080a3960 addr v2:172.21.15.47:3300/0 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): picked mon.noname-h con 0x7f84081380d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 0x7f840806d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): picked mon.noname-d con 0x7f840806cc90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:54.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840809ef40 con 0x7f84080a3960 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8408138a10 con 0x7f84081380d0 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f84080a3d40 con 0x7f840806cc90 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.534+0000 7f840f9f5700 10 monclient(hunting): authenticate will time out at 1753.935885s 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 0x7f840806d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f84080a3960 0x7f8408137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 0x7f840806d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50084/0 (socket says 172.21.15.47:50084) 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 1 -- 172.21.15.47:0/3279217425 learned_addr learned my addr 172.21.15.47:0/3279217425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): get_auth_request con 0x7f840806cc90 auth_method 0 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): get_auth_request con 0x7f84080a3960 auth_method 0 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840cf90700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840cf90700 10 monclient(hunting): get_auth_request con 0x7f84081380d0 auth_method 0 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840cf90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840cf90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840cf90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient(hunting): handle_auth_done global_id 34164 payload 274 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 -- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 msgr2=0x7f840806d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 0x7f840806d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 -- 172.21.15.47:0/3279217425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 msgr2=0x7f84081384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 monclient: found mon.noname-a 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 -- 172.21.15.47:0/3279217425 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080751e0 con 0x7f84080a3960 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840df92700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 0x7f840806d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840d791700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84080a3960 0x7f8408137b90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8404009d90 tx=0x7f840400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbf7357275f3810 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 1 -- 172.21.15.47:0/3279217425 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f840400e070 con 0x7f84080a3960 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:54.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: dump: 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.550 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 1 -- 172.21.15.47:0/3279217425 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84040098f0 con 0x7f84080a3960 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840f9f5700 5 monclient: authenticate success, global_id 34164 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840f9f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840f9f5700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f840f9f5700 4 set_mon_vals no callback set 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 1 -- 172.21.15.47:0/3279217425 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84040128a0 con 0x7f84080a3960 2024-04-20T02:17:54.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.535+0000 7f83fffff700 10 monclient: dump: 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.553 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.554 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 10 monclient: shutdown 2024-04-20T02:17:54.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 -- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84080a3960 msgr2=0x7f8408137b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84080a3960 0x7f8408137b90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8404009d90 tx=0x7f840400b330 comp rx=0 tx=0).stop 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 -- 172.21.15.47:0/3279217425 shutdown_connections 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84080a3960 0x7f8408137b90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f840806cc90 0x7f840806d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3279217425 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 -- 172.21.15.47:0/3279217425 >> 172.21.15.47:0/3279217425 conn(0x7f84080a4a50 msgr2=0x7f84080a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 -- 172.21.15.47:0/3279217425 shutdown_connections 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.536+0000 7f840f9f5700 1 -- 172.21.15.47:0/3279217425 wait complete. 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 10 monclient: monmap: 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.556 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 1 librados: starting msgr at 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 1 librados: starting objecter 2024-04-20T02:17:54.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 1 Processor -- start 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 1 -- start start 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 1 librados: setting wanted keys 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 1 librados: calling monclient init 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.537+0000 7f840f9f5700 10 monclient: init 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient: _add_conns ranks=[0,3,5,8,4,6,7,2,1] 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f840806cc90 0x7f84081cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): picked mon.noname-a con 0x7f840806cc90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84080a3960 0x7f8408134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): picked mon.noname-g con 0x7f84080a3960 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081350f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): picked mon.noname-h con 0x7f84081380d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84080720f0 con 0x7f840806cc90 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8408072590 con 0x7f84080a3960 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8408122590 con 0x7f84081380d0 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840f9f5700 10 monclient(hunting): authenticate will time out at 1753.939241s 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f840806cc90 0x7f84081cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f840806cc90 0x7f84081cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47648/0 (socket says 172.21.15.47:47648) 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 -- 172.21.15.47:0/3501713418 learned_addr learned my addr 172.21.15.47:0/3501713418 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): get_auth_request con 0x7f840806cc90 auth_method 0 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): get_auth_request con 0x7f84081380d0 auth_method 0 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840cf90700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84080a3960 0x7f8408134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840cf90700 10 monclient(hunting): get_auth_request con 0x7f84080a3960 auth_method 0 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840cf90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840cf90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840cf90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840df92700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient(hunting): handle_auth_done global_id 34173 payload 824 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 msgr2=0x7f84081350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f84081380d0 0x7f84081350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84080a3960 msgr2=0x7f8408134bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84080a3960 0x7f8408134bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 monclient: found mon.noname-a 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400009550 con 0x7f840806cc90 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840cf90700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f84080a3960 0x7f8408134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f840d791700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f840806cc90 0x7f84081cc550 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f840400b370 tx=0x7f8404004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f83fdffb700 10 client.?.objecter ms_handle_connect 0x7f840806cc90 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.538+0000 7f83fdffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f84040048f0 con 0x7f840806cc90 2024-04-20T02:17:54.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: dump: 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.562 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8404012430 con 0x7f840806cc90 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 5 monclient: authenticate success, global_id 34173 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 10 monclient: _renew_subs 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8404009a70 con 0x7f840806cc90 2024-04-20T02:17:54.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8404012a10 con 0x7f840806cc90 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83ff7fe700 4 set_mon_vals no callback set 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 10 client.34173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f83fdffb700 10 monclient: dump: 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.564 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 10 monclient: _renew_subs 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8408121cf0 con 0x7f840806cc90 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.539+0000 7f840f9f5700 1 librados: init done 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f84040125a0 con 0x7f840806cc90 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840f9f5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 0x7f840811e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8408121e80 con 0x7f840806aac0 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f83ec03a2f0 0x7f83ec03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 0x7f840811e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7788+0+0 (secure 0 0 0) 0x7f840404c950 con 0x7f840806cc90 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient: get_auth_request con 0x7f840806aac0 auth_method 0 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 10 client.34173.objecter ms_dispatch 0x7f8408001550 osd_map(84..84 src has 1..84) v4 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 3 client.34173.objecter handle_osd_map got epochs [84,84] > 0 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 3 client.34173.objecter handle_osd_map decoding full epoch 84 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 20 client.34173.objecter dump_active .. 0 homeless 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840df92700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f83ec03a2f0 0x7f83ec03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840df92700 10 monclient: get_auth_request con 0x7f83ec03a2f0 auth_method 0 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 10 monclient(hunting): handle_auth_done global_id 23060 payload 824 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f840806e7a0 con 0x7f840806aac0 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 10 client.34173.objecter ms_handle_connect 0x7f840806aac0 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f83fdffb700 10 client.34173.objecter resend_mon_ops 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.540+0000 7f840cf90700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 0x7f840811e5e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f83f800f550 tx=0x7f83f800f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f83f8014ae0 con 0x7f840806aac0 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 10 monclient: dump: 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.568 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f840df92700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f83ec03a2f0 0x7f83ec03c7b0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f84081ccb20 tx=0x7f84000092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 10 client.34173.objecter ms_handle_connect 0x7f83ec03a2f0 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f83f8014c40 con 0x7f840806aac0 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 msgr2=0x7f840811e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 0x7f840811e5e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f83f800f550 tx=0x7f83f800f910 comp rx=0 tx=0).stop 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 msgr2=0x7f840811e5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.541+0000 7f83fdffb700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840806aac0 0x7f840811e5e0 secure :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f83f800f550 tx=0x7f83f800f910 comp rx=0 tx=0).stop 2024-04-20T02:17:54.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840f9f5700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 0x7f840811ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840f9f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8408068430 con 0x7f840811eb20 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 0x7f840811ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient: get_auth_request con 0x7f840811eb20 auth_method 0 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 10 monclient(hunting): handle_auth_done global_id 23069 payload 824 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 1 -- 172.21.15.47:0/3501713418 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f840806e1e0 con 0x7f840811eb20 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f840cf90700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 0x7f840811ef30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f83f800f550 tx=0x7f83f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f83fdffb700 10 client.34173.objecter ms_handle_connect 0x7f840811eb20 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f83fdffb700 10 client.34173.objecter resend_mon_ops 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f83f801dad0 con 0x7f840811eb20 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f83fdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f83fdffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.549+0000 7f83fdffb700 10 monclient: dump: 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.573 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.574 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f83f801dc30 con 0x7f840811eb20 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 msgr2=0x7f840811ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 0x7f840811ef30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f83f800f550 tx=0x7f83f8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 msgr2=0x7f840811ef30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f83fdffb700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f840811eb20 0x7f840811ef30 secure :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f83f800f550 tx=0x7f83f8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:54.576 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.577 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.578 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:54.579 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.580 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:54.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.588 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:54.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:54.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.597 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:54.597 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 10 librados: watch_flush enter 2024-04-20T02:17:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 10 librados: watch_flush exit 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 20 client.34173.objecter shutdown clearing up homeless session... 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 10 client.34173.objecter successfully canceled tick 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f83ec03a2f0 msgr2=0x7f83ec03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f83ec03a2f0 0x7f83ec03c7b0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f84081ccb20 tx=0x7f84000092c0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 10 monclient: shutdown 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f840806cc90 msgr2=0x7f84081cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 --2- 172.21.15.47:0/3501713418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f840806cc90 0x7f84081cc550 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f840400b370 tx=0x7f8404004ad0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840df92700 1 -- 172.21.15.47:0/3501713418 reap_dead start 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840df92700 1 -- 172.21.15.47:0/3501713418 reap_dead start 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 shutdown_connections 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 >> 172.21.15.47:0/3501713418 conn(0x7f84080a4a50 msgr2=0x7f8408070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 shutdown_connections 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 -- 172.21.15.47:0/3501713418 wait complete. 2024-04-20T02:17:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.550+0000 7f840f9f5700 1 librados: shutdown 2024-04-20T02:17:54.600 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:54.600 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:17:54.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:17:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.738+0000 7f0878f8e700 10 monclient: get_monmap_and_config 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.738+0000 7f0878f8e700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.738+0000 7f0878f8e700 10 monclient: monmap: 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:54.740068+0000 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:54.740068+0000 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.739+0000 7f0878f8e700 1 Processor -- start 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.739+0000 7f0878f8e700 1 -- start start 2024-04-20T02:17:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.739+0000 7f0878f8e700 10 monclient: init 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient: _add_conns ranks=[6,5,4,2,0,3,1,10,7,9,8] 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 0x7f0874135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): picked mon.noname-d con 0x7f087409fa80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f087409fe60 0x7f08740a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): picked mon.noname-h con 0x7f087409fe60 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08740a1f10 0x7f08740a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): picked mon.noname-c con 0x7f08740a1f10 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08740a0780 con 0x7f08740a1f10 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f087409df90 con 0x7f087409fe60 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f087409dcb0 con 0x7f087409fa80 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0878f8e700 10 monclient(hunting): authenticate will time out at 1754.141429s 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 0x7f0874135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08740a1f10 0x7f08740a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 0x7f0874135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50086/0 (socket says 172.21.15.47:50086) 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 1 -- 172.21.15.47:0/3034431542 learned_addr learned my addr 172.21.15.47:0/3034431542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): get_auth_request con 0x7f087409fa80 auth_method 0 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): get_auth_request con 0x7f08740a1f10 auth_method 0 2024-04-20T02:17:54.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f087409fe60 0x7f08740a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): get_auth_request con 0x7f087409fe60 auth_method 0 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0871d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient(hunting): handle_auth_done global_id 22963 payload 274 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 -- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 msgr2=0x7f0874135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 0x7f0874135df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 -- 172.21.15.47:0/3034431542 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f087409fe60 msgr2=0x7f08740a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f087409fe60 0x7f08740a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 monclient: found mon.noname-c 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f0872d9d700 1 -- 172.21.15.47:0/3034431542 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740751e0 con 0x7f08740a1f10 2024-04-20T02:17:54.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.740+0000 7f087259c700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 0x7f0874135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0872d9d700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08740a1f10 0x7f08740a2350 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f086c009dc0 tx=0x7f086c00b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b2ded0f73959d2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 1 -- 172.21.15.47:0/3034431542 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f086c00e070 con 0x7f08740a1f10 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: dump: 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.756 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 5 monclient: authenticate success, global_id 22963 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 1 -- 172.21.15.47:0/3034431542 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c0098f0 con 0x7f08740a1f10 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:54.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 4 set_mon_vals no callback set 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 1 -- 172.21.15.47:0/3034431542 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f086c0128a0 con 0x7f08740a1f10 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f087159a700 10 monclient: dump: 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.758 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 10 monclient: shutdown 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 1 -- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08740a1f10 msgr2=0x7f08740a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.741+0000 7f0878f8e700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08740a1f10 0x7f08740a2350 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f086c009dc0 tx=0x7f086c00b330 comp rx=0 tx=0).stop 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 -- 172.21.15.47:0/3034431542 shutdown_connections 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f08740a1f10 0x7f08740a2350 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f087409fa80 0x7f0874135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 --2- 172.21.15.47:0/3034431542 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f087409fe60 0x7f08740a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 -- 172.21.15.47:0/3034431542 >> 172.21.15.47:0/3034431542 conn(0x7f08740a3000 msgr2=0x7f08740a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 -- 172.21.15.47:0/3034431542 shutdown_connections 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 1 -- 172.21.15.47:0/3034431542 wait complete. 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.742+0000 7f0878f8e700 10 monclient: monmap: 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 librados: starting msgr at 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 librados: starting objecter 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 Processor -- start 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 -- start start 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 librados: setting wanted keys 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 librados: calling monclient init 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient: init 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient: _add_conns ranks=[3,0,7,2,1,6,8,5,4] 2024-04-20T02:17:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f087409fa80 0x7f087406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): picked mon.noname-g con 0x7f087409fa80 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f087409fe60 0x7f087406fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): picked mon.noname-a con 0x7f087409fe60 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f08740a1f10 0x7f0874070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): picked mon.noname-i con 0x7f08740a1f10 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087409d010 con 0x7f087409fe60 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740710a0 con 0x7f087409fa80 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0874072bd0 con 0x7f08740a1f10 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0878f8e700 10 monclient(hunting): authenticate will time out at 1754.144827s 2024-04-20T02:17:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0871d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f087409fe60 0x7f087406fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.743+0000 7f0871d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f087409fe60 0x7f087406fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47662/0 (socket says 172.21.15.47:47662) 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 -- 172.21.15.47:0/4041556378 learned_addr learned my addr 172.21.15.47:0/4041556378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): get_auth_request con 0x7f087409fe60 auth_method 0 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f08740a1f10 0x7f0874070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f087409fa80 0x7f087406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): get_auth_request con 0x7f08740a1f10 auth_method 0 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): get_auth_request con 0x7f087409fa80 auth_method 0 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0872d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient(hunting): handle_auth_done global_id 34182 payload 824 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f08740a1f10 msgr2=0x7f0874070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f08740a1f10 0x7f0874070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f087409fa80 msgr2=0x7f087406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f087409fa80 0x7f087406f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 monclient: found mon.noname-a 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009550 con 0x7f087409fe60 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0871d9b700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f087409fe60 0x7f087406fd00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f086400e9b0 tx=0x7f086400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 client.?.objecter ms_handle_connect 0x7f087409fe60 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0864009710 con 0x7f087409fe60 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: dump: 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 5 monclient: authenticate success, global_id 34182 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086400ee50 con 0x7f087409fe60 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 10 monclient: _renew_subs 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c009a70 con 0x7f087409fe60 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f086400ccc0 con 0x7f087409fe60 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0870d99700 4 set_mon_vals no callback set 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 10 client.34182.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0862ffd700 10 monclient: dump: 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.768 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 10 monclient: _renew_subs 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0874071980 con 0x7f087409fe60 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.744+0000 7f0878f8e700 1 librados: init done 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f086400c850 con 0x7f087409fe60 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f0878f8e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f0878f8e700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 0x7f087412f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0874068180 con 0x7f087406aac0 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f0862ffd700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f085403a2f0 0x7f085403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.745+0000 7f087259c700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 0x7f087412f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7788+0+0 (secure 0 0 0) 0x7f086404ae20 con 0x7f087409fe60 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 client.34182.objecter ms_dispatch 0x7f0874001550 osd_map(84..84 src has 1..84) v4 2024-04-20T02:17:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 3 client.34182.objecter handle_osd_map got epochs [84,84] > 0 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 3 client.34182.objecter handle_osd_map decoding full epoch 84 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient: get_auth_request con 0x7f087406aac0 auth_method 0 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0872d9d700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f085403a2f0 0x7f085403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 20 client.34182.objecter dump_active .. 0 homeless 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0872d9d700 10 monclient: get_auth_request con 0x7f085403a2f0 auth_method 0 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 10 monclient(hunting): handle_auth_done global_id 22972 payload 824 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f087409e550 con 0x7f087406aac0 2024-04-20T02:17:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f087259c700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 0x7f087412f9d0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0868011bd0 tx=0x7f0868011f90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 client.34182.objecter ms_handle_connect 0x7f087406aac0 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 client.34182.objecter resend_mon_ops 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f086800fe50 con 0x7f087406aac0 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 monclient: dump: 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0872d9d700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f085403a2f0 0x7f085403c7b0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f086c009dc0 tx=0x7f086c011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 client.34182.objecter ms_handle_connect 0x7f085403a2f0 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f086800f9e0 con 0x7f087406aac0 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 msgr2=0x7f087412f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 0x7f087412f9d0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0868011bd0 tx=0x7f0868011f90 comp rx=0 tx=0).stop 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.746+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 msgr2=0x7f087412f9d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.747+0000 7f0862ffd700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087406aac0 0x7f087412f9d0 secure :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0868011bd0 tx=0x7f0868011f90 comp rx=0 tx=0).stop 2024-04-20T02:17:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f0878f8e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f0878f8e700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 0x7f0874130320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f0878f8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f087406a400 con 0x7f087412ff10 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f087259c700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 0x7f0874130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f087259c700 10 monclient: get_auth_request con 0x7f087412ff10 auth_method 0 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f087259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f087259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.754+0000 7f087259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f087259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f087259c700 10 monclient(hunting): handle_auth_done global_id 22981 payload 824 2024-04-20T02:17:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f087259c700 1 -- 172.21.15.47:0/4041556378 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f08740668e0 con 0x7f087412ff10 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f087259c700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 0x7f0874130320 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0868011bd0 tx=0x7f0868004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 client.34182.objecter ms_handle_connect 0x7f087412ff10 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 client.34182.objecter resend_mon_ops 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f086802c8b0 con 0x7f087412ff10 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 monclient: dump: 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f086802c440 con 0x7f087412ff10 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 msgr2=0x7f0874130320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 0x7f0874130320 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0868011bd0 tx=0x7f0868004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 msgr2=0x7f0874130320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.755+0000 7f0862ffd700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f087412ff10 0x7f0874130320 secure :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0868011bd0 tx=0x7f0868004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:54.778 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:54.781 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.782 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.786 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:54.788 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.790 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.792 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.794 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:54.796 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.798 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.799 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.800 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:54.800 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 10 librados: watch_flush enter 2024-04-20T02:17:54.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 10 librados: watch_flush exit 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 20 client.34182.objecter shutdown clearing up homeless session... 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 10 client.34182.objecter successfully canceled tick 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f085403a2f0 msgr2=0x7f085403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f085403a2f0 0x7f085403c7b0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f086c009dc0 tx=0x7f086c011000 comp rx=0 tx=0).stop 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 10 monclient: shutdown 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f087409fe60 msgr2=0x7f087406fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 --2- 172.21.15.47:0/4041556378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f087409fe60 0x7f087406fd00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f086400e9b0 tx=0x7f086400ecc0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0872d9d700 1 -- 172.21.15.47:0/4041556378 reap_dead start 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0872d9d700 1 -- 172.21.15.47:0/4041556378 reap_dead start 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 shutdown_connections 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 >> 172.21.15.47:0/4041556378 conn(0x7f08740a3000 msgr2=0x7f0874132d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 shutdown_connections 2024-04-20T02:17:54.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 -- 172.21.15.47:0/4041556378 wait complete. 2024-04-20T02:17:54.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.756+0000 7f0878f8e700 1 librados: shutdown 2024-04-20T02:17:54.803 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:54.803 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:17:54.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.941+0000 7ff6bff42700 10 monclient: get_monmap_and_config 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.941+0000 7ff6bff42700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.941+0000 7ff6bff42700 10 monclient: monmap: 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:54.942816+0000 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:54.942816+0000 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:54.942 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.942+0000 7ff6bff42700 1 Processor -- start 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.942+0000 7ff6bff42700 1 -- start start 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.942+0000 7ff6bff42700 10 monclient: init 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.942+0000 7ff6bff42700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient: _add_conns ranks=[5,10,7,2,8,3,9,0,1,4,6] 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff6b80716f0 0x7ff6b8070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): picked mon.noname-h con 0x7ff6b80716f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6b8071ad0 0x7ff6b80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): picked mon.noname-f con 0x7ff6b8071ad0 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:54.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 0x7ff6b80a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): picked mon.noname-i con 0x7ff6b8071f30 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b80712e0 con 0x7ff6b8071ad0 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b806e740 con 0x7ff6b80716f0 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b806cce0 con 0x7ff6b8071f30 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bff42700 10 monclient(hunting): authenticate will time out at 1754.344097s 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 0x7ff6b80a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6b8071ad0 0x7ff6b80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff6b80716f0 0x7ff6b8070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 0x7ff6b80a3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:41132/0 (socket says 172.21.15.47:41132) 2024-04-20T02:17:54.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 -- 172.21.15.47:0/3580235106 learned_addr learned my addr 172.21.15.47:0/3580235106 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6b8071ad0 0x7ff6b80a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:33254/0 (socket says 172.21.15.47:33254) 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): get_auth_request con 0x7ff6b8071f30 auth_method 0 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff6b80716f0 0x7ff6b8070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:53348/0 (socket says 172.21.15.47:53348) 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): get_auth_request con 0x7ff6b80716f0 auth_method 0 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): get_auth_request con 0x7ff6b8071ad0 auth_method 0 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient(hunting): handle_auth_done global_id 20824 payload 274 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 -- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff6b80716f0 msgr2=0x7ff6b8070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 --2- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff6b80716f0 0x7ff6b8070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 -- 172.21.15.47:0/3580235106 >> v2:172.21.15.179:3300/0 conn(0x7ff6b8071ad0 msgr2=0x7ff6b80a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 --2- 172.21.15.47:0/3580235106 >> v2:172.21.15.179:3300/0 conn(0x7ff6b8071ad0 0x7ff6b80a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 monclient: found mon.noname-i 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.943+0000 7ff6be4df700 1 -- 172.21.15.47:0/3580235106 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b80751e0 con 0x7ff6b8071f30 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6be4df700 1 --2- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 0x7ff6b80a3e30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009e50 tx=0x7ff6b000b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=49af6a6a102c601 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 1 -- 172.21.15.47:0/3580235106 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b000e070 con 0x7ff6b8071f30 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: dump: 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 1 -- 172.21.15.47:0/3580235106 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b00099b0 con 0x7ff6b8071f30 2024-04-20T02:17:54.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 5 monclient: authenticate success, global_id 20824 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 1 -- 172.21.15.47:0/3580235106 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b000bd80 con 0x7ff6b8071f30 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 4 set_mon_vals no callback set 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bccdc700 10 monclient: dump: 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.959 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 10 monclient: shutdown 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 -- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 msgr2=0x7ff6b80a3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 --2- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 0x7ff6b80a3e30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009e50 tx=0x7ff6b000b330 comp rx=0 tx=0).stop 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 -- 172.21.15.47:0/3580235106 shutdown_connections 2024-04-20T02:17:54.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 --2- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff6b8071f30 0x7ff6b80a3e30 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 --2- 172.21.15.47:0/3580235106 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff6b80716f0 0x7ff6b8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 --2- 172.21.15.47:0/3580235106 >> v2:172.21.15.179:3300/0 conn(0x7ff6b8071ad0 0x7ff6b80a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 -- 172.21.15.47:0/3580235106 >> 172.21.15.47:0/3580235106 conn(0x7ff6b80a8510 msgr2=0x7ff6b80a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 -- 172.21.15.47:0/3580235106 shutdown_connections 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.944+0000 7ff6bff42700 1 -- 172.21.15.47:0/3580235106 wait complete. 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.945+0000 7ff6bff42700 10 monclient: build_initial_monmap 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.945+0000 7ff6bff42700 10 monclient: monmap: 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:54.961 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.945+0000 7ff6bff42700 1 librados: starting msgr at 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.945+0000 7ff6bff42700 1 librados: starting objecter 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.945+0000 7ff6bff42700 1 Processor -- start 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 -- start start 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 librados: setting wanted keys 2024-04-20T02:17:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 librados: calling monclient init 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient: init 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient: _add_conns ranks=[1,0,3,8,5,6,4,7,2] 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6b80716f0 0x7ff6b809da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): picked mon.noname-f con 0x7ff6b80716f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 0x7ff6b81beff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): picked mon.noname-a con 0x7ff6b8071ad0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6b8071f30 0x7ff6b81aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): picked mon.noname-g con 0x7ff6b8071f30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81ae410 con 0x7ff6b8071ad0 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81bf960 con 0x7ff6b80716f0 2024-04-20T02:17:54.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81cd770 con 0x7ff6b8071f30 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bff42700 10 monclient(hunting): authenticate will time out at 1754.347545s 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 0x7ff6b81beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 0x7ff6b81beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47670/0 (socket says 172.21.15.47:47670) 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 1 -- 172.21.15.47:0/1823740244 learned_addr learned my addr 172.21.15.47:0/1823740244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): get_auth_request con 0x7ff6b8071ad0 auth_method 0 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6b80716f0 0x7ff6b809da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): get_auth_request con 0x7ff6b80716f0 auth_method 0 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6be4df700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6b8071f30 0x7ff6b81aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6be4df700 10 monclient(hunting): get_auth_request con 0x7ff6b8071f30 auth_method 0 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6be4df700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6be4df700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6be4df700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.946+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 10 monclient(hunting): handle_auth_done global_id 34191 payload 824 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6b8071f30 msgr2=0x7ff6b81aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6b8071f30 0x7ff6b81aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6b80716f0 msgr2=0x7ff6b809da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6b80716f0 0x7ff6b809da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 10 monclient: _finish_hunting 0 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 monclient: found mon.noname-a 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009b30 con 0x7ff6b8071ad0 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6be4df700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6b8071f30 0x7ff6b81aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bd4dd700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 0x7ff6b81beff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e9b0 tx=0x7ff6a800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 client.?.objecter ms_handle_connect 0x7ff6b8071ad0 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6a8009710 con 0x7ff6b8071ad0 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: dump: 2024-04-20T02:17:54.966 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: _finish_auth 0 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 5 monclient: authenticate success, global_id 34191 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a800ccc0 con 0x7ff6b8071ad0 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6a800ce30 con 0x7ff6b8071ad0 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 10 monclient: _renew_subs 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b80723f0 con 0x7ff6b8071ad0 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6af7fe700 4 set_mon_vals no callback set 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6ae7fc700 10 monclient: dump: 2024-04-20T02:17:54.968 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 10 client.34191.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 10 monclient: _renew_subs 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b81ae130 con 0x7ff6b8071ad0 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.947+0000 7ff6bff42700 1 librados: init done 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bff42700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bff42700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 0x7ff6b81abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80682f0 con 0x7ff6b81bf530 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff6a800c850 con 0x7ff6b8071ad0 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 0x7ff6b81abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient: get_auth_request con 0x7ff6b81bf530 auth_method 0 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff69c03a2f0 0x7ff69c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7788+0+0 (secure 0 0 0) 0x7ff6a804ae40 con 0x7ff6b8071ad0 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 10 client.34191.objecter ms_dispatch 0x7ff6b8001550 osd_map(84..84 src has 1..84) v4 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 3 client.34191.objecter handle_osd_map got epochs [84,84] > 0 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 3 client.34191.objecter handle_osd_map decoding full epoch 84 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 20 client.34191.objecter dump_active .. 0 homeless 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6be4df700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff69c03a2f0 0x7ff69c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6be4df700 10 monclient: get_auth_request con 0x7ff69c03a2f0 auth_method 0 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_done global_id 22470 payload 824 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6b806e3d0 con 0x7ff6b81bf530 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6bdcde700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 0x7ff6b81abf90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40119d0 tx=0x7ff6b4011d90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 10 client.34191.objecter ms_handle_connect 0x7ff6b81bf530 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.948+0000 7ff6ae7fc700 10 client.34191.objecter resend_mon_ops 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b40036a0 con 0x7ff6b81bf530 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 10 monclient: dump: 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 10 client.34191.objecter ms_handle_connect 0x7ff69c03a2f0 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6be4df700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff69c03a2f0 0x7ff69c03c7b0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b370 tx=0x7ff6b0009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff6b4004240 con 0x7ff6b81bf530 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 msgr2=0x7ff6b81abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 0x7ff6b81abf90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40119d0 tx=0x7ff6b4011d90 comp rx=0 tx=0).stop 2024-04-20T02:17:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 msgr2=0x7ff6b81abf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.949+0000 7ff6ae7fc700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b81bf530 0x7ff6b81abf90 secure :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40119d0 tx=0x7ff6b4011d90 comp rx=0 tx=0).stop 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bff42700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bff42700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 0x7ff6b81ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bff42700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b806a400 con 0x7ff6b806aac0 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 0x7ff6b81ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient: get_auth_request con 0x7ff6b806aac0 auth_method 0 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 10 monclient(hunting): handle_auth_done global_id 22479 payload 824 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 1 -- 172.21.15.47:0/1823740244 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff6b809ee40 con 0x7ff6b806aac0 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6ae7fc700 10 client.34191.objecter ms_handle_connect 0x7ff6b806aac0 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6ae7fc700 10 client.34191.objecter resend_mon_ops 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6bdcde700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 0x7ff6b81ac4d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff6b401b000 tx=0x7ff6b4004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b400fdd0 con 0x7ff6b806aac0 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6ae7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.957+0000 7ff6ae7fc700 10 monclient: dump: 2024-04-20T02:17:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7ff6b40283a0 con 0x7ff6b806aac0 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 msgr2=0x7ff6b81ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 0x7ff6b81ac4d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff6b401b000 tx=0x7ff6b4004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 msgr2=0x7ff6b81ac4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6ae7fc700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff6b806aac0 0x7ff6b81ac4d0 secure :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff6b401b000 tx=0x7ff6b4004750 comp rx=0 tx=0).stop 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:54.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.980 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.981 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:54.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:54.984 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:54.985 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.989 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:54.992 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.995 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:54.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:54.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.001 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:55.001 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:55.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 10 librados: watch_flush enter 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 10 librados: watch_flush exit 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 20 client.34191.objecter shutdown clearing up homeless session... 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 10 client.34191.objecter successfully canceled tick 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff69c03a2f0 msgr2=0x7ff69c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff69c03a2f0 0x7ff69c03c7b0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b370 tx=0x7ff6b0009000 comp rx=0 tx=0).stop 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 10 monclient: shutdown 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 msgr2=0x7ff6b81beff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 0x7ff6b81beff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e9b0 tx=0x7ff6a800ecc0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6be4df700 1 -- 172.21.15.47:0/1823740244 reap_dead start 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 shutdown_connections 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 --2- 172.21.15.47:0/1823740244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff6b8071ad0 0x7ff6b81beff0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 >> 172.21.15.47:0/1823740244 conn(0x7ff6b80a8510 msgr2=0x7ff6b80a4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 shutdown_connections 2024-04-20T02:17:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 -- 172.21.15.47:0/1823740244 wait complete. 2024-04-20T02:17:55.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:54.958+0000 7ff6bff42700 1 librados: shutdown 2024-04-20T02:17:55.004 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:55.004 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:17:55.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: seed 47037 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2000 ms) 2024-04-20T02:17:55.119 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2015 ms) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2041 ms) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2007 ms) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3903 ms) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11966 ms total) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4211 ms) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3995 ms) 2024-04-20T02:17:55.120 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5951 ms) 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3930 ms) 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2022 ms) 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.2.2-1252-g15eecde2/rpm/el8/BUILD/ceph-18.2.2-1252-g15eecde2/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi179-47037-7 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: waiting for snaps to purge 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18009 ms) 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38118 ms total) 2024-04-20T02:17:55.121 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (7 ms) 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (9252 ms) 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (9259 ms total) 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3025 ms) 2024-04-20T02:17:55.122 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.152+0000 7fd2546a0700 10 monclient: get_monmap_and_config 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.152+0000 7fd2546a0700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.152+0000 7fd2546a0700 10 monclient: monmap: 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:55.153800+0000 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:55.153800+0000 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:55.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.153+0000 7fd2546a0700 1 Processor -- start 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.153+0000 7fd2546a0700 1 -- start start 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.153+0000 7fd2546a0700 10 monclient: init 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient: _add_conns ranks=[2,10,7,3,0,1,4,6,5,9,8] 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 0x7fd24c0a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): picked mon.noname-b con 0x7fd24c066610 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd24c099e00 0x7fd24c0a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): picked mon.noname-f con 0x7fd24c099e00 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd24c0a2990 0x7fd24c13e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): picked mon.noname-i con 0x7fd24c0a2990 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c09ba30 con 0x7fd24c099e00 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0a3f70 con 0x7fd24c066610 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c09ced0 con 0x7fd24c0a2990 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd2546a0700 10 monclient(hunting): authenticate will time out at 1754.555562s 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 0x7fd24c0a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 0x7fd24c0a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39360/0 (socket says 172.21.15.47:39360) 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 1 -- 172.21.15.47:0/2050257347 learned_addr learned my addr 172.21.15.47:0/2050257347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): get_auth_request con 0x7fd24c066610 auth_method 0 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 1 --2- 172.21.15.47:0/2050257347 >> v2:172.21.15.179:3300/0 conn(0x7fd24c099e00 0x7fd24c0a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 1 --2- 172.21.15.47:0/2050257347 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd24c0a2990 0x7fd24c13e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): get_auth_request con 0x7fd24c099e00 auth_method 0 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): get_auth_request con 0x7fd24c0a2990 auth_method 0 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd25243c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd251c3b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.154+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 10 monclient(hunting): handle_auth_done global_id 23078 payload 274 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 -- 172.21.15.47:0/2050257347 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd24c0a2990 msgr2=0x7fd24c13e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 --2- 172.21.15.47:0/2050257347 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd24c0a2990 0x7fd24c13e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 -- 172.21.15.47:0/2050257347 >> v2:172.21.15.179:3300/0 conn(0x7fd24c099e00 msgr2=0x7fd24c0a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 --2- 172.21.15.47:0/2050257347 >> v2:172.21.15.179:3300/0 conn(0x7fd24c099e00 0x7fd24c0a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 monclient: found mon.noname-b 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 -- 172.21.15.47:0/2050257347 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0751e0 con 0x7fd24c066610 2024-04-20T02:17:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd251c3b700 1 --2- 172.21.15.47:0/2050257347 >> v2:172.21.15.179:3300/0 conn(0x7fd24c099e00 0x7fd24c0a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25243c700 1 --2- 172.21.15.47:0/2050257347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 0x7fd24c0a1f10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd2480098f0 tx=0x7fd248009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4a15e0229da805 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 1 -- 172.21.15.47:0/2050257347 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd248011aa0 con 0x7fd24c066610 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: dump: 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 1 -- 172.21.15.47:0/2050257347 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd248011c00 con 0x7fd24c066610 2024-04-20T02:17:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd2546a0700 5 monclient: authenticate success, global_id 23078 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd2546a0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd2546a0700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd2546a0700 4 set_mon_vals no callback set 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 1 -- 172.21.15.47:0/2050257347 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd248011d70 con 0x7fd24c066610 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.155+0000 7fd25143a700 10 monclient: dump: 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.166 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.156+0000 7fd2546a0700 10 monclient: shutdown 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.156+0000 7fd2546a0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.156+0000 7fd2546a0700 1 -- 172.21.15.47:0/2050257347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 msgr2=0x7fd24c0a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.156+0000 7fd2546a0700 1 --2- 172.21.15.47:0/2050257347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 0x7fd24c0a1f10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd2480098f0 tx=0x7fd248009c00 comp rx=0 tx=0).stop 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.157+0000 7fd2546a0700 1 -- 172.21.15.47:0/2050257347 shutdown_connections 2024-04-20T02:17:55.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.157+0000 7fd2546a0700 1 --2- 172.21.15.47:0/2050257347 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd24c0a2990 0x7fd24c13e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.157+0000 7fd2546a0700 1 --2- 172.21.15.47:0/2050257347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd24c066610 0x7fd24c0a1f10 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.157+0000 7fd2546a0700 1 --2- 172.21.15.47:0/2050257347 >> v2:172.21.15.179:3300/0 conn(0x7fd24c099e00 0x7fd24c0a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.157+0000 7fd2546a0700 1 -- 172.21.15.47:0/2050257347 >> 172.21.15.47:0/2050257347 conn(0x7fd24c0a3000 msgr2=0x7fd24c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.158+0000 7fd2546a0700 1 -- 172.21.15.47:0/2050257347 shutdown_connections 2024-04-20T02:17:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.158+0000 7fd2546a0700 1 -- 172.21.15.47:0/2050257347 wait complete. 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.159+0000 7fd2546a0700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.159+0000 7fd2546a0700 10 monclient: monmap: 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:55.176 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.159+0000 7fd2546a0700 1 librados: starting msgr at 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.159+0000 7fd2546a0700 1 librados: starting objecter 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 Processor -- start 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 -- start start 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 librados: setting wanted keys 2024-04-20T02:17:55.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 librados: calling monclient init 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient: init 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient: _add_conns ranks=[8,3,5,7,1,4,2,0,6] 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c066610 0x7fd24c098960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): picked mon.noname-e con 0x7fd24c066610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd24c0a2990 0x7fd24c1c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): picked mon.noname-g con 0x7fd24c0a2990 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd24c070d80 0x7fd24c1c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): picked mon.noname-h con 0x7fd24c070d80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0708b0 con 0x7fd24c0a2990 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c090710 con 0x7fd24c070d80 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c1cd020 con 0x7fd24c066610 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd2546a0700 10 monclient(hunting): authenticate will time out at 1754.561679s 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd24c0a2990 0x7fd24c1c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd24c0a2990 0x7fd24c1c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:37148/0 (socket says 172.21.15.47:37148) 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 1 -- 172.21.15.47:0/4189763140 learned_addr learned my addr 172.21.15.47:0/4189763140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 10 monclient(hunting): get_auth_request con 0x7fd24c0a2990 auth_method 0 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd251c3b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd252c3d700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd24c070d80 0x7fd24c1c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd252c3d700 10 monclient(hunting): get_auth_request con 0x7fd24c070d80 auth_method 0 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd252c3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd252c3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.160+0000 7fd252c3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c066610 0x7fd24c098960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): get_auth_request con 0x7fd24c066610 auth_method 0 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd25243c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 10 monclient(hunting): handle_auth_done global_id 20919 payload 824 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c066610 msgr2=0x7fd24c098960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c066610 0x7fd24c098960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd24c070d80 msgr2=0x7fd24c1c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd24c070d80 0x7fd24c1c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 monclient: found mon.noname-g 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0094a0 con 0x7fd24c0a2990 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd252c3d700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd24c070d80 0x7fd24c1c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd251c3b700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd24c0a2990 0x7fd24c1c9070 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ec10 tx=0x7fd23c00ef20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 client.?.objecter ms_handle_connect 0x7fd24c0a2990 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd23c00ccc0 con 0x7fd24c0a2990 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:17:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 monclient: dump: 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c00ce40 con 0x7fd24c0a2990 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd2546a0700 5 monclient: authenticate success, global_id 20919 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd2546a0700 10 monclient: _renew_subs 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd2546a0700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.161+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd248009550 con 0x7fd24c0a2990 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd242ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd2546a0700 10 client.20919.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd23c0138e0 con 0x7fd24c0a2990 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd2546a0700 10 monclient: _renew_subs 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd2546a0700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:17:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c09d0b0 con 0x7fd24c0a2990 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd2546a0700 1 librados: init done 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd250c39700 4 set_mon_vals no callback set 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd242ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd242ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd242ffd700 10 monclient: dump: 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.162+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd23c013470 con 0x7fd24c0a2990 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd242ffd700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd23003a2f0 0x7fd23003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd23c04ab90 con 0x7fd24c0a2990 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd242ffd700 10 client.20919.objecter ms_dispatch 0x7fd24c001550 osd_map(85..85 src has 1..85) v4 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd242ffd700 3 client.20919.objecter handle_osd_map got epochs [85,85] > 0 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd242ffd700 3 client.20919.objecter handle_osd_map decoding full epoch 85 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd25243c700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd23003a2f0 0x7fd23003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd2546a0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd242ffd700 20 client.20919.objecter dump_active .. 0 homeless 2024-04-20T02:17:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd2546a0700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 0x7fd24c1c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c068180 con 0x7fd24c099e00 2024-04-20T02:17:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd25243c700 10 monclient: get_auth_request con 0x7fd23003a2f0 auth_method 0 2024-04-20T02:17:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 0x7fd24c1c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient: get_auth_request con 0x7fd24c099e00 auth_method 0 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.163+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd252c3d700 10 monclient(hunting): handle_auth_done global_id 34145 payload 824 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd252c3d700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd24c0714e0 con 0x7fd24c099e00 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 client.20919.objecter ms_handle_connect 0x7fd24c099e00 2024-04-20T02:17:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 client.20919.objecter resend_mon_ops 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd252c3d700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 0x7fd24c1c9af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd244011460 tx=0x7fd244011820 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 client.20919.objecter ms_handle_connect 0x7fd23003a2f0 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd25243c700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd23003a2f0 0x7fd23003c7b0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fd248009840 tx=0x7fd248008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd24400e040 con 0x7fd24c099e00 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 monclient: dump: 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd24400f040 con 0x7fd24c099e00 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 msgr2=0x7fd24c1c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 0x7fd24c1c9af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd244011460 tx=0x7fd244011820 comp rx=0 tx=0).stop 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 msgr2=0x7fd24c1c9af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.164+0000 7fd242ffd700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c099e00 0x7fd24c1c9af0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd244011460 tx=0x7fd244011820 comp rx=0 tx=0).stop 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd2546a0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd2546a0700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 0x7fd24c098ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd2546a0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c06a400 con 0x7fd24c06aac0 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 0x7fd24c098ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient: get_auth_request con 0x7fd24c06aac0 auth_method 0 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 10 monclient(hunting): handle_auth_done global_id 34154 payload 824 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 1 -- 172.21.15.47:0/4189763140 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd24c0669f0 con 0x7fd24c06aac0 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd252c3d700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 0x7fd24c098ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd244011460 tx=0x7fd244004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd242ffd700 10 client.20919.objecter ms_handle_connect 0x7fd24c06aac0 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd242ffd700 10 client.20919.objecter resend_mon_ops 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd244023070 con 0x7fd24c06aac0 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd242ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd242ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.173+0000 7fd242ffd700 10 monclient: dump: 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.192 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fd244004000 con 0x7fd24c06aac0 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 msgr2=0x7fd24c098ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 0x7fd24c098ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd244011460 tx=0x7fd244004ad0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 msgr2=0x7fd24c098ea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd242ffd700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd24c06aac0 0x7fd24c098ea0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd244011460 tx=0x7fd244004ad0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.193 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:55.194 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:17:55.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.196 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:55.197 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:55.199 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:55.200 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:55.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.204 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:55.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:55.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:17:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:55.217 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:55.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 10 librados: watch_flush enter 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 10 librados: watch_flush exit 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 20 client.20919.objecter shutdown clearing up homeless session... 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 10 client.20919.objecter successfully canceled tick 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd23003a2f0 msgr2=0x7fd23003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd23003a2f0 0x7fd23003c7b0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fd248009840 tx=0x7fd248008030 comp rx=0 tx=0).stop 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 10 monclient: shutdown 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd24c0a2990 msgr2=0x7fd24c1c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 --2- 172.21.15.47:0/4189763140 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd24c0a2990 0x7fd24c1c9070 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ec10 tx=0x7fd23c00ef20 comp rx=0 tx=0).stop 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd252c3d700 1 -- 172.21.15.47:0/4189763140 reap_dead start 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd252c3d700 1 -- 172.21.15.47:0/4189763140 reap_dead start 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 shutdown_connections 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 >> 172.21.15.47:0/4189763140 conn(0x7fd24c0a3000 msgr2=0x7fd24c1bfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 shutdown_connections 2024-04-20T02:17:55.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 -- 172.21.15.47:0/4189763140 wait complete. 2024-04-20T02:17:55.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.174+0000 7fd2546a0700 1 librados: shutdown 2024-04-20T02:17:55.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:55.220 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:17:55.220 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.359+0000 7f628eb57700 10 monclient: get_monmap_and_config 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.359+0000 7f628eb57700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.359+0000 7f628eb57700 10 monclient: monmap: 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:55.360845+0000 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:55.360845+0000 2024-04-20T02:17:55.359 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:55.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.360+0000 7f628eb57700 1 Processor -- start 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.360+0000 7f628eb57700 1 -- start start 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.360+0000 7f628eb57700 10 monclient: init 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.360+0000 7f628eb57700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient: _add_conns ranks=[2,8,4,5,0,10,3,1,6,7,9] 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 0x7f62880a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): picked mon.noname-b con 0x7f62880a1510 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 0x7f62880a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): picked mon.noname-e con 0x7f628809f9a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f628809fe00 0x7f62880a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): picked mon.noname-c con 0x7f628809fe00 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62880720a0 con 0x7f62880a1510 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6288070640 con 0x7f628809fe00 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6288071d30 con 0x7f628809f9a0 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628eb57700 10 monclient(hunting): authenticate will time out at 1754.762109s 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 0x7f62880a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 0x7f62880a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 0x7f62880a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39370/0 (socket says 172.21.15.47:39370) 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 1 -- 172.21.15.47:0/3863907206 learned_addr learned my addr 172.21.15.47:0/3863907206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): get_auth_request con 0x7f62880a1510 auth_method 0 2024-04-20T02:17:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 0x7f62880a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39822/0 (socket says 172.21.15.47:39822) 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): get_auth_request con 0x7f628809f9a0 auth_method 0 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f628809fe00 0x7f62880a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): get_auth_request con 0x7f628809fe00 auth_method 0 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628d0f4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient(hunting): handle_auth_done global_id 34163 payload 274 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 -- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f628809fe00 msgr2=0x7f62880a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f628809fe00 0x7f62880a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 -- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 msgr2=0x7f62880a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 0x7f62880a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 monclient: found mon.noname-e 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 -- 172.21.15.47:0/3863907206 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880751e0 con 0x7f628809f9a0 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f628c8f3700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 0x7f62880a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.361+0000 7f6287fff700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 0x7f62880a38f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6274009870 tx=0x7f6274009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=24af95677435cf59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 1 -- 172.21.15.47:0/3863907206 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6274011aa0 con 0x7f628809f9a0 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: dump: 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 1 -- 172.21.15.47:0/3863907206 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6274011c00 con 0x7f628809f9a0 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 5 monclient: authenticate success, global_id 34163 2024-04-20T02:17:55.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 1 -- 172.21.15.47:0/3863907206 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6274011d70 con 0x7f628809f9a0 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 4 set_mon_vals no callback set 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f62877fe700 10 monclient: dump: 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 10 monclient: shutdown 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 -- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 msgr2=0x7f62880a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 0x7f62880a38f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6274009870 tx=0x7f6274009b80 comp rx=0 tx=0).stop 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 -- 172.21.15.47:0/3863907206 shutdown_connections 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f628809fe00 0x7f62880a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f62880a1510 0x7f62880a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 --2- 172.21.15.47:0/3863907206 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f628809f9a0 0x7f62880a38f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 -- 172.21.15.47:0/3863907206 >> 172.21.15.47:0/3863907206 conn(0x7f62880a8510 msgr2=0x7f62880a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 -- 172.21.15.47:0/3863907206 shutdown_connections 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.362+0000 7f628eb57700 1 -- 172.21.15.47:0/3863907206 wait complete. 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 10 monclient: monmap: 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 1 librados: starting msgr at 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 1 librados: starting objecter 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 1 Processor -- start 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 1 -- start start 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 1 librados: setting wanted keys 2024-04-20T02:17:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 1 librados: calling monclient init 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.363+0000 7f628eb57700 10 monclient: init 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient: _add_conns ranks=[2,3,5,6,0,4,1,8,7] 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f628809f9a0 0x7f62881cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): picked mon.noname-b con 0x7f628809f9a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f628809fe00 0x7f62881cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): picked mon.noname-g con 0x7f628809fe00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f62880a1510 0x7f62881ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): picked mon.noname-h con 0x7f62880a1510 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62881281f0 con 0x7f628809f9a0 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6288126400 con 0x7f628809fe00 2024-04-20T02:17:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6288127530 con 0x7f62880a1510 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628eb57700 10 monclient(hunting): authenticate will time out at 1754.765580s 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f628809f9a0 0x7f62881cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f628809f9a0 0x7f62881cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39384/0 (socket says 172.21.15.47:39384) 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 1 -- 172.21.15.47:0/68941469 learned_addr learned my addr 172.21.15.47:0/68941469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): get_auth_request con 0x7f628809f9a0 auth_method 0 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f628809fe00 0x7f62881cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628d0f4700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f62880a1510 0x7f62881ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): get_auth_request con 0x7f628809fe00 auth_method 0 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628d0f4700 10 monclient(hunting): get_auth_request con 0x7f62880a1510 auth_method 0 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628d0f4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628d0f4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628d0f4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 10 monclient(hunting): handle_auth_done global_id 23105 payload 824 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f62880a1510 msgr2=0x7f62881ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f62880a1510 0x7f62881ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.364+0000 7f628c8f3700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f628809fe00 msgr2=0x7f62881cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f628809fe00 0x7f62881cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 1 monclient: found mon.noname-b 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c009550 con 0x7f628809f9a0 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628d0f4700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f62880a1510 0x7f62881ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628c8f3700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f628809f9a0 0x7f62881cc540 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f627c00e9b0 tx=0x7f627c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 client.?.objecter ms_handle_connect 0x7f628809f9a0 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f627c00ccc0 con 0x7f628809f9a0 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: dump: 2024-04-20T02:17:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c00ce40 con 0x7f628809f9a0 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 5 monclient: authenticate success, global_id 23105 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 10 monclient: _renew_subs 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62740094a0 con 0x7f628809f9a0 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f627c0138e0 con 0x7f628809f9a0 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f6286ffd700 4 set_mon_vals no callback set 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f62857fa700 10 monclient: dump: 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.388 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 10 client.23105.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 10 monclient: _renew_subs 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62881358e0 con 0x7f628809f9a0 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.365+0000 7f628eb57700 1 librados: init done 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f628eb57700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f628eb57700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 0x7f62880a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62880682f0 con 0x7f628806aac0 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f627c013470 con 0x7f628809f9a0 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f626c03a2f0 0x7f626c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 0x7f62880a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient: get_auth_request con 0x7f628806aac0 auth_method 0 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f627c04a9f0 con 0x7f628809f9a0 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 10 client.23105.objecter ms_dispatch 0x7f6288001550 osd_map(85..85 src has 1..85) v4 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 3 client.23105.objecter handle_osd_map got epochs [85,85] > 0 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 3 client.23105.objecter handle_osd_map decoding full epoch 85 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f62857fa700 20 client.23105.objecter dump_active .. 0 homeless 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f628d0f4700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f626c03a2f0 0x7f626c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f628d0f4700 10 monclient: get_auth_request con 0x7f626c03a2f0 auth_method 0 2024-04-20T02:17:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 10 monclient(hunting): handle_auth_done global_id 25723 payload 824 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.366+0000 7f6287fff700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62880a1e30 con 0x7f628806aac0 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 client.23105.objecter ms_handle_connect 0x7f628806aac0 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 client.23105.objecter resend_mon_ops 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f6287fff700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 0x7f62880a27e0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f62880a8fa0 tx=0x7f6274004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 client.23105.objecter ms_handle_connect 0x7f626c03a2f0 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f628d0f4700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f626c03a2f0 0x7f626c03c7b0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f6288125620 tx=0x7f628000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6274020070 con 0x7f628806aac0 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 monclient: dump: 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6274004440 con 0x7f628806aac0 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 msgr2=0x7f62880a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 0x7f62880a27e0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f62880a8fa0 tx=0x7f6274004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 msgr2=0x7f62880a27e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.367+0000 7f62857fa700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f628806aac0 0x7f62880a27e0 secure :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f62880a8fa0 tx=0x7f6274004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f628eb57700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f628eb57700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 0x7f62880a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f628eb57700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628806a400 con 0x7f62880a2d20 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 0x7f62880a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient: get_auth_request con 0x7f62880a2d20 auth_method 0 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.375+0000 7f6287fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f6287fff700 10 monclient(hunting): handle_auth_done global_id 25732 payload 824 2024-04-20T02:17:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f6287fff700 1 -- 172.21.15.47:0/68941469 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6288070960 con 0x7f62880a2d20 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f6287fff700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 0x7f62880a3130 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f6274009790 tx=0x7f6274011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 client.23105.objecter ms_handle_connect 0x7f62880a2d20 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 client.23105.objecter resend_mon_ops 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f627402b2b0 con 0x7f62880a2d20 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 monclient: dump: 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f627402b890 con 0x7f62880a2d20 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 msgr2=0x7f62880a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 0x7f62880a3130 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f6274009790 tx=0x7f6274011000 comp rx=0 tx=0).stop 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 msgr2=0x7f62880a3130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.376+0000 7f62857fa700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62880a2d20 0x7f62880a3130 secure :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f6274009790 tx=0x7f6274011000 comp rx=0 tx=0).stop 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:55.398 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:55.399 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:55.400 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.401 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:55.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.403 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:55.406 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.408 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:55.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:55.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.414 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:55.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.418 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:55.419 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 10 librados: watch_flush enter 2024-04-20T02:17:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 10 librados: watch_flush exit 2024-04-20T02:17:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 20 client.23105.objecter shutdown clearing up homeless session... 2024-04-20T02:17:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 10 client.23105.objecter successfully canceled tick 2024-04-20T02:17:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f626c03a2f0 msgr2=0x7f626c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f626c03a2f0 0x7f626c03c7b0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f6288125620 tx=0x7f628000b040 comp rx=0 tx=0).stop 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 10 monclient: shutdown 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f628809f9a0 msgr2=0x7f62881cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 --2- 172.21.15.47:0/68941469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f628809f9a0 0x7f62881cc540 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f627c00e9b0 tx=0x7f627c00ecc0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628d0f4700 1 -- 172.21.15.47:0/68941469 reap_dead start 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628d0f4700 1 -- 172.21.15.47:0/68941469 reap_dead start 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 shutdown_connections 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 >> 172.21.15.47:0/68941469 conn(0x7f62880a8510 msgr2=0x7f62880a7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 shutdown_connections 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 -- 172.21.15.47:0/68941469 wait complete. 2024-04-20T02:17:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.377+0000 7f628eb57700 1 librados: shutdown 2024-04-20T02:17:55.423 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:55.423 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:17:55.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.560+0000 7fbecaec2700 10 monclient: get_monmap_and_config 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.560+0000 7fbecaec2700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.560+0000 7fbecaec2700 10 monclient: monmap: 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:55.561822+0000 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:55.561822+0000 2024-04-20T02:17:55.560 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:55.561 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.561+0000 7fbecaec2700 1 Processor -- start 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.561+0000 7fbecaec2700 1 -- start start 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.561+0000 7fbecaec2700 10 monclient: init 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient: _add_conns ranks=[4,8,2,9,10,3,7,6,5,0,1] 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 0x7fbec4137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): picked mon.noname-c con 0x7fbec40a50f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 0x7fbec4138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): picked mon.noname-e con 0x7fbec41378a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec40a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): picked mon.noname-b con 0x7fbec40a38f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40a54d0 con 0x7fbec40a38f0 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41387b0 con 0x7fbec40a50f0 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbec406da20 con 0x7fbec41378a0 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbecaec2700 10 monclient(hunting): authenticate will time out at 1754.963123s 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 0x7fbec4138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec40a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 0x7fbec4137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 0x7fbec4138270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39828/0 (socket says 172.21.15.47:39828) 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec40a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39396/0 (socket says 172.21.15.47:39396) 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 1 -- 172.21.15.47:0/2836450873 learned_addr learned my addr 172.21.15.47:0/2836450873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 0x7fbec4137d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:49878/0 (socket says 172.21.15.47:49878) 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): get_auth_request con 0x7fbec40a50f0 auth_method 0 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): get_auth_request con 0x7fbec41378a0 auth_method 0 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): get_auth_request con 0x7fbec40a38f0 auth_method 0 2024-04-20T02:17:55.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_done global_id 22999 payload 274 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 -- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 msgr2=0x7fbec4138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 0x7fbec4138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 -- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 msgr2=0x7fbec40a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec40a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 monclient: found mon.noname-c 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 -- 172.21.15.47:0/2836450873 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40751e0 con 0x7fbec40a50f0 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec3fff700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 0x7fbec4138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec945f700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec40a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.562+0000 7fbec8c5e700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 0x7fbec4137d30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009870 tx=0x7fbeb8009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f74d88c8296bf344 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 1 -- 172.21.15.47:0/2836450873 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb8011aa0 con 0x7fbec40a50f0 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: dump: 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.577 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 1 -- 172.21.15.47:0/2836450873 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb8011c00 con 0x7fbec40a50f0 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 5 monclient: authenticate success, global_id 22999 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 1 -- 172.21.15.47:0/2836450873 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb8011d70 con 0x7fbec40a50f0 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:55.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 4 set_mon_vals no callback set 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbec37fe700 10 monclient: dump: 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 10 monclient: shutdown 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 -- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 msgr2=0x7fbec4137d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 0x7fbec4137d30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009870 tx=0x7fbeb8009b80 comp rx=0 tx=0).stop 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 -- 172.21.15.47:0/2836450873 shutdown_connections 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec40a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbec40a50f0 0x7fbec4137d30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 --2- 172.21.15.47:0/2836450873 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec41378a0 0x7fbec4138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 -- 172.21.15.47:0/2836450873 >> 172.21.15.47:0/2836450873 conn(0x7fbec40a8510 msgr2=0x7fbec40a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 -- 172.21.15.47:0/2836450873 shutdown_connections 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.563+0000 7fbecaec2700 1 -- 172.21.15.47:0/2836450873 wait complete. 2024-04-20T02:17:55.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 10 monclient: monmap: 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.581 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 1 librados: starting msgr at 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 1 librados: starting objecter 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 1 Processor -- start 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 1 -- start start 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 1 librados: setting wanted keys 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 1 librados: calling monclient init 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.564+0000 7fbecaec2700 10 monclient: init 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient: _add_conns ranks=[2,7,8,3,0,1,4,5,6] 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec41cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): picked mon.noname-b con 0x7fbec40a38f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbec40a50f0 0x7fbec4136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): picked mon.noname-i con 0x7fbec40a50f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec4137100 0x7fbec4120370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): picked mon.noname-e con 0x7fbec4137100 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4123d90 con 0x7fbec40a38f0 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41cc8c0 con 0x7fbec40a50f0 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41229f0 con 0x7fbec4137100 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbecaec2700 10 monclient(hunting): authenticate will time out at 1754.966513s 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec4137100 0x7fbec4120370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec41cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec4137100 0x7fbec4120370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39844/0 (socket says 172.21.15.47:39844) 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 1 -- 172.21.15.47:0/3112709788 learned_addr learned my addr 172.21.15.47:0/3112709788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): get_auth_request con 0x7fbec40a38f0 auth_method 0 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): get_auth_request con 0x7fbec4137100 auth_method 0 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec945f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec3fff700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbec40a50f0 0x7fbec4136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec3fff700 10 monclient(hunting): get_auth_request con 0x7fbec40a50f0 auth_method 0 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient(hunting): handle_auth_done global_id 23123 payload 824 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec4137100 msgr2=0x7fbec4120370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec4137100 0x7fbec4120370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbec40a50f0 msgr2=0x7fbec4136bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbec40a50f0 0x7fbec4136bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 monclient: found mon.noname-b 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.565+0000 7fbec8c5e700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc009a70 con 0x7fbec40a38f0 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec945f700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbec4137100 0x7fbec4120370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec8c5e700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec41cc380 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009840 tx=0x7fbeb8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 client.?.objecter ms_handle_connect 0x7fbec40a38f0 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb8004330 con 0x7fbec40a38f0 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: dump: 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.586 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb801f070 con 0x7fbec40a38f0 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 5 monclient: authenticate success, global_id 23123 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb80044d0 con 0x7fbec40a38f0 2024-04-20T02:17:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec2ffd700 4 set_mon_vals no callback set 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbec17fa700 10 monclient: dump: 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.588 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 10 monclient: _renew_subs 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb0009550 con 0x7fbec40a38f0 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 10 client.23123.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 10 monclient: _renew_subs 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb80094a0 con 0x7fbec40a38f0 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.566+0000 7fbecaec2700 1 librados: init done 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbecaec2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbecaec2700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 0x7fbec41208b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4075760 con 0x7fbec406aac0 2024-04-20T02:17:55.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbeb8021030 con 0x7fbec40a38f0 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 0x7fbec41208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient: get_auth_request con 0x7fbec406aac0 auth_method 0 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbea803a2f0 0x7fbea803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbeb804bb50 con 0x7fbec40a38f0 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 10 client.23123.objecter ms_dispatch 0x7fbec4001550 osd_map(85..85 src has 1..85) v4 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 3 client.23123.objecter handle_osd_map got epochs [85,85] > 0 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 3 client.23123.objecter handle_osd_map decoding full epoch 85 2024-04-20T02:17:55.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec17fa700 20 client.23123.objecter dump_active .. 0 homeless 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec945f700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbea803a2f0 0x7fbea803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec945f700 10 monclient: get_auth_request con 0x7fbea803a2f0 auth_method 0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 10 monclient(hunting): handle_auth_done global_id 20946 payload 824 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.567+0000 7fbec3fff700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbec406e370 con 0x7fbec406aac0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 client.23123.objecter ms_handle_connect 0x7fbec406aac0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 client.23123.objecter resend_mon_ops 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec3fff700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 0x7fbec41208b0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fbec409edc0 tx=0x7fbeb0011b90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb00044a0 con 0x7fbec406aac0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec945f700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbea803a2f0 0x7fbea803c7b0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7fbec41232c0 tx=0x7fbebc00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 monclient: dump: 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.591 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 client.23123.objecter ms_handle_connect 0x7fbea803a2f0 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbeb0004030 con 0x7fbec406aac0 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:55.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 msgr2=0x7fbec41208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 0x7fbec41208b0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fbec409edc0 tx=0x7fbeb0011b90 comp rx=0 tx=0).stop 2024-04-20T02:17:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 msgr2=0x7fbec41208b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.568+0000 7fbec17fa700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec406aac0 0x7fbec41208b0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbecaec2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbecaec2700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 0x7fbec4120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbecaec2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40682f0 con 0x7fbec4137830 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 0x7fbec4120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient: get_auth_request con 0x7fbec4137830 auth_method 0 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.576+0000 7fbec3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec3fff700 10 monclient(hunting): handle_auth_done global_id 20955 payload 824 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec3fff700 1 -- 172.21.15.47:0/3112709788 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbec406e650 con 0x7fbec4137830 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec3fff700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 0x7fbec4120df0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbeb001b000 tx=0x7fbeb0004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 client.23123.objecter ms_handle_connect 0x7fbec4137830 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 client.23123.objecter resend_mon_ops 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbeb0005e90 con 0x7fbec4137830 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 monclient: dump: 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.596 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fbeb0015d10 con 0x7fbec4137830 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 msgr2=0x7fbec4120df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 0x7fbec4120df0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbeb001b000 tx=0x7fbeb0004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 msgr2=0x7fbec4120df0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.577+0000 7fbec17fa700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbec4137830 0x7fbec4120df0 secure :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbeb001b000 tx=0x7fbeb0004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:55.598 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.599 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.600 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:55.601 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.602 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:55.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.606 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:55.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:55.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.612 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.614 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.618 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:55.619 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 10 librados: watch_flush enter 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 10 librados: watch_flush exit 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 20 client.23123.objecter shutdown clearing up homeless session... 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 10 client.23123.objecter successfully canceled tick 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbea803a2f0 msgr2=0x7fbea803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbea803a2f0 0x7fbea803c7b0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7fbec41232c0 tx=0x7fbebc00b040 comp rx=0 tx=0).stop 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 10 monclient: shutdown 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 msgr2=0x7fbec41cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 --2- 172.21.15.47:0/3112709788 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbec40a38f0 0x7fbec41cc380 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009840 tx=0x7fbeb8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbec945f700 1 -- 172.21.15.47:0/3112709788 reap_dead start 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbec945f700 1 -- 172.21.15.47:0/3112709788 reap_dead start 2024-04-20T02:17:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 shutdown_connections 2024-04-20T02:17:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 >> 172.21.15.47:0/3112709788 conn(0x7fbec40a8510 msgr2=0x7fbec4070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 shutdown_connections 2024-04-20T02:17:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 -- 172.21.15.47:0/3112709788 wait complete. 2024-04-20T02:17:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.578+0000 7fbecaec2700 1 librados: shutdown 2024-04-20T02:17:55.622 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:55.622 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:17:55.622 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.761+0000 7fba8153d700 10 monclient: get_monmap_and_config 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.761+0000 7fba8153d700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.761+0000 7fba8153d700 10 monclient: monmap: 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:55.762542+0000 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:55.762542+0000 2024-04-20T02:17:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 Processor -- start 2024-04-20T02:17:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 -- start start 2024-04-20T02:17:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient: init 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient: _add_conns ranks=[9,8,5,6,3,2,10,7,4,1,0] 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba7c137d90 0x7fba7c0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): picked mon.noname-a con 0x7fba7c137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 0x7fba7c0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): picked mon.noname-e con 0x7fba7c138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c1385b0 0x7fba7c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): picked mon.noname-h con 0x7fba7c1385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7c09fa40 con 0x7fba7c137d90 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c09dfc0 con 0x7fba7c1385b0 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0a7bc0 con 0x7fba7c138170 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba8153d700 10 monclient(hunting): authenticate will time out at 1755.163667s 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 0x7fba7c0a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 0x7fba7c0a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39846/0 (socket says 172.21.15.47:39846) 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 1 -- 172.21.15.47:0/2916171821 learned_addr learned my addr 172.21.15.47:0/2916171821 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba7c137d90 0x7fba7c0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): get_auth_request con 0x7fba7c138170 auth_method 0 2024-04-20T02:17:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7affd700 10 monclient(hunting): get_auth_request con 0x7fba7c137d90 auth_method 0 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7b7fe700 1 --2- 172.21.15.47:0/2916171821 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c1385b0 0x7fba7c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.762+0000 7fba7b7fe700 10 monclient(hunting): get_auth_request con 0x7fba7c1385b0 auth_method 0 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_done global_id 34190 payload 274 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 -- 172.21.15.47:0/2916171821 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c1385b0 msgr2=0x7fba7c0a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 --2- 172.21.15.47:0/2916171821 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c1385b0 0x7fba7c0a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 -- 172.21.15.47:0/2916171821 >> v2:172.21.15.47:3300/0 conn(0x7fba7c137d90 msgr2=0x7fba7c0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 --2- 172.21.15.47:0/2916171821 >> v2:172.21.15.47:3300/0 conn(0x7fba7c137d90 0x7fba7c0a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 monclient: found mon.noname-e 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 -- 172.21.15.47:0/2916171821 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c0751e0 con 0x7fba7c138170 2024-04-20T02:17:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba7a7fc700 1 --2- 172.21.15.47:0/2916171821 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 0x7fba7c0a5c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba70009920 tx=0x7fba70009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=127e4949f8026203 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 1 -- 172.21.15.47:0/2916171821 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba70011aa0 con 0x7fba7c138170 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: dump: 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba8153d700 5 monclient: authenticate success, global_id 34190 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 1 -- 172.21.15.47:0/2916171821 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70011c00 con 0x7fba7c138170 2024-04-20T02:17:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba8153d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba8153d700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 1 -- 172.21.15.47:0/2916171821 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba70011d70 con 0x7fba7c138170 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba8153d700 4 set_mon_vals no callback set 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.763+0000 7fba79ffb700 10 monclient: dump: 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 10 monclient: shutdown 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 -- 172.21.15.47:0/2916171821 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 msgr2=0x7fba7c0a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 --2- 172.21.15.47:0/2916171821 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 0x7fba7c0a5c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fba70009920 tx=0x7fba70009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 -- 172.21.15.47:0/2916171821 shutdown_connections 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 --2- 172.21.15.47:0/2916171821 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c1385b0 0x7fba7c0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 --2- 172.21.15.47:0/2916171821 >> v2:172.21.15.47:3300/0 conn(0x7fba7c137d90 0x7fba7c0a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 --2- 172.21.15.47:0/2916171821 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fba7c138170 0x7fba7c0a5c90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 -- 172.21.15.47:0/2916171821 >> 172.21.15.47:0/2916171821 conn(0x7fba7c0aa8b0 msgr2=0x7fba7c0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 -- 172.21.15.47:0/2916171821 shutdown_connections 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 1 -- 172.21.15.47:0/2916171821 wait complete. 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.764+0000 7fba8153d700 10 monclient: monmap: 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 librados: starting msgr at 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 librados: starting objecter 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 Processor -- start 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 -- start start 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 librados: setting wanted keys 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 librados: calling monclient init 2024-04-20T02:17:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 10 monclient: init 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 10 monclient: _add_conns ranks=[4,3,0,2,8,1,7,6,5] 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fba7c137d90 0x7fba7c1cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 10 monclient(hunting): picked mon.noname-c con 0x7fba7c137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.765+0000 7fba8153d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba7c138170 0x7fba7c136fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): picked mon.noname-g con 0x7fba7c138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c1385b0 0x7fba7c1c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): picked mon.noname-a con 0x7fba7c1385b0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c124a20 con 0x7fba7c1385b0 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1cccd0 con 0x7fba7c138170 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c09dc20 con 0x7fba7c137d90 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient(hunting): authenticate will time out at 1755.167048s 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fba7c137d90 0x7fba7c1cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c1385b0 0x7fba7c1c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fba7c137d90 0x7fba7c1cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:49884/0 (socket says 172.21.15.47:49884) 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 1 -- 172.21.15.47:0/104586613 learned_addr learned my addr 172.21.15.47:0/104586613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): get_auth_request con 0x7fba7c137d90 auth_method 0 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): get_auth_request con 0x7fba7c1385b0 auth_method 0 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba7c138170 0x7fba7c136fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): get_auth_request con 0x7fba7c138170 auth_method 0 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient(hunting): handle_auth_done global_id 34218 payload 824 2024-04-20T02:17:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fba7c137d90 msgr2=0x7fba7c1cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fba7c137d90 0x7fba7c1cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba7c138170 msgr2=0x7fba7c136fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fba7c138170 0x7fba7c136fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 monclient: found mon.noname-a 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70009550 con 0x7fba7c1385b0 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7affd700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fba7c137d90 0x7fba7c1cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba7b7fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c1385b0 0x7fba7c1c8d70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba7400eb70 tx=0x7fba7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 client.?.objecter ms_handle_connect 0x7fba7c1385b0 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba74009a80 con 0x7fba7c1385b0 2024-04-20T02:17:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: dump: 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 5 monclient: authenticate success, global_id 34218 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7400f040 con 0x7fba7c1385b0 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient: _renew_subs 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c09e9e0 con 0x7fba7c1385b0 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba7400cc10 con 0x7fba7c1385b0 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba8153d700 10 client.34218.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba797fa700 4 set_mon_vals no callback set 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.766+0000 7fba637fe700 10 monclient: dump: 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 10 monclient: _renew_subs 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c1240f0 con 0x7fba7c1385b0 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 1 librados: init done 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fba7400cd70 con 0x7fba7c1385b0 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 0x7fba7c1c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c068180 con 0x7fba7c06aac0 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.767+0000 7fba637fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fba6803a2f0 0x7fba6803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fba7404b050 con 0x7fba7c1385b0 2024-04-20T02:17:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 10 client.34218.objecter ms_dispatch 0x7fba7c001550 osd_map(85..85 src has 1..85) v4 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 3 client.34218.objecter handle_osd_map got epochs [85,85] > 0 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 3 client.34218.objecter handle_osd_map decoding full epoch 85 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 20 client.34218.objecter dump_active .. 0 homeless 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 0x7fba7c1c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7a7fc700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fba6803a2f0 0x7fba6803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7a7fc700 10 monclient: get_auth_request con 0x7fba6803a2f0 auth_method 0 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient: get_auth_request con 0x7fba7c06aac0 auth_method 0 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 10 monclient(hunting): handle_auth_done global_id 34187 payload 824 2024-04-20T02:17:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba7c09f760 con 0x7fba7c06aac0 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 10 client.34218.objecter ms_handle_connect 0x7fba6803a2f0 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7a7fc700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fba6803a2f0 0x7fba6803c7b0 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7fba70009920 tx=0x7fba70008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 10 client.34218.objecter ms_handle_connect 0x7fba7c06aac0 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba637fe700 10 client.34218.objecter resend_mon_ops 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.768+0000 7fba7affd700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 0x7fba7c1c92b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011db0 tx=0x7fba6c00f520 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba6c00fe20 con 0x7fba7c06aac0 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 10 monclient: dump: 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fba6c014440 con 0x7fba7c06aac0 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 msgr2=0x7fba7c1c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 0x7fba7c1c92b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011db0 tx=0x7fba6c00f520 comp rx=0 tx=0).stop 2024-04-20T02:17:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 msgr2=0x7fba7c1c92b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.769+0000 7fba637fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c06aac0 0x7fba7c1c92b0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011db0 tx=0x7fba6c00f520 comp rx=0 tx=0).stop 2024-04-20T02:17:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba8153d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba8153d700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 0x7fba7c1c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba8153d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c06a400 con 0x7fba7c137510 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 0x7fba7c1c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient: get_auth_request con 0x7fba7c137510 auth_method 0 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 10 monclient(hunting): handle_auth_done global_id 34196 payload 824 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.777+0000 7fba7affd700 1 -- 172.21.15.47:0/104586613 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fba7c124880 con 0x7fba7c137510 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba7affd700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 0x7fba7c1c97f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011db0 tx=0x7fba6c004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 client.34218.objecter ms_handle_connect 0x7fba7c137510 2024-04-20T02:17:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 client.34218.objecter resend_mon_ops 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba6c004020 con 0x7fba7c137510 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 monclient: dump: 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fba6c023040 con 0x7fba7c137510 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 msgr2=0x7fba7c1c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 0x7fba7c1c97f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011db0 tx=0x7fba6c004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 msgr2=0x7fba7c1c97f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba637fe700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fba7c137510 0x7fba7c1c97f0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011db0 tx=0x7fba6c004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.798 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:55.799 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.800 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.801 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:55.802 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:55.803 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:55.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.809 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.811 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:55.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.814 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:55.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:55.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:55.817 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:55.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:55.819 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba8153d700 10 librados: watch_flush enter 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.778+0000 7fba8153d700 10 librados: watch_flush exit 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 20 client.34218.objecter shutdown clearing up homeless session... 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 10 client.34218.objecter successfully canceled tick 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fba6803a2f0 msgr2=0x7fba6803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fba6803a2f0 0x7fba6803c7b0 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7fba70009920 tx=0x7fba70008030 comp rx=0 tx=0).stop 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 10 monclient: shutdown 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c1385b0 msgr2=0x7fba7c1c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 --2- 172.21.15.47:0/104586613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba7c1385b0 0x7fba7c1c8d70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba7400eb70 tx=0x7fba7400ee80 comp rx=0 tx=0).stop 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba7b7fe700 1 -- 172.21.15.47:0/104586613 reap_dead start 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba7b7fe700 1 -- 172.21.15.47:0/104586613 reap_dead start 2024-04-20T02:17:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 shutdown_connections 2024-04-20T02:17:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 >> 172.21.15.47:0/104586613 conn(0x7fba7c0aa8b0 msgr2=0x7fba7c0a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 shutdown_connections 2024-04-20T02:17:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 -- 172.21.15.47:0/104586613 wait complete. 2024-04-20T02:17:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.779+0000 7fba8153d700 1 librados: shutdown 2024-04-20T02:17:55.822 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:55.822 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:17:55.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.960+0000 7f831eb35700 10 monclient: get_monmap_and_config 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.960+0000 7f831eb35700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.960+0000 7f831eb35700 10 monclient: monmap: 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:55.961726+0000 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:55.961726+0000 2024-04-20T02:17:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:55.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 Processor -- start 2024-04-20T02:17:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 -- start start 2024-04-20T02:17:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient: init 2024-04-20T02:17:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient: _add_conns ranks=[5,9,7,0,3,10,4,6,1,2,8] 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f83180a0830 0x7f83180a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): picked mon.noname-h con 0x7f83180a0830 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83180a01b0 0x7f83180a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): picked mon.noname-a con 0x7f83180a01b0 addr v2:172.21.15.47:3300/0 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83180a1150 0x7f83180a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): picked mon.noname-i con 0x7f83180a1150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831809dee0 con 0x7f83180a01b0 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8318067820 con 0x7f83180a0830 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f831809eee0 con 0x7f83180a1150 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.961+0000 7f831eb35700 10 monclient(hunting): authenticate will time out at 1755.362997s 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83180a01b0 0x7f83180a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f83180a01b0 0x7f83180a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47694/0 (socket says 172.21.15.47:47694) 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 -- 172.21.15.47:0/528505199 learned_addr learned my addr 172.21.15.47:0/528505199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): get_auth_request con 0x7f83180a01b0 auth_method 0 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f83180a0830 0x7f83180a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831d0d2700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83180a1150 0x7f83180a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): get_auth_request con 0x7f83180a0830 auth_method 0 2024-04-20T02:17:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831d0d2700 10 monclient(hunting): get_auth_request con 0x7f83180a1150 auth_method 0 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831d0d2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831d0d2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831d0d2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient(hunting): handle_auth_done global_id 34227 payload 274 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 -- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83180a1150 msgr2=0x7f83180a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83180a1150 0x7f83180a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 -- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f83180a0830 msgr2=0x7f83180a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f83180a0830 0x7f83180a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 monclient: found mon.noname-a 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 -- 172.21.15.47:0/528505199 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180751e0 con 0x7f83180a01b0 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831d0d2700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83180a1150 0x7f83180a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f8317fff700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83180a01b0 0x7f83180a2950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8304009920 tx=0x7f8304009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322f279e22e80472 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 1 -- 172.21.15.47:0/528505199 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8304011aa0 con 0x7f83180a01b0 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: dump: 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 1 -- 172.21.15.47:0/528505199 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8304011c00 con 0x7f83180a01b0 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831eb35700 5 monclient: authenticate success, global_id 34227 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831eb35700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 1 -- 172.21.15.47:0/528505199 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8304011d70 con 0x7f83180a01b0 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831eb35700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f831eb35700 4 set_mon_vals no callback set 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.962+0000 7f83177fe700 10 monclient: dump: 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 10 monclient: shutdown 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 -- 172.21.15.47:0/528505199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83180a01b0 msgr2=0x7f83180a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83180a01b0 0x7f83180a2950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8304009920 tx=0x7f8304009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 -- 172.21.15.47:0/528505199 shutdown_connections 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83180a1150 0x7f83180a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f83180a0830 0x7f83180a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 --2- 172.21.15.47:0/528505199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f83180a01b0 0x7f83180a2950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 -- 172.21.15.47:0/528505199 >> 172.21.15.47:0/528505199 conn(0x7f83180a7570 msgr2=0x7f83180a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 -- 172.21.15.47:0/528505199 shutdown_connections 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.963+0000 7f831eb35700 1 -- 172.21.15.47:0/528505199 wait complete. 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 10 monclient: build_initial_monmap 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 10 monclient: monmap: 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 1 librados: starting msgr at 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 1 librados: starting objecter 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 1 Processor -- start 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 1 -- start start 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 1 librados: setting wanted keys 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 1 librados: calling monclient init 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.964+0000 7f831eb35700 10 monclient: init 2024-04-20T02:17:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient: _add_conns ranks=[6,1,2,5,0,3,4,8,7] 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f83180a01b0 0x7f83181cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): picked mon.noname-d con 0x7f83180a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83180a0830 0x7f83181bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): picked mon.noname-f con 0x7f83180a0830 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 0x7f83181bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): picked mon.noname-b con 0x7f83180a1150 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83181ab0e0 con 0x7f83180a0830 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83181a9fb0 con 0x7f83180a1150 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f831809e900 con 0x7f83180a01b0 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831eb35700 10 monclient(hunting): authenticate will time out at 1755.366515s 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 0x7f83181bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f83180a01b0 0x7f83181cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f83180a01b0 0x7f83181cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50116/0 (socket says 172.21.15.47:50116) 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 0x7f83181bc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39418/0 (socket says 172.21.15.47:39418) 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 1 -- 172.21.15.47:0/636581807 learned_addr learned my addr 172.21.15.47:0/636581807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): get_auth_request con 0x7f83180a01b0 auth_method 0 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): get_auth_request con 0x7f83180a1150 auth_method 0 2024-04-20T02:17:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83180a0830 0x7f83181bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): get_auth_request con 0x7f83180a0830 auth_method 0 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f8317fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.965+0000 7f831d0d2700 10 monclient(hunting): handle_auth_done global_id 23132 payload 824 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f83180a01b0 msgr2=0x7f83181cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f83180a01b0 0x7f83181cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83180a0830 msgr2=0x7f83181bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83180a0830 0x7f83181bbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 10 monclient: _finish_hunting 0 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 monclient: found mon.noname-b 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304009550 con 0x7f83180a1150 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831c8d1700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f83180a01b0 0x7f83181cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831d0d2700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 0x7f83181bc4c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8310007db0 tx=0x7f831000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 client.?.objecter ms_handle_connect 0x7f83180a1150 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8310003a20 con 0x7f83180a1150 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: dump: 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: _finish_auth 0 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: _check_auth_tickets 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831000f040 con 0x7f83180a1150 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8310003ba0 con 0x7f83180a1150 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 5 monclient: authenticate success, global_id 23132 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 10 monclient: _renew_subs 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831809e440 con 0x7f83180a1150 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f8316ffd700 4 set_mon_vals no callback set 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f83157fa700 10 monclient: dump: 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 10 client.23132.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 10 monclient: _renew_subs 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83181a95b0 con 0x7f83180a1150 2024-04-20T02:17:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.966+0000 7f831eb35700 1 librados: init done 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831eb35700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831eb35700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 0x7f83181a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8318068310 con 0x7f831806aac0 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8310003d00 con 0x7f83180a1150 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 0x7f83181a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f82fc03a220 0x7f82fc03c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient: get_auth_request con 0x7f831806aac0 auth_method 0 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f831004a4e0 con 0x7f83180a1150 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 10 client.23132.objecter ms_dispatch 0x7f8318001550 osd_map(85..85 src has 1..85) v4 2024-04-20T02:17:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 3 client.23132.objecter handle_osd_map got epochs [85,85] > 0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 3 client.23132.objecter handle_osd_map decoding full epoch 85 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f8317fff700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f82fc03a220 0x7f82fc03c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f83157fa700 20 client.23132.objecter dump_active .. 0 homeless 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f8317fff700 10 monclient: get_auth_request con 0x7f82fc03a220 auth_method 0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 10 monclient(hunting): handle_auth_done global_id 20887 payload 824 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.967+0000 7f831c8d1700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83180672f0 con 0x7f831806aac0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 client.23132.objecter ms_handle_connect 0x7f831806aac0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 client.23132.objecter resend_mon_ops 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f831c8d1700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 0x7f83181a7a20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f830c009f40 tx=0x7f830c00f520 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 client.23132.objecter ms_handle_connect 0x7f82fc03a220 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f8317fff700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f82fc03a220 0x7f82fc03c6e0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f8304009920 tx=0x7f8304008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f830c00fd80 con 0x7f831806aac0 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 monclient: dump: 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f830c014440 con 0x7f831806aac0 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 msgr2=0x7f83181a7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 0x7f83181a7a20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f830c009f40 tx=0x7f830c00f520 comp rx=0 tx=0).stop 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 msgr2=0x7f83181a7a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.968+0000 7f83157fa700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f831806aac0 0x7f83181a7a20 secure :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f830c009f40 tx=0x7f830c00f520 comp rx=0 tx=0).stop 2024-04-20T02:17:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831eb35700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:55.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831eb35700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 0x7f83181a8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831eb35700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f831806a400 con 0x7f83181a7f60 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831c8d1700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 0x7f83181a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831c8d1700 10 monclient: get_auth_request con 0x7f83181a7f60 auth_method 0 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831c8d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831c8d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.976+0000 7f831c8d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f831c8d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f831c8d1700 10 monclient(hunting): handle_auth_done global_id 20896 payload 824 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f831c8d1700 1 -- 172.21.15.47:0/636581807 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f831809f1c0 con 0x7f83181a7f60 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f831c8d1700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 0x7f83181a8370 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f830c01b000 tx=0x7f830c004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 client.23132.objecter ms_handle_connect 0x7f83181a7f60 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 client.23132.objecter resend_mon_ops 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f830c02b070 con 0x7f83181a7f60 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 monclient: dump: 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f830c023040 con 0x7f83181a7f60 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 msgr2=0x7f83181a8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 0x7f83181a8370 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f830c01b000 tx=0x7f830c004750 comp rx=0 tx=0).stop 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 msgr2=0x7f83181a8370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.977+0000 7f83157fa700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f83181a7f60 0x7f83181a8370 secure :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f830c01b000 tx=0x7f830c004750 comp rx=0 tx=0).stop 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:56.000 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:56.001 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:56.002 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:56.003 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:56.004 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.006 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:56.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:56.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.011 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.014 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:56.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:56.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:56.019 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.020 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:56.021 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 10 librados: watch_flush enter 2024-04-20T02:17:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 10 librados: watch_flush exit 2024-04-20T02:17:56.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 20 client.23132.objecter shutdown clearing up homeless session... 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 10 client.23132.objecter successfully canceled tick 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f82fc03a220 msgr2=0x7f82fc03c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f82fc03a220 0x7f82fc03c6e0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f8304009920 tx=0x7f8304008030 comp rx=0 tx=0).stop 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 10 monclient: shutdown 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 msgr2=0x7f83181bc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831d0d2700 1 -- 172.21.15.47:0/636581807 reap_dead start 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 0x7f83181bc4c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8310007db0 tx=0x7f831000d3b0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 shutdown_connections 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 --2- 172.21.15.47:0/636581807 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f83180a1150 0x7f83181bc4c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 >> 172.21.15.47:0/636581807 conn(0x7f83180a7570 msgr2=0x7f83180a4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 shutdown_connections 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 -- 172.21.15.47:0/636581807 wait complete. 2024-04-20T02:17:56.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:55.978+0000 7f831eb35700 1 librados: shutdown 2024-04-20T02:17:56.025 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:56.025 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:17:56.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.172+0000 7f536e7cb700 10 monclient: get_monmap_and_config 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.172+0000 7f536e7cb700 10 monclient: build_initial_monmap 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.172+0000 7f536e7cb700 10 monclient: monmap: 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:56.173735+0000 2024-04-20T02:17:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:56.173735+0000 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:56.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 1 Processor -- start 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 1 -- start start 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 10 monclient: init 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 10 monclient: _add_conns ranks=[5,7,2,1,9,4,3,8,6,10,0] 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 0x7f53680a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 10 monclient(hunting): picked mon.noname-h con 0x7f53680a2950 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f536809eac0 0x7f53680a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 10 monclient(hunting): picked mon.noname-i con 0x7f536809eac0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 0x7f5368134700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.173+0000 7f536e7cb700 10 monclient(hunting): picked mon.noname-b con 0x7f53680a11f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53680a02e0 con 0x7f53680a11f0 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53680a37b0 con 0x7f53680a2950 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53680a0810 con 0x7f536809eac0 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536e7cb700 10 monclient(hunting): authenticate will time out at 1755.575046s 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 0x7f5368134700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 0x7f53680a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 0x7f5368134700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39432/0 (socket says 172.21.15.47:39432) 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 0x7f53680a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:53406/0 (socket says 172.21.15.47:53406) 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 1 -- 172.21.15.47:0/1532008207 learned_addr learned my addr 172.21.15.47:0/1532008207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f536809eac0 0x7f53680a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): get_auth_request con 0x7f53680a11f0 auth_method 0 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): get_auth_request con 0x7f53680a2950 auth_method 0 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): get_auth_request con 0x7f536809eac0 auth_method 0 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:56.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f53677fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient(hunting): handle_auth_done global_id 34214 payload 274 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 -- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f536809eac0 msgr2=0x7f53680a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f536809eac0 0x7f53680a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 -- 172.21.15.47:0/1532008207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 msgr2=0x7f5368134700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 0x7f5368134700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 monclient: found mon.noname-h 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:17:56.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 -- 172.21.15.47:0/1532008207 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680751e0 con 0x7f53680a2950 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f536cd68700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 0x7f5368134700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.174+0000 7f5367fff700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 0x7f53680a2d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5358009870 tx=0x7f5358009b80 comp rx=0 tx=0).ready entity=mon.5 client_cookie=5d5d915a5e6eba07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 1 -- 172.21.15.47:0/1532008207 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5358011aa0 con 0x7f53680a2950 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: dump: 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:56.189 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 1 -- 172.21.15.47:0/1532008207 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5358011c00 con 0x7f53680a2950 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 5 monclient: authenticate success, global_id 34214 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:56.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 1 -- 172.21.15.47:0/1532008207 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5358011d70 con 0x7f53680a2950 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 4 set_mon_vals no callback set 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f5366ffd700 10 monclient: dump: 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:56.191 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 10 monclient: shutdown 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 -- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 msgr2=0x7f53680a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 0x7f53680a2d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5358009870 tx=0x7f5358009b80 comp rx=0 tx=0).stop 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 -- 172.21.15.47:0/1532008207 shutdown_connections 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f53680a11f0 0x7f5368134700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f53680a2950 0x7f53680a2d30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 --2- 172.21.15.47:0/1532008207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f536809eac0 0x7f53680a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 -- 172.21.15.47:0/1532008207 >> 172.21.15.47:0/1532008207 conn(0x7f53680a7570 msgr2=0x7f53680a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 -- 172.21.15.47:0/1532008207 shutdown_connections 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.175+0000 7f536e7cb700 1 -- 172.21.15.47:0/1532008207 wait complete. 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.176+0000 7f536e7cb700 10 monclient: build_initial_monmap 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.176+0000 7f536e7cb700 10 monclient: monmap: 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:56.193 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.176+0000 7f536e7cb700 1 librados: starting msgr at 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.176+0000 7f536e7cb700 1 librados: starting objecter 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.176+0000 7f536e7cb700 1 Processor -- start 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 -- start start 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 librados: setting wanted keys 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 librados: calling monclient init 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient: init 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:56.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient: _add_conns ranks=[6,3,4,1,0,2,7,5,8] 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 0x7f53681bab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): picked mon.noname-d con 0x7f536809eac0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53680a11f0 0x7f53681c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): picked mon.noname-g con 0x7f53680a11f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53680a2950 0x7f53681c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): picked mon.noname-c con 0x7f53680a2950 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53681a9f60 con 0x7f53680a11f0 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f536809f260 con 0x7f53680a2950 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5368072410 con 0x7f536809eac0 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536e7cb700 10 monclient(hunting): authenticate will time out at 1755.578616s 2024-04-20T02:17:56.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 0x7f53681bab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53680a2950 0x7f53681c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 0x7f53681bab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50130/0 (socket says 172.21.15.47:50130) 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): get_auth_request con 0x7f53680a2950 auth_method 0 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 1 -- 172.21.15.47:0/691571824 learned_addr learned my addr 172.21.15.47:0/691571824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): get_auth_request con 0x7f536809eac0 auth_method 0 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53680a11f0 0x7f53681c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): get_auth_request con 0x7f53680a11f0 auth_method 0 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:56.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f5367fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f536cd68700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.177+0000 7f53677fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 10 monclient(hunting): handle_auth_done global_id 22515 payload 824 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 -- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53680a2950 msgr2=0x7f53681c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53680a2950 0x7f53681c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 -- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53680a11f0 msgr2=0x7f53681c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53680a11f0 0x7f53681c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 monclient: found mon.noname-d 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 -- 172.21.15.47:0/691571824 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360009b30 con 0x7f536809eac0 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536cd68700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53680a2950 0x7f53681c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f53677fe700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53680a11f0 0x7f53681c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5367fff700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 0x7f53681bab40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5358014040 tx=0x7f5358004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 client.?.objecter ms_handle_connect 0x7f536809eac0 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53580036a0 con 0x7f536809eac0 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: dump: 2024-04-20T02:17:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:56.199 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: _finish_auth 0 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535801f070 con 0x7f536809eac0 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 5 monclient: authenticate success, global_id 22515 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 10 monclient: _renew_subs 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5358003c90 con 0x7f536809eac0 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53580094a0 con 0x7f536809eac0 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f53667fc700 4 set_mon_vals no callback set 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 10 client.22515.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f5364ff9700 10 monclient: dump: 2024-04-20T02:17:56.200 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:56.201 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 10 monclient: _renew_subs 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53681a9c80 con 0x7f536809eac0 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.178+0000 7f536e7cb700 1 librados: init done 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5358021030 con 0x7f536809eac0 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f536e7cb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f536e7cb700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f536e7cb700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53681aa1a0 con 0x7f536809eac0 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f534803a2f0 0x7f534803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8301+0+0 (secure 0 0 0) 0x7f535804c110 con 0x7f536809eac0 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 10 client.22515.objecter ms_dispatch 0x7f5368001550 osd_map(86..86 src has 1..86) v4 2024-04-20T02:17:56.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 3 client.22515.objecter handle_osd_map got epochs [86,86] > 0 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 3 client.22515.objecter handle_osd_map decoding full epoch 86 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f53677fe700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f534803a2f0 0x7f534803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f53677fe700 10 monclient: get_auth_request con 0x7f534803a2f0 auth_method 0 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.179+0000 7f5364ff9700 20 client.22515.objecter dump_active .. 0 homeless 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.180+0000 7f53677fe700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f534803a2f0 0x7f534803c7b0 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f5368075640 tx=0x7f5350009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.180+0000 7f5364ff9700 10 client.22515.objecter ms_handle_connect 0x7f534803a2f0 2024-04-20T02:17:56.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.185+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f5358019d40 con 0x7f536809eac0 2024-04-20T02:17:56.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.185+0000 7f5364ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:56.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.185+0000 7f5364ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.494+0000 7f536e7cb700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:17:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.494+0000 7f536e7cb700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:17:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.494+0000 7f536e7cb700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:17:56.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.494+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f53680a4eb0 con 0x7f536809eac0 2024-04-20T02:17:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.497+0000 7f5364ff9700 1 -- 172.21.15.47:0/691571824 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5358019b60 con 0x7f536809eac0 2024-04-20T02:17:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.497+0000 7f5364ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:17:56.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.497+0000 7f5364ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 10 librados: watch_flush enter 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 10 librados: watch_flush exit 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 20 client.22515.objecter shutdown clearing up homeless session... 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 10 client.22515.objecter successfully canceled tick 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f534803a2f0 msgr2=0x7f534803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f534803a2f0 0x7f534803c7b0 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f5368075640 tx=0x7f5350009210 comp rx=0 tx=0).stop 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 10 monclient: shutdown 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 msgr2=0x7f53681bab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 0x7f53681bab40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5358014040 tx=0x7f5358004750 comp rx=0 tx=0).stop 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 shutdown_connections 2024-04-20T02:17:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f534803a2f0 0x7f534803c7b0 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f53680a2950 0x7f53681c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f536809eac0 0x7f53681bab40 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 --2- 172.21.15.47:0/691571824 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53680a11f0 0x7f53681c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.500+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 >> 172.21.15.47:0/691571824 conn(0x7f53680a7570 msgr2=0x7f536809c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.501+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 shutdown_connections 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.501+0000 7f536e7cb700 1 -- 172.21.15.47:0/691571824 wait complete. 2024-04-20T02:17:56.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:56.501+0000 7f536e7cb700 1 librados: shutdown 2024-04-20T02:17:56.517 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=1,health=12,kv=11,logm=67} crc {auth=3772617737,config=1228700967,health=2724922895,kv=287005021,logm=1959530421}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=11,kv=11,logm=66} crc {auth=40576483,config=692619927,health=2252974882,kv=1530195301,logm=450005857}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=2,health=12,kv=10,logm=67} crc {auth=3772617737,config=692619927,health=2470548483,kv=435135402,logm=4147797651}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=9,config=2,health=12,kv=11,logm=66} crc {auth=2130445154,config=692619927,health=886868997,kv=1530195301,logm=2902940061}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.497+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2749866343,kv=1530195301,logm=3964542710}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=233140797,config=692619927,health=2724922895,kv=1530195301,logm=3504961815}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=10,config=2,health=11,kv=11,logm=66} crc {auth=40576483,config=692619927,health=183780791,kv=1530195301,logm=1031905941}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=40576483,config=692619927,health=2724922895,kv=1530195301,logm=3820587971}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.498+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=2699176486,config=692619927,health=2724922895,kv=287005021,logm=2432650876}) 2024-04-20T02:17:56.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1790732816}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=727993377}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.499+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1419624165}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1593272281}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1576059207}) 2024-04-20T02:17:56.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3153561803}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2205012018}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3698132046}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.500+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2238208701}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2821620797}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=132235124}) 2024-04-20T02:17:56.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1753553030}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2527956781}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3767496663}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2736692459}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1192085363}) 2024-04-20T02:17:56.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4047713722}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.501+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2269454783}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619990374}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2133513839}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4138040166}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2321461861}) 2024-04-20T02:17:56.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184438984}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=994374172}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=556672928}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4233798248}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.503+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3592023193}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1267997328}) 2024-04-20T02:17:56.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2896217788}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=410896881}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=287935069}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1725403158}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=328329726}) 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3432286227}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.504+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2710791489}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.505+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4001841816}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=287031466}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1714297588}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3127974615}) 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=756702445}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3667059519}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2098062050}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3983337414}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2310941524}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.506+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1545442602}) 2024-04-20T02:17:56.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1816726856}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221593980}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1819094377}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=673597977}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1028378831}) 2024-04-20T02:17:56.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=566217666}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=803735817}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925860268}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.508+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1554615107}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2107367015}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2684282278}) 2024-04-20T02:17:56.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2357859054}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2654964353}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708873038}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=756159534}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3414709806}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2925709159}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.510+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3881492441}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.511+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.511+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=170552608}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3487576772}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=869274513}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=648785376}) 2024-04-20T02:17:56.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=431651992}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2791071577}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1966902209}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728276657}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.512+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=340821257}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2047042585}) 2024-04-20T02:17:56.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3537155812}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=474865767}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1641522221}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3750878445}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1481950972}) 2024-04-20T02:17:56.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1994149387}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498038359}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.514+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1770975380}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.515+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=22368426}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1952351939}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=500197463}) 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4230715222}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275432560}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2130890617}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3527404373}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2773994568}) 2024-04-20T02:17:56.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.516+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3799182884}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2294396196}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=786829313}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1774935848}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4284690719}) 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865290156}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2734832328}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1928112953}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1917783615}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.517+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1880531292}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3048717910}) 2024-04-20T02:17:56.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2099224037}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1706562617}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1069724405}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2488794434}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3309468739}) 2024-04-20T02:17:56.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1686491316}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3841083727}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.519+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1804446979}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1038730537}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4285164503}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4259801945}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4162350933}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815857833}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=815088838}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4275164528}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4086937066}) 2024-04-20T02:17:56.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.520+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788665052}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=15,osdmap=26} crc {logm=1707980661,mdsmap=1531151339,mgr=2750768054,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3691363624,mon_config_key=3905112211,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=2950792396,osdmap=458179484}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=15,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=14,osdmap=29} crc {logm=2980075174,mdsmap=1531151339,mgr=4024091195,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2042943380,mon_config_key=3954964383,monmap=3385036551,osd_metadata=2211062212,osd_pg_creating=1716736971,osd_snap=1657413211,osdmap=4061884419}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=28} crc {logm=4252528381,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3383040861,mon_config_key=2979802314,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=4094040681,osdmap=3857562319}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=13,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=13,osdmap=31} crc {logm=3021577573,mdsmap=3585761405,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391055,mon_config_key=2868358642,monmap=3140514009,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=525895521,osdmap=4254222851}) 2024-04-20T02:17:56.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=17,osdmap=23} crc {logm=286545433,mdsmap=1531151339,mgr=3502831154,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391055,mon_config_key=2868358642,monmap=3385036551,osd_metadata=1881248415,osd_pg_creating=1716736971,osd_snap=1620308443,osdmap=3333775971}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=17,osdmap=22} crc {logm=3072407926,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391055,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2211062212,osd_pg_creating=1716736971,osd_snap=1620308443,osdmap=702975985}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=16,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=16,osdmap=25} crc {logm=3072407926,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391055,mon_config_key=2120219358,monmap=3385036551,osd_metadata=2211062212,osd_pg_creating=1716736971,osd_snap=2449538360,osdmap=3753101217}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=16,osdmap=23} crc {logm=3072407926,mdsmap=1531151339,mgr=3280844453,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391056,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=934290010,osdmap=873653697}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.523+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=15,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14,osdmap=27} crc {logm=1341158022,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1565391055,mon_config_key=759064529,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1716736971,osd_snap=76657430,osdmap=647986171}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4213795056}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2658874819}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=271603565}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1723286662}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=402611571}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4055077069}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1842822512}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3315235573}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.526+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3404184497}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=19} crc {osdmap=3578547760}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=16} crc {osdmap=2704770825}) 2024-04-20T02:17:56.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=18} crc {osdmap=76138706}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=19} crc {osdmap=3578547760}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=18} crc {osdmap=1453059510}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=17} crc {osdmap=751776924}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=17} crc {osdmap=3476644519}) 2024-04-20T02:17:56.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.531+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:17:56.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.532+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=19} crc {osdmap=4269046270}) 2024-04-20T02:17:56.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:17:56.532+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=18} crc {osdmap=1271048293}) 2024-04-20T02:17:57.518 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:17:57.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:17:57.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.661+0000 7faf194b3700 10 monclient: get_monmap_and_config 2024-04-20T02:17:57.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.661+0000 7faf194b3700 10 monclient: build_initial_monmap 2024-04-20T02:17:57.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.661+0000 7faf194b3700 10 monclient: monmap: 2024-04-20T02:17:57.661 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:57.661 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:57.663025+0000 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:57.663025+0000 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:57.662 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:57.663 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:57.663 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.662+0000 7faf194b3700 1 Processor -- start 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.662+0000 7faf194b3700 1 -- start start 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.662+0000 7faf194b3700 10 monclient: init 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient: _add_conns ranks=[6,3,9,10,8,7,1,0,5,4,2] 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 0x7faf140a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): picked mon.noname-d con 0x7faf140a4c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf140a0d20 0x7faf140a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): picked mon.noname-g con 0x7faf140a0d20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 0x7faf14137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): picked mon.noname-a con 0x7faf140a55a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf140a0000 con 0x7faf140a55a0 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140a6a10 con 0x7faf140a0d20 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faf1409e440 con 0x7faf140a4c80 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf194b3700 10 monclient(hunting): authenticate will time out at 1757.064196s 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 0x7faf140a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 0x7faf14137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 0x7faf14137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47706/0 (socket says 172.21.15.47:47706) 2024-04-20T02:17:57.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 0x7faf140a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50146/0 (socket says 172.21.15.47:50146) 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 1 -- 172.21.15.47:0/4270389980 learned_addr learned my addr 172.21.15.47:0/4270389980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): get_auth_request con 0x7faf140a55a0 auth_method 0 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): get_auth_request con 0x7faf140a4c80 auth_method 0 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf127fc700 1 --2- 172.21.15.47:0/4270389980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf140a0d20 0x7faf140a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:57.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf127fc700 10 monclient(hunting): get_auth_request con 0x7faf140a0d20 auth_method 0 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf127fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf127fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf127fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient(hunting): handle_auth_done global_id 22524 payload 274 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 -- 172.21.15.47:0/4270389980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf140a0d20 msgr2=0x7faf140a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 --2- 172.21.15.47:0/4270389980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf140a0d20 0x7faf140a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 -- 172.21.15.47:0/4270389980 >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 msgr2=0x7faf14137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 --2- 172.21.15.47:0/4270389980 >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 0x7faf14137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 monclient: found mon.noname-d 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:17:57.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 -- 172.21.15.47:0/4270389980 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140751e0 con 0x7faf140a4c80 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf137fe700 1 --2- 172.21.15.47:0/4270389980 >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 0x7faf14137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.663+0000 7faf12ffd700 1 --2- 172.21.15.47:0/4270389980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 0x7faf140a07e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7faf04009840 tx=0x7faf04009b50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a968f29aac7deb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 1 -- 172.21.15.47:0/4270389980 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf04011aa0 con 0x7faf140a4c80 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: dump: 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:57.677 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: _finish_auth 0 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 5 monclient: authenticate success, global_id 22524 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 1 -- 172.21.15.47:0/4270389980 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf04011c00 con 0x7faf140a4c80 2024-04-20T02:17:57.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 4 set_mon_vals no callback set 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 1 -- 172.21.15.47:0/4270389980 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf04011d70 con 0x7faf140a4c80 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf11ffb700 10 monclient: dump: 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:57.679 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 10 monclient: shutdown 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 -- 172.21.15.47:0/4270389980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 msgr2=0x7faf140a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 --2- 172.21.15.47:0/4270389980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 0x7faf140a07e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7faf04009840 tx=0x7faf04009b50 comp rx=0 tx=0).stop 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 -- 172.21.15.47:0/4270389980 shutdown_connections 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 --2- 172.21.15.47:0/4270389980 >> v2:172.21.15.47:3300/0 conn(0x7faf140a55a0 0x7faf14137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 --2- 172.21.15.47:0/4270389980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf140a4c80 0x7faf140a07e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 --2- 172.21.15.47:0/4270389980 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf140a0d20 0x7faf140a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 -- 172.21.15.47:0/4270389980 >> 172.21.15.47:0/4270389980 conn(0x7faf140aa280 msgr2=0x7faf140aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 -- 172.21.15.47:0/4270389980 shutdown_connections 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.664+0000 7faf194b3700 1 -- 172.21.15.47:0/4270389980 wait complete. 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.665+0000 7faf194b3700 10 monclient: build_initial_monmap 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.665+0000 7faf194b3700 10 monclient: monmap: 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:57.681 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.665+0000 7faf194b3700 1 librados: starting msgr at 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.665+0000 7faf194b3700 1 librados: starting objecter 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 Processor -- start 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 -- start start 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 librados: setting wanted keys 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 librados: calling monclient init 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient: init 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient: _add_conns ranks=[7,1,0,8,6,2,5,3,4] 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf140a0d20 0x7faf141cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): picked mon.noname-i con 0x7faf140a0d20 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:57.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf140a4c80 0x7faf14136a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): picked mon.noname-f con 0x7faf140a4c80 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 0x7faf14136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): picked mon.noname-a con 0x7faf140a55a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14127ee0 con 0x7faf140a55a0 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14128410 con 0x7faf140a4c80 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faf1409dd20 con 0x7faf140a0d20 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf194b3700 10 monclient(hunting): authenticate will time out at 1757.067642s 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 0x7faf14136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 0x7faf14136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47722/0 (socket says 172.21.15.47:47722) 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 1 -- 172.21.15.47:0/911999710 learned_addr learned my addr 172.21.15.47:0/911999710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:57.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): get_auth_request con 0x7faf140a55a0 auth_method 0 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf140a4c80 0x7faf14136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf140a0d20 0x7faf141cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): get_auth_request con 0x7faf140a4c80 auth_method 0 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 10 monclient(hunting): get_auth_request con 0x7faf140a0d20 auth_method 0 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:57.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf137fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf127fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.666+0000 7faf12ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf12ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 10 monclient(hunting): handle_auth_done global_id 34245 payload 824 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 -- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf140a0d20 msgr2=0x7faf141cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf140a0d20 0x7faf141cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 -- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf140a4c80 msgr2=0x7faf14136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf140a4c80 0x7faf14136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 monclient: found mon.noname-a 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 -- 172.21.15.47:0/911999710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c009b30 con 0x7faf140a55a0 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf137fe700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 0x7faf14136f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00efc0 tx=0x7faf0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 client.?.objecter ms_handle_connect 0x7faf140a55a0 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf0c018070 con 0x7faf140a55a0 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: dump: 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 5 monclient: authenticate success, global_id 34245 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c00f040 con 0x7faf140a55a0 2024-04-20T02:17:57.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf0c0092e0 con 0x7faf140a55a0 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf117fa700 4 set_mon_vals no callback set 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faefb7fe700 10 monclient: dump: 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:57.688 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 10 monclient: _renew_subs 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf040094a0 con 0x7faf140a55a0 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 10 client.34245.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 10 monclient: _renew_subs 2024-04-20T02:17:57.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf14127b70 con 0x7faf140a55a0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.667+0000 7faf194b3700 1 librados: init done 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7faf0c00c880 con 0x7faf140a55a0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faf194b3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faf194b3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faf194b3700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf14137f60 con 0x7faf140a55a0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faf0003a2f0 0x7faf0003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8704+0+0 (secure 0 0 0) 0x7faf0c04af60 con 0x7faf140a55a0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 10 client.34245.objecter ms_dispatch 0x7faf14001550 osd_map(87..87 src has 1..87) v4 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 3 client.34245.objecter handle_osd_map got epochs [87,87] > 0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 3 client.34245.objecter handle_osd_map decoding full epoch 87 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faefb7fe700 20 client.34245.objecter dump_active .. 0 homeless 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.668+0000 7faf12ffd700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faf0003a2f0 0x7faf0003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:57.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.669+0000 7faf12ffd700 10 monclient: get_auth_request con 0x7faf0003a2f0 auth_method 0 2024-04-20T02:17:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.669+0000 7faefb7fe700 10 client.34245.objecter ms_handle_connect 0x7faf0003a2f0 2024-04-20T02:17:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.669+0000 7faf12ffd700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faf0003a2f0 0x7faf0003c7b0 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7faf04009840 tx=0x7faf04008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.674+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7faf14137f60 con 0x7faf140a55a0 2024-04-20T02:17:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.674+0000 7faefb7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:57.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.674+0000 7faefb7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:57.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.977+0000 7faf194b3700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:17:57.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.977+0000 7faf194b3700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:57.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.977+0000 7faf194b3700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:57.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.977+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faf14128270 con 0x7faf140a55a0 2024-04-20T02:17:57.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.978+0000 7faefb7fe700 1 -- 172.21.15.47:0/911999710 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7faf14128270 con 0x7faf140a55a0 2024-04-20T02:17:57.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.978+0000 7faefb7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:17:57.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.978+0000 7faefb7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:57.978 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":18,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":5,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.979+0000 7faf194b3700 10 librados: watch_flush enter 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 10 librados: watch_flush exit 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 20 client.34245.objecter shutdown clearing up homeless session... 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 10 client.34245.objecter successfully canceled tick 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faf0003a2f0 msgr2=0x7faf0003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faf0003a2f0 0x7faf0003c7b0 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7faf04009840 tx=0x7faf04008030 comp rx=0 tx=0).stop 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 10 monclient: shutdown 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 msgr2=0x7faf14136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 0x7faf14136f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00efc0 tx=0x7faf0c00c3a0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 shutdown_connections 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf140a55a0 0x7faf14136f50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf140a0d20 0x7faf141cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faf0003a2f0 0x7faf0003c7b0 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 --2- 172.21.15.47:0/911999710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf140a4c80 0x7faf14136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 >> 172.21.15.47:0/911999710 conn(0x7faf140aa280 msgr2=0x7faf140a3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 shutdown_connections 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 -- 172.21.15.47:0/911999710 wait complete. 2024-04-20T02:17:57.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:57.980+0000 7faf194b3700 1 librados: shutdown 2024-04-20T02:17:57.992 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:17:57.993 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:17:57.993 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:17:57.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.129+0000 7fea7a6dc700 10 monclient: get_monmap_and_config 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.129+0000 7fea7a6dc700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.129+0000 7fea7a6dc700 10 monclient: monmap: 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:58.130733+0000 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:58.130733+0000 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:58.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 Processor -- start 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 -- start start 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient: init 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient: _add_conns ranks=[3,1,10,6,4,8,0,5,9,7,2] 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea740a18a0 0x7fea74135e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient(hunting): picked mon.noname-g con 0x7fea740a18a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fea740a1c80 addr v1:172.21.15.179:6789/0 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea740a2100 0x7fea740a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient(hunting): picked mon.noname-f con 0x7fea740a2100 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea740751e0 con 0x7fea740a1c80 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea74135bd0 con 0x7fea740a2100 2024-04-20T02:17:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea74071eb0 con 0x7fea740a18a0 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.130+0000 7fea7a6dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea7a6dc700 10 monclient(hunting): authenticate will time out at 1757.532008s 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea740a18a0 0x7fea74135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea740a18a0 0x7fea74135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:37208/0 (socket says 172.21.15.47:37208) 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea740a2100 0x7fea740a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 1 -- 172.21.15.47:0/3222679391 learned_addr learned my addr 172.21.15.47:0/3222679391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea737fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fea740a1c80 0x7fea740a3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:44214/0 (socket says 172.21.15.47:44214) 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): get_auth_request con 0x7fea740a18a0 auth_method 0 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): get_auth_request con 0x7fea740a2100 auth_method 0 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea73fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient(hunting): handle_auth_done global_id 25759 payload 274 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 1 -- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea740a18a0 msgr2=0x7fea74135e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 1 --2- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea740a18a0 0x7fea74135e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 1 -- 172.21.15.47:0/3222679391 >> v1:172.21.15.179:6789/0 conn(0x7fea740a1c80 legacy=0x7fea740a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 1 monclient: found mon.noname-f 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:17:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.131+0000 7fea78c79700 1 -- 172.21.15.47:0/3222679391 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea74072190 con 0x7fea740a2100 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea78c79700 1 --2- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a2100 0x7fea740a3f90 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009d90 tx=0x7fea6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afbecc98efdc30b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 1 -- 172.21.15.47:0/3222679391 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c00e070 con 0x7fea740a2100 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: dump: 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 1 -- 172.21.15.47:0/3222679391 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c012430 con 0x7fea740a2100 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 1 -- 172.21.15.47:0/3222679391 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c012a10 con 0x7fea740a2100 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:17:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea72ffd700 10 monclient: dump: 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.140 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea7a6dc700 5 monclient: authenticate success, global_id 25759 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea7a6dc700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.132+0000 7fea7a6dc700 4 set_mon_vals no callback set 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 10 monclient: shutdown 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 -- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a2100 msgr2=0x7fea740a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a2100 0x7fea740a3f90 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009d90 tx=0x7fea6c00b330 comp rx=0 tx=0).stop 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 -- 172.21.15.47:0/3222679391 shutdown_connections 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a2100 0x7fea740a3f90 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/3222679391 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea740a18a0 0x7fea74135e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 -- 172.21.15.47:0/3222679391 >> 172.21.15.47:0/3222679391 conn(0x7fea740a50b0 msgr2=0x7fea74066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.133+0000 7fea7a6dc700 1 -- 172.21.15.47:0/3222679391 shutdown_connections 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.134+0000 7fea7a6dc700 1 -- 172.21.15.47:0/3222679391 wait complete. 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.135+0000 7fea7a6dc700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.135+0000 7fea7a6dc700 10 monclient: monmap: 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.142 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.135+0000 7fea7a6dc700 1 librados: starting msgr at 2024-04-20T02:17:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.135+0000 7fea7a6dc700 1 librados: starting objecter 2024-04-20T02:17:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.135+0000 7fea7a6dc700 1 Processor -- start 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 -- start start 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 librados: setting wanted keys 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 librados: calling monclient init 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient: init 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient: _add_conns ranks=[1,6,8,3,4,5,0,7,2] 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a18a0 0x7fea741ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): picked mon.noname-f con 0x7fea740a18a0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:17:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea740a1c80 0x7fea74134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): picked mon.noname-d con 0x7fea740a1c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea740a2100 0x7fea74134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): picked mon.noname-e con 0x7fea740a2100 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea741cafe0 con 0x7fea740a18a0 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea7409ee50 con 0x7fea740a1c80 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fea741ca7d0 con 0x7fea740a2100 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.136+0000 7fea7a6dc700 10 monclient(hunting): authenticate will time out at 1757.537866s 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea740a2100 0x7fea74134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea740a2100 0x7fea74134ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39848/0 (socket says 172.21.15.47:39848) 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 1 -- 172.21.15.47:0/2124664400 learned_addr learned my addr 172.21.15.47:0/2124664400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): get_auth_request con 0x7fea740a2100 auth_method 0 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.138+0000 7fea78c79700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 10 monclient(hunting): handle_auth_done global_id 34199 payload 824 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea740a1c80 msgr2=0x7fea74134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea740a1c80 0x7fea74134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a18a0 msgr2=0x7fea741ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea740a18a0 0x7fea741ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 1 monclient: found mon.noname-e 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.139+0000 7fea78c79700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c009a70 con 0x7fea740a2100 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.140+0000 7fea78c79700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea740a2100 0x7fea74134ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fea6c014040 tx=0x7fea6c004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.140+0000 7fea70ff9700 10 client.?.objecter ms_handle_connect 0x7fea740a2100 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.140+0000 7fea70ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c012d20 con 0x7fea740a2100 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: dump: 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c004040 con 0x7fea740a2100 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 5 monclient: authenticate success, global_id 34199 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 10 monclient: _renew_subs 2024-04-20T02:17:58.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea74120bb0 con 0x7fea740a2100 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea6c004620 con 0x7fea740a2100 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea70ff9700 10 monclient: dump: 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 10 client.34199.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 10 monclient: _renew_subs 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea74122060 con 0x7fea740a2100 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea7a6dc700 1 librados: init done 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.141+0000 7fea727fc700 4 set_mon_vals no callback set 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fea6c004780 con 0x7fea740a2100 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea6003a2f0 0x7fea6003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7958+0+0 (secure 0 0 0) 0x7fea6c04cca0 con 0x7fea740a2100 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 10 client.34199.objecter ms_dispatch 0x7fea74001550 osd_map(88..88 src has 1..88) v4 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 3 client.34199.objecter handle_osd_map got epochs [88,88] > 0 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 3 client.34199.objecter handle_osd_map decoding full epoch 88 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.144+0000 7fea70ff9700 20 client.34199.objecter dump_active .. 0 homeless 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.146+0000 7fea7a6dc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.146+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 0x7fea7411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.146+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.146+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea74068430 con 0x7fea7406aac0 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.146+0000 7fea73fff700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea6003a2f0 0x7fea6003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.146+0000 7fea73fff700 10 monclient: get_auth_request con 0x7fea6003a2f0 auth_method 0 2024-04-20T02:17:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 0x7fea7411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient: get_auth_request con 0x7fea7406aac0 auth_method 0 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 10 monclient(hunting): handle_auth_done global_id 34254 payload 824 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.147+0000 7fea737fe700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea74070bb0 con 0x7fea7406aac0 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.148+0000 7fea70ff9700 10 client.34199.objecter ms_handle_connect 0x7fea7406aac0 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.148+0000 7fea70ff9700 10 client.34199.objecter resend_mon_ops 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.148+0000 7fea737fe700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 0x7fea7411e4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea7411fb00 tx=0x7fea6800c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.148+0000 7fea73fff700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea6003a2f0 0x7fea6003c7b0 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7fea64009520 tx=0x7fea64009470 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.149+0000 7fea70ff9700 10 client.34199.objecter ms_handle_connect 0x7fea6003a2f0 2024-04-20T02:17:58.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea68004ce0 con 0x7fea7406aac0 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 10 monclient: dump: 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.156 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fea68004e40 con 0x7fea7406aac0 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 msgr2=0x7fea7411e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 0x7fea7411e4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea7411fb00 tx=0x7fea6800c710 comp rx=0 tx=0).stop 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 msgr2=0x7fea7411e4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.150+0000 7fea70ff9700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7406aac0 0x7fea7411e4c0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fea7411fb00 tx=0x7fea6800c710 comp rx=0 tx=0).stop 2024-04-20T02:17:58.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.165+0000 7fea7a6dc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:58.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.165+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 0x7fea7411ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.165+0000 7fea7a6dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.165+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7406a400 con 0x7fea7411ea00 2024-04-20T02:17:58.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 0x7fea7411ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient: get_auth_request con 0x7fea7411ea00 auth_method 0 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 10 monclient(hunting): handle_auth_done global_id 34263 payload 824 2024-04-20T02:17:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.166+0000 7fea737fe700 1 -- 172.21.15.47:0/2124664400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea74067150 con 0x7fea7411ea00 2024-04-20T02:17:58.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.167+0000 7fea737fe700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 0x7fea7411ede0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea68000f60 tx=0x7fea68000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.169+0000 7fea70ff9700 10 client.34199.objecter ms_handle_connect 0x7fea7411ea00 2024-04-20T02:17:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.169+0000 7fea70ff9700 10 client.34199.objecter resend_mon_ops 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.172+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea68020070 con 0x7fea7411ea00 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.172+0000 7fea70ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.172+0000 7fea70ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.172+0000 7fea70ff9700 10 monclient: dump: 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fea68019680 con 0x7fea7411ea00 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 msgr2=0x7fea7411ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 0x7fea7411ede0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea68000f60 tx=0x7fea68000ed0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 msgr2=0x7fea7411ede0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.174+0000 7fea70ff9700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea7411ea00 0x7fea7411ede0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea68000f60 tx=0x7fea68000ed0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:58.175 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:58.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.179 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:58.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:58.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.189 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:58.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.194 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:58.195 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 10 librados: watch_flush enter 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 10 librados: watch_flush exit 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 20 client.34199.objecter shutdown clearing up homeless session... 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 10 client.34199.objecter successfully canceled tick 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea6003a2f0 msgr2=0x7fea6003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea6003a2f0 0x7fea6003c7b0 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7fea64009520 tx=0x7fea64009470 comp rx=0 tx=0).stop 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 10 monclient: shutdown 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea740a2100 msgr2=0x7fea74134ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 1 --2- 172.21.15.47:0/2124664400 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fea740a2100 0x7fea74134ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fea6c014040 tx=0x7fea6c004ad0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea78c79700 1 -- 172.21.15.47:0/2124664400 reap_dead start 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea78c79700 1 -- 172.21.15.47:0/2124664400 reap_dead start 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 shutdown_connections 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.175+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 >> 172.21.15.47:0/2124664400 conn(0x7fea740a50b0 msgr2=0x7fea740a0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.176+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 shutdown_connections 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.176+0000 7fea7a6dc700 1 -- 172.21.15.47:0/2124664400 wait complete. 2024-04-20T02:17:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.176+0000 7fea7a6dc700 1 librados: shutdown 2024-04-20T02:17:58.198 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:58.198 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:17:58.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.336+0000 7feb4a843700 10 monclient: get_monmap_and_config 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.336+0000 7feb4a843700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.336+0000 7feb4a843700 10 monclient: monmap: 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:58.337399+0000 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:58.337399+0000 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:58.336 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:58.337 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 Processor -- start 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 -- start start 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient: init 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient: _add_conns ranks=[7,6,0,8,2,10,3,4,5,9,1] 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb440a50f0 0x7feb44137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): picked mon.noname-i con 0x7feb440a50f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb441378a0 0x7feb44138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): picked mon.noname-d con 0x7feb441378a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feb440a38f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb440751e0 con 0x7feb440a38f0 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb440a54d0 con 0x7feb441378a0 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb441387b0 con 0x7feb440a50f0 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb4a843700 10 monclient(hunting): authenticate will time out at 1757.738733s 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb48de0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb440a38f0 0x7feb440a3d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48228/0 (socket says 172.21.15.47:48228) 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb437fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb441378a0 0x7feb44138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb48de0700 1 -- 172.21.15.47:0/1886952471 learned_addr learned my addr 172.21.15.47:0/1886952471 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb437fe700 10 monclient(hunting): get_auth_request con 0x7feb441378a0 auth_method 0 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb43fff700 1 --2- 172.21.15.47:0/1886952471 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb440a50f0 0x7feb44137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb43fff700 10 monclient(hunting): get_auth_request con 0x7feb440a50f0 auth_method 0 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb43fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb43fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.337+0000 7feb43fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb43fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb43fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb43fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 10 monclient(hunting): handle_auth_done global_id 22542 payload 274 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 1 -- 172.21.15.47:0/1886952471 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505370205 0 0) 0x7feb440751e0 con 0x7feb440a38f0 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 1 -- 172.21.15.47:0/1886952471 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb440a50f0 msgr2=0x7feb44137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 1 --2- 172.21.15.47:0/1886952471 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb440a50f0 0x7feb44137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 1 -- 172.21.15.47:0/1886952471 >> v1:172.21.15.47:6789/0 conn(0x7feb440a38f0 legacy=0x7feb440a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 1 monclient: found mon.noname-d 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:17:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 1 -- 172.21.15.47:0/1886952471 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4406da20 con 0x7feb441378a0 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb437fe700 1 --2- 172.21.15.47:0/1886952471 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb441378a0 0x7feb44138270 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feb38009870 tx=0x7feb38009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=71a32f275a4e94ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 1 -- 172.21.15.47:0/1886952471 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb38011aa0 con 0x7feb441378a0 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: dump: 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 1 -- 172.21.15.47:0/1886952471 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38011c00 con 0x7feb441378a0 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb4a843700 5 monclient: authenticate success, global_id 22542 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb4a843700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 1 -- 172.21.15.47:0/1886952471 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb38011d70 con 0x7feb441378a0 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb4a843700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb4a843700 4 set_mon_vals no callback set 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.338+0000 7feb42ffd700 10 monclient: dump: 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 10 monclient: shutdown 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 -- 172.21.15.47:0/1886952471 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb441378a0 msgr2=0x7feb44138270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 --2- 172.21.15.47:0/1886952471 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb441378a0 0x7feb44138270 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feb38009870 tx=0x7feb38009b80 comp rx=0 tx=0).stop 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 -- 172.21.15.47:0/1886952471 shutdown_connections 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 --2- 172.21.15.47:0/1886952471 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb440a50f0 0x7feb44137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 --2- 172.21.15.47:0/1886952471 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb441378a0 0x7feb44138270 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 -- 172.21.15.47:0/1886952471 >> 172.21.15.47:0/1886952471 conn(0x7feb440a8510 msgr2=0x7feb440a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 -- 172.21.15.47:0/1886952471 shutdown_connections 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 1 -- 172.21.15.47:0/1886952471 wait complete. 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.339+0000 7feb4a843700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 10 monclient: monmap: 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 1 librados: starting msgr at 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 1 librados: starting objecter 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 1 Processor -- start 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 1 -- start start 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 1 librados: setting wanted keys 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 1 librados: calling monclient init 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.340+0000 7feb4a843700 10 monclient: init 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient: _add_conns ranks=[8,1,7,0,4,2,5,6,3] 2024-04-20T02:17:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb440a38f0 0x7feb441cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): picked mon.noname-e con 0x7feb440a38f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb440a50f0 0x7feb44136950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): picked mon.noname-f con 0x7feb440a50f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb44136e90 0x7feb44120360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): picked mon.noname-i con 0x7feb44136e90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44072260 con 0x7feb440a50f0 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb44071e80 con 0x7feb44136e90 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7feb44124310 con 0x7feb440a38f0 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb4a843700 10 monclient(hunting): authenticate will time out at 1757.742155s 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb440a38f0 0x7feb441cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb440a38f0 0x7feb441cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39852/0 (socket says 172.21.15.47:39852) 2024-04-20T02:17:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 -- 172.21.15.47:0/801796738 learned_addr learned my addr 172.21.15.47:0/801796738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): get_auth_request con 0x7feb440a38f0 auth_method 0 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb44136e90 0x7feb44120360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb440a50f0 0x7feb44136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): get_auth_request con 0x7feb44136e90 auth_method 0 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): get_auth_request con 0x7feb440a50f0 auth_method 0 2024-04-20T02:17:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb48de0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient(hunting): handle_auth_done global_id 34217 payload 824 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb44136e90 msgr2=0x7feb44120360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7feb44136e90 0x7feb44120360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb440a50f0 msgr2=0x7feb44136950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb440a50f0 0x7feb44136950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 monclient: found mon.noname-e 2024-04-20T02:17:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380094a0 con 0x7feb440a38f0 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb43fff700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb440a38f0 0x7feb441cc1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9b0 tx=0x7feb3400ecc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb40ff9700 10 client.?.objecter ms_handle_connect 0x7feb440a38f0 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb40ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb34009710 con 0x7feb440a38f0 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb40ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.341+0000 7feb40ff9700 10 monclient: dump: 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3400ee50 con 0x7feb440a38f0 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 5 monclient: authenticate success, global_id 34217 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 monclient: _renew_subs 2024-04-20T02:17:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb44122890 con 0x7feb440a38f0 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb3400ccc0 con 0x7feb440a38f0 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 10 monclient: dump: 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb427fc700 4 set_mon_vals no callback set 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 client.34217.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 monclient: _renew_subs 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb44121f40 con 0x7feb440a38f0 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 1 librados: init done 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7feb3400c850 con 0x7feb440a38f0 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 0x7feb441208a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.342+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb440682f0 con 0x7feb441378a0 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 0x7feb441208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient: get_auth_request con 0x7feb441378a0 auth_method 0 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb3003a220 0x7feb3003c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7958+0+0 (secure 0 0 0) 0x7feb3404afd0 con 0x7feb440a38f0 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 client.34217.objecter ms_dispatch 0x7feb44001550 osd_map(88..88 src has 1..88) v4 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 3 client.34217.objecter handle_osd_map got epochs [88,88] > 0 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 3 client.34217.objecter handle_osd_map decoding full epoch 88 2024-04-20T02:17:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb48de0700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb3003a220 0x7feb3003c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 20 client.34217.objecter dump_active .. 0 homeless 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb48de0700 10 monclient: get_auth_request con 0x7feb3003a220 auth_method 0 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 10 monclient(hunting): handle_auth_done global_id 23168 payload 824 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb4406e370 con 0x7feb441378a0 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 client.34217.objecter ms_handle_connect 0x7feb441378a0 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 client.34217.objecter resend_mon_ops 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb437fe700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 0x7feb441208a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7feb440a49b0 tx=0x7feb380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb38020040 con 0x7feb441378a0 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 monclient: dump: 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb40ff9700 10 client.34217.objecter ms_handle_connect 0x7feb3003a220 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.343+0000 7feb48de0700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb3003a220 0x7feb3003c6e0 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7feb3c001c70 tx=0x7feb3c001c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7feb380036a0 con 0x7feb441378a0 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 msgr2=0x7feb441208a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 0x7feb441208a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7feb440a49b0 tx=0x7feb380047c0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 msgr2=0x7feb441208a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.344+0000 7feb40ff9700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb441378a0 0x7feb441208a0 secure :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7feb440a49b0 tx=0x7feb380047c0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.351+0000 7feb4a843700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.351+0000 7feb4a843700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 0x7feb44120de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb4a843700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4406a400 con 0x7feb4406aac0 2024-04-20T02:17:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 0x7feb44120de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient: get_auth_request con 0x7feb4406aac0 auth_method 0 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 10 monclient(hunting): handle_auth_done global_id 23177 payload 824 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 1 -- 172.21.15.47:0/801796738 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feb4406e650 con 0x7feb4406aac0 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 client.34217.objecter ms_handle_connect 0x7feb4406aac0 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 client.34217.objecter resend_mon_ops 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb437fe700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 0x7feb44120de0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7feb38009840 tx=0x7feb38004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb3802d300 con 0x7feb4406aac0 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 monclient: dump: 2024-04-20T02:17:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.371 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7feb38031070 con 0x7feb4406aac0 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 msgr2=0x7feb44120de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 0x7feb44120de0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7feb38009840 tx=0x7feb38004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 msgr2=0x7feb44120de0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.352+0000 7feb40ff9700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb4406aac0 0x7feb44120de0 secure :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7feb38009840 tx=0x7feb38004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:58.393 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 10 librados: watch_flush enter 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 10 librados: watch_flush exit 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 20 client.34217.objecter shutdown clearing up homeless session... 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 10 client.34217.objecter successfully canceled tick 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb3003a220 msgr2=0x7feb3003c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb3003a220 0x7feb3003c6e0 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7feb3c001c70 tx=0x7feb3c001c00 comp rx=0 tx=0).stop 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 10 monclient: shutdown 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb440a38f0 msgr2=0x7feb441cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 --2- 172.21.15.47:0/801796738 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb440a38f0 0x7feb441cc1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9b0 tx=0x7feb3400ecc0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb48de0700 1 -- 172.21.15.47:0/801796738 reap_dead start 2024-04-20T02:17:58.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb48de0700 1 -- 172.21.15.47:0/801796738 reap_dead start 2024-04-20T02:17:58.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 shutdown_connections 2024-04-20T02:17:58.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 >> 172.21.15.47:0/801796738 conn(0x7feb440a8510 msgr2=0x7feb441bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 shutdown_connections 2024-04-20T02:17:58.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 -- 172.21.15.47:0/801796738 wait complete. 2024-04-20T02:17:58.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.353+0000 7feb4a843700 1 librados: shutdown 2024-04-20T02:17:58.396 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:58.396 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:17:58.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:17:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.534+0000 7efee9be9700 10 monclient: get_monmap_and_config 2024-04-20T02:17:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.534+0000 7efee9be9700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.534+0000 7efee9be9700 10 monclient: monmap: 2024-04-20T02:17:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:58.535233+0000 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:58.535233+0000 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:58.535 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.534+0000 7efee9be9700 1 Processor -- start 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.534+0000 7efee9be9700 1 -- start start 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.534+0000 7efee9be9700 10 monclient: init 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient: _add_conns ranks=[2,10,5,7,9,6,0,1,4,3,8] 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 0x7efee40a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): picked mon.noname-b con 0x7efee406cf40 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efee40a2db0 0x7efee40a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): picked mon.noname-f con 0x7efee40a2db0 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efee4134690 0x7efee4134b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): picked mon.noname-h con 0x7efee4134690 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee40a36f0 con 0x7efee40a2db0 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee40a0240 con 0x7efee406cf40 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee406d320 con 0x7efee4134690 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee9be9700 10 monclient(hunting): authenticate will time out at 1757.936481s 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 0x7efee40a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 0x7efee40a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39458/0 (socket says 172.21.15.47:39458) 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 -- 172.21.15.47:0/4283126811 learned_addr learned my addr 172.21.15.47:0/4283126811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): get_auth_request con 0x7efee406cf40 auth_method 0 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee2ffd700 1 --2- 172.21.15.47:0/4283126811 >> v2:172.21.15.179:3300/0 conn(0x7efee40a2db0 0x7efee40a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee3fff700 1 --2- 172.21.15.47:0/4283126811 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efee4134690 0x7efee4134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee3fff700 10 monclient(hunting): get_auth_request con 0x7efee4134690 auth_method 0 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee2ffd700 10 monclient(hunting): get_auth_request con 0x7efee40a2db0 auth_method 0 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 10 monclient(hunting): handle_auth_done global_id 23186 payload 274 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 -- 172.21.15.47:0/4283126811 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efee4134690 msgr2=0x7efee4134b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 --2- 172.21.15.47:0/4283126811 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efee4134690 0x7efee4134b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 -- 172.21.15.47:0/4283126811 >> v2:172.21.15.179:3300/0 conn(0x7efee40a2db0 msgr2=0x7efee40a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.535+0000 7efee37fe700 1 --2- 172.21.15.47:0/4283126811 >> v2:172.21.15.179:3300/0 conn(0x7efee40a2db0 0x7efee40a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee37fe700 1 monclient: found mon.noname-b 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee37fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee37fe700 1 -- 172.21.15.47:0/4283126811 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee40751e0 con 0x7efee406cf40 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee37fe700 1 --2- 172.21.15.47:0/4283126811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 0x7efee40a2870 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7efed4009920 tx=0x7efed4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691a205ec023020 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 1 -- 172.21.15.47:0/4283126811 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed4009e70 con 0x7efee406cf40 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: dump: 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 1 -- 172.21.15.47:0/4283126811 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400bc90 con 0x7efee406cf40 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 5 monclient: authenticate success, global_id 23186 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 1 -- 172.21.15.47:0/4283126811 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed400be00 con 0x7efee406cf40 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 4 set_mon_vals no callback set 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee27fc700 10 monclient: dump: 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.551 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 10 monclient: shutdown 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 -- 172.21.15.47:0/4283126811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 msgr2=0x7efee40a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 --2- 172.21.15.47:0/4283126811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 0x7efee40a2870 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7efed4009920 tx=0x7efed4009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 -- 172.21.15.47:0/4283126811 shutdown_connections 2024-04-20T02:17:58.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 --2- 172.21.15.47:0/4283126811 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efee4134690 0x7efee4134b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 --2- 172.21.15.47:0/4283126811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efee406cf40 0x7efee40a2870 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 --2- 172.21.15.47:0/4283126811 >> v2:172.21.15.179:3300/0 conn(0x7efee40a2db0 0x7efee40a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.536+0000 7efee9be9700 1 -- 172.21.15.47:0/4283126811 >> 172.21.15.47:0/4283126811 conn(0x7efee40a4010 msgr2=0x7efee406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.537+0000 7efee9be9700 1 -- 172.21.15.47:0/4283126811 shutdown_connections 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.537+0000 7efee9be9700 1 -- 172.21.15.47:0/4283126811 wait complete. 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.537+0000 7efee9be9700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.537+0000 7efee9be9700 10 monclient: monmap: 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:58.553 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.537+0000 7efee9be9700 1 librados: starting msgr at 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.537+0000 7efee9be9700 1 librados: starting objecter 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 Processor -- start 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 -- start start 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 librados: setting wanted keys 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 librados: calling monclient init 2024-04-20T02:17:58.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient: init 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient: _add_conns ranks=[3,4,0,1,5,2,8,6,7] 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efee406cf40 0x7efee41c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): picked mon.noname-g con 0x7efee406cf40 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee40a2db0 0x7efee4133ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): picked mon.noname-c con 0x7efee40a2db0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 0x7efee41cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): picked mon.noname-a con 0x7efee4134690 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee409bea0 con 0x7efee4134690 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee409f3e0 con 0x7efee406cf40 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee41c9cf0 con 0x7efee40a2db0 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee9be9700 10 monclient(hunting): authenticate will time out at 1757.939863s 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee2ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee40a2db0 0x7efee4133ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.538+0000 7efee3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 0x7efee41cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee40a2db0 0x7efee4133ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:49894/0 (socket says 172.21.15.47:49894) 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 1 -- 172.21.15.47:0/2110971304 learned_addr learned my addr 172.21.15.47:0/2110971304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 0x7efee41cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47742/0 (socket says 172.21.15.47:47742) 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): get_auth_request con 0x7efee4134690 auth_method 0 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): get_auth_request con 0x7efee40a2db0 auth_method 0 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee37fe700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efee406cf40 0x7efee41c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee37fe700 10 monclient(hunting): get_auth_request con 0x7efee406cf40 auth_method 0 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient(hunting): handle_auth_done global_id 34299 payload 824 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee40a2db0 msgr2=0x7efee4133ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee40a2db0 0x7efee4133ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efee406cf40 msgr2=0x7efee41c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efee406cf40 0x7efee41c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 monclient: found mon.noname-a 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009550 con 0x7efee4134690 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee37fe700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efee406cf40 0x7efee41c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee2ffd700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee40a2db0 0x7efee4133ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee3fff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 0x7efee41cc280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efedc007cf0 tx=0x7efedc00e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 client.?.objecter ms_handle_connect 0x7efee4134690 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efedc018070 con 0x7efee4134690 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: dump: 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.558 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc00ee90 con 0x7efee4134690 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 5 monclient: authenticate success, global_id 34299 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efedc014890 con 0x7efee4134690 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efecbfff700 10 monclient: dump: 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee1ffb700 4 set_mon_vals no callback set 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 10 monclient: _renew_subs 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee40725a0 con 0x7efee4134690 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 10 client.34299.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 10 monclient: _renew_subs 2024-04-20T02:17:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee4124f00 con 0x7efee4134690 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.539+0000 7efee9be9700 1 librados: init done 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee9be9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee9be9700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 0x7efee41cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee4068430 con 0x7efee4134000 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7efedc014420 con 0x7efee4134690 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 0x7efee41cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient: get_auth_request con 0x7efee4134000 auth_method 0 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.540+0000 7efecbfff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efed003a2f0 0x7efed003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7958+0+0 (secure 0 0 0) 0x7efedc04ab00 con 0x7efee4134690 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 client.34299.objecter ms_dispatch 0x7efee4001550 osd_map(88..88 src has 1..88) v4 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 3 client.34299.objecter handle_osd_map got epochs [88,88] > 0 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 3 client.34299.objecter handle_osd_map decoding full epoch 88 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efee37fe700 10 monclient(hunting): handle_auth_done global_id 23035 payload 824 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efee37fe700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efee40a11e0 con 0x7efee4134000 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efee2ffd700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efed003a2f0 0x7efed003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 20 client.34299.objecter dump_active .. 0 homeless 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efee2ffd700 10 monclient: get_auth_request con 0x7efed003a2f0 auth_method 0 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 client.34299.objecter ms_handle_connect 0x7efee4134000 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 client.34299.objecter resend_mon_ops 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efee37fe700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 0x7efee41cc7c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed4004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed4004310 con 0x7efee4134000 2024-04-20T02:17:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 monclient: dump: 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 client.34299.objecter ms_handle_connect 0x7efed003a2f0 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efee2ffd700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efed003a2f0 0x7efed003c7b0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7efee409c090 tx=0x7efed8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efed4004470 con 0x7efee4134000 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 msgr2=0x7efee41cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 0x7efee41cc7c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed4004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 msgr2=0x7efee41cc7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.541+0000 7efecbfff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee4134000 0x7efee41cc7c0 secure :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7efed4009fd0 tx=0x7efed4004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee9be9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee9be9700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 0x7efee41ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee9be9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee4125370 con 0x7efee406aac0 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 0x7efee41ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient: get_auth_request con 0x7efee406aac0 auth_method 0 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 10 monclient(hunting): handle_auth_done global_id 23044 payload 824 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.549+0000 7efee37fe700 1 -- 172.21.15.47:0/2110971304 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efee409f090 con 0x7efee406aac0 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 client.34299.objecter ms_handle_connect 0x7efee406aac0 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 client.34299.objecter resend_mon_ops 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee37fe700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 0x7efee41ccd00 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efed40299f0 con 0x7efee406aac0 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 monclient: dump: 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7efed4029580 con 0x7efee406aac0 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 msgr2=0x7efee41ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 0x7efee41ccd00 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4011000 comp rx=0 tx=0).stop 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 msgr2=0x7efee41ccd00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efecbfff700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efee406aac0 0x7efee41ccd00 secure :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed4011000 comp rx=0 tx=0).stop 2024-04-20T02:17:58.570 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:58.571 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.572 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:58.574 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:58.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:58.576 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.578 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.581 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.587 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.589 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.590 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.591 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:17:58.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.591 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:58.591 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 10 librados: watch_flush enter 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 10 librados: watch_flush exit 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 20 client.34299.objecter shutdown clearing up homeless session... 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 10 client.34299.objecter successfully canceled tick 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efed003a2f0 msgr2=0x7efed003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efed003a2f0 0x7efed003c7b0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7efee409c090 tx=0x7efed8008040 comp rx=0 tx=0).stop 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 10 monclient: shutdown 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.550+0000 7efee9be9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 msgr2=0x7efee41cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 0x7efee41cc280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efedc007cf0 tx=0x7efedc00e0e0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee3fff700 1 -- 172.21.15.47:0/2110971304 reap_dead start 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 shutdown_connections 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 --2- 172.21.15.47:0/2110971304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee4134690 0x7efee41cc280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 >> 172.21.15.47:0/2110971304 conn(0x7efee40a4010 msgr2=0x7efee4067730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 shutdown_connections 2024-04-20T02:17:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 -- 172.21.15.47:0/2110971304 wait complete. 2024-04-20T02:17:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.551+0000 7efee9be9700 1 librados: shutdown 2024-04-20T02:17:58.594 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:58.594 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:17:58.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.732+0000 7ff48ece6700 10 monclient: get_monmap_and_config 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.732+0000 7ff48ece6700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.732+0000 7ff48ece6700 10 monclient: monmap: 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:58.733944+0000 2024-04-20T02:17:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:58.733944+0000 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:58.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.733+0000 7ff48ece6700 1 Processor -- start 2024-04-20T02:17:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.733+0000 7ff48ece6700 1 -- start start 2024-04-20T02:17:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.733+0000 7ff48ece6700 10 monclient: init 2024-04-20T02:17:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient: _add_conns ranks=[10,3,1,5,7,6,9,4,0,2,8] 2024-04-20T02:17:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff4880a3960 0x7ff488071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): picked mon.noname-f con 0x7ff4880a3960 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff488137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): picked mon.noname-g con 0x7ff488137b20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff4881384a0 addr v1:172.21.15.179:6789/0 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880751e0 con 0x7ff4881384a0 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48806e530 con 0x7ff4880a3960 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48809ef40 con 0x7ff488137b20 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ece6700 10 monclient(hunting): authenticate will time out at 1758.135254s 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff488137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff4880a3960 0x7ff488071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff488137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:37220/0 (socket says 172.21.15.47:37220) 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 -- 172.21.15.47:0/2108302858 learned_addr learned my addr 172.21.15.47:0/2108302858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): get_auth_request con 0x7ff488137b20 auth_method 0 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): get_auth_request con 0x7ff4880a3960 auth_method 0 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient(hunting): handle_auth_done global_id 21018 payload 274 2024-04-20T02:17:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 -- 172.21.15.47:0/2108302858 >> v2:172.21.15.179:3300/0 conn(0x7ff4880a3960 msgr2=0x7ff488071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 --2- 172.21.15.47:0/2108302858 >> v2:172.21.15.179:3300/0 conn(0x7ff4880a3960 0x7ff488071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 -- 172.21.15.47:0/2108302858 >> v1:172.21.15.179:6789/0 conn(0x7ff4881384a0 legacy=0x7ff48813ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 monclient: found mon.noname-g 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff487fff700 1 -- 172.21.15.47:0/2108302858 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880a3d40 con 0x7ff488137b20 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.734+0000 7ff48ca82700 1 --2- 172.21.15.47:0/2108302858 >> v2:172.21.15.179:3300/0 conn(0x7ff4880a3960 0x7ff488071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff487fff700 1 --2- 172.21.15.47:0/2108302858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff488137f60 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff4700098f0 tx=0x7ff470009c00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=9e1c6118b9e649f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 1 -- 172.21.15.47:0/2108302858 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff470011aa0 con 0x7ff488137b20 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: dump: 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 1 -- 172.21.15.47:0/2108302858 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff470011c00 con 0x7ff488137b20 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 5 monclient: authenticate success, global_id 21018 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 1 -- 172.21.15.47:0/2108302858 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff470011d70 con 0x7ff488137b20 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 4 set_mon_vals no callback set 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff4877fe700 10 monclient: dump: 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 10 monclient: shutdown 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 1 -- 172.21.15.47:0/2108302858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 msgr2=0x7ff488137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.735+0000 7ff48ece6700 1 --2- 172.21.15.47:0/2108302858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff488137f60 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff4700098f0 tx=0x7ff470009c00 comp rx=0 tx=0).stop 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 -- 172.21.15.47:0/2108302858 shutdown_connections 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 --2- 172.21.15.47:0/2108302858 >> v2:172.21.15.179:3300/0 conn(0x7ff4880a3960 0x7ff488071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 --2- 172.21.15.47:0/2108302858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff488137f60 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 -- 172.21.15.47:0/2108302858 >> 172.21.15.47:0/2108302858 conn(0x7ff4880a4a50 msgr2=0x7ff4880a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 -- 172.21.15.47:0/2108302858 shutdown_connections 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 -- 172.21.15.47:0/2108302858 wait complete. 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 10 monclient: monmap: 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 librados: starting msgr at 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.736+0000 7ff48ece6700 1 librados: starting objecter 2024-04-20T02:17:58.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 Processor -- start 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 -- start start 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 librados: setting wanted keys 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 librados: calling monclient init 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient: init 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient: _add_conns ranks=[8,3,4,0,1,5,7,2,6] 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4880a3960 0x7ff4881ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): picked mon.noname-e con 0x7ff4880a3960 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff4881cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): picked mon.noname-g con 0x7ff488137b20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4881384a0 0x7ff488121030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): picked mon.noname-c con 0x7ff4881384a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488124ca0 con 0x7ff488137b20 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff488124ef0 con 0x7ff4881384a0 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881236f0 con 0x7ff4880a3960 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ece6700 10 monclient(hunting): authenticate will time out at 1758.138786s 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4880a3960 0x7ff4881ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48d283700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4881384a0 0x7ff488121030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4880a3960 0x7ff4881ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39866/0 (socket says 172.21.15.47:39866) 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 1 -- 172.21.15.47:0/4278089889 learned_addr learned my addr 172.21.15.47:0/4278089889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 10 monclient(hunting): get_auth_request con 0x7ff4880a3960 auth_method 0 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48ca82700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48d283700 10 monclient(hunting): get_auth_request con 0x7ff4881384a0 auth_method 0 2024-04-20T02:17:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48d283700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48d283700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.737+0000 7ff48d283700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff4881cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): get_auth_request con 0x7ff488137b20 auth_method 0 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 10 monclient(hunting): handle_auth_done global_id 23053 payload 824 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4880a3960 msgr2=0x7ff4881ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4880a3960 0x7ff4881ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 msgr2=0x7ff4881cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff4881cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 monclient: found mon.noname-c 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780094a0 con 0x7ff4881384a0 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff487fff700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff488137b20 0x7ff4881cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ca82700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4880a3960 0x7ff4881ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48d283700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4881384a0 0x7ff488121030 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff48000cb10 tx=0x7ff48000ce20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 client.?.objecter ms_handle_connect 0x7ff4881384a0 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff480003e70 con 0x7ff4881384a0 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: dump: 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.757 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 5 monclient: authenticate success, global_id 23053 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480001b80 con 0x7ff4881384a0 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 10 monclient: _renew_subs 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff470009550 con 0x7ff4881384a0 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff48000ac10 con 0x7ff4881384a0 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:58.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff486ffd700 4 set_mon_vals no callback set 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 10 client.23053.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff4857fa700 10 monclient: dump: 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.759 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 10 monclient: _renew_subs 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48806d9d0 con 0x7ff4881384a0 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.738+0000 7ff48ece6700 1 librados: init done 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff48000a7a0 con 0x7ff4881384a0 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ece6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ece6700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 0x7ff488121570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff488068430 con 0x7ff48806aac0 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff4857fa700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46803a350 0x7ff46803c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ca82700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 0x7ff488121570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ca82700 10 monclient: get_auth_request con 0x7ff48806aac0 auth_method 0 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ca82700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ca82700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.739+0000 7ff48ca82700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7958+0+0 (secure 0 0 0) 0x7ff48004aba0 con 0x7ff4881384a0 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 client.23053.objecter ms_dispatch 0x7ff488001550 osd_map(88..88 src has 1..88) v4 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 3 client.23053.objecter handle_osd_map got epochs [88,88] > 0 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 3 client.23053.objecter handle_osd_map decoding full epoch 88 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 20 client.23053.objecter dump_active .. 0 homeless 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff487fff700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46803a350 0x7ff46803c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff487fff700 10 monclient: get_auth_request con 0x7ff46803a350 auth_method 0 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff48ca82700 10 monclient(hunting): handle_auth_done global_id 22551 payload 824 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff48ca82700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff488125060 con 0x7ff48806aac0 2024-04-20T02:17:58.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 client.23053.objecter ms_handle_connect 0x7ff48806aac0 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff48ca82700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 0x7ff488121570 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff47800f4a0 tx=0x7ff47800f860 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 client.23053.objecter resend_mon_ops 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4780044a0 con 0x7ff48806aac0 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 monclient: dump: 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 client.23053.objecter ms_handle_connect 0x7ff46803a350 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff487fff700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46803a350 0x7ff46803c810 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7ff470009c40 tx=0x7ff470008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff478004030 con 0x7ff48806aac0 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 msgr2=0x7ff488121570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 0x7ff488121570 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff47800f4a0 tx=0x7ff47800f860 comp rx=0 tx=0).stop 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 msgr2=0x7ff488121570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.740+0000 7ff4857fa700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff48806aac0 0x7ff488121570 secure :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff47800f4a0 tx=0x7ff47800f860 comp rx=0 tx=0).stop 2024-04-20T02:17:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ece6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ece6700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 0x7ff488121ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ece6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff48806a400 con 0x7ff488121ab0 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 0x7ff488121ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient: get_auth_request con 0x7ff488121ab0 auth_method 0 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.748+0000 7ff48ca82700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff48ca82700 10 monclient(hunting): handle_auth_done global_id 22560 payload 824 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff48ca82700 1 -- 172.21.15.47:0/4278089889 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff488138930 con 0x7ff488121ab0 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff48ca82700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 0x7ff488121ec0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff478009720 tx=0x7ff478004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 client.23053.objecter ms_handle_connect 0x7ff488121ab0 2024-04-20T02:17:58.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 client.23053.objecter resend_mon_ops 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff478014dc0 con 0x7ff488121ab0 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 monclient: dump: 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7ff478014950 con 0x7ff488121ab0 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 msgr2=0x7ff488121ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 0x7ff488121ec0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff478009720 tx=0x7ff478004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 msgr2=0x7ff488121ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff4857fa700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff488121ab0 0x7ff488121ec0 secure :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff478009720 tx=0x7ff478004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.768 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.782 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.785 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.787 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.789 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.789 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:58.789 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.749+0000 7ff48ece6700 10 librados: watch_flush enter 2024-04-20T02:17:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 10 librados: watch_flush exit 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 20 client.23053.objecter shutdown clearing up homeless session... 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 10 client.23053.objecter successfully canceled tick 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46803a350 msgr2=0x7ff46803c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46803a350 0x7ff46803c810 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7ff470009c40 tx=0x7ff470008030 comp rx=0 tx=0).stop 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 10 monclient: shutdown 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4881384a0 msgr2=0x7ff488121030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4881384a0 0x7ff488121030 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff48000cb10 tx=0x7ff48000ce20 comp rx=0 tx=0).stop 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48d283700 1 -- 172.21.15.47:0/4278089889 reap_dead start 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 shutdown_connections 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 --2- 172.21.15.47:0/4278089889 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4881384a0 0x7ff488121030 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 >> 172.21.15.47:0/4278089889 conn(0x7ff4880a4a50 msgr2=0x7ff488070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 shutdown_connections 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 -- 172.21.15.47:0/4278089889 wait complete. 2024-04-20T02:17:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.750+0000 7ff48ece6700 1 librados: shutdown 2024-04-20T02:17:58.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:58.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:17:58.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:17:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.930+0000 7f3f88b2e700 10 monclient: get_monmap_and_config 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.930+0000 7f3f88b2e700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.930+0000 7f3f88b2e700 10 monclient: monmap: 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:58.931471+0000 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:58.931471+0000 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.931 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:58.931 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:58.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.930+0000 7f3f88b2e700 1 Processor -- start 2024-04-20T02:17:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 -- start start 2024-04-20T02:17:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient: init 2024-04-20T02:17:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient: _add_conns ranks=[10,9,5,0,4,6,1,7,8,2,3] 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3f84066610 0x7f3f840a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): picked mon.noname-f con 0x7f3f84066610 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f84099e00 0x7f3f840a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): picked mon.noname-a con 0x7f3f84099e00 addr v2:172.21.15.47:3300/0 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3f840a2990 0x7f3f8413e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): picked mon.noname-h con 0x7f3f840a2990 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8409ba30 con 0x7f3f84099e00 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f840a3f70 con 0x7f3f84066610 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8409ced0 con 0x7f3f840a2990 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f88b2e700 10 monclient(hunting): authenticate will time out at 1758.332805s 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f84099e00 0x7f3f840a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3f84099e00 0x7f3f840a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47746/0 (socket says 172.21.15.47:47746) 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 1 -- 172.21.15.47:0/2073793649 learned_addr learned my addr 172.21.15.47:0/2073793649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 10 monclient(hunting): get_auth_request con 0x7f3f84099e00 auth_method 0 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.931+0000 7f3f81d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/2073793649 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3f840a2990 0x7f3f8413e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 1 --2- 172.21.15.47:0/2073793649 >> v2:172.21.15.179:3300/0 conn(0x7f3f84066610 0x7f3f840a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): get_auth_request con 0x7f3f840a2990 auth_method 0 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): get_auth_request con 0x7f3f84066610 auth_method 0 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f82d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_done global_id 34308 payload 274 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 -- 172.21.15.47:0/2073793649 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3f840a2990 msgr2=0x7f3f8413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/2073793649 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3f840a2990 0x7f3f8413e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 -- 172.21.15.47:0/2073793649 >> v2:172.21.15.179:3300/0 conn(0x7f3f84066610 msgr2=0x7f3f840a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/2073793649 >> v2:172.21.15.179:3300/0 conn(0x7f3f84066610 0x7f3f840a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 monclient: found mon.noname-a 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 -- 172.21.15.47:0/2073793649 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840751e0 con 0x7f3f84099e00 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/2073793649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84099e00 0x7f3f840a2450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3f8406d050 tx=0x7f3f78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94cc63279a08112 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 1 -- 172.21.15.47:0/2073793649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f78011aa0 con 0x7f3f84099e00 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: dump: 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f88b2e700 5 monclient: authenticate success, global_id 34308 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 1 -- 172.21.15.47:0/2073793649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78011c00 con 0x7f3f84099e00 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f88b2e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f88b2e700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f88b2e700 4 set_mon_vals no callback set 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 1 -- 172.21.15.47:0/2073793649 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f78011d70 con 0x7f3f84099e00 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.932+0000 7f3f8159a700 10 monclient: dump: 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 10 monclient: shutdown 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 -- 172.21.15.47:0/2073793649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84099e00 msgr2=0x7f3f840a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/2073793649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84099e00 0x7f3f840a2450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3f8406d050 tx=0x7f3f78009c00 comp rx=0 tx=0).stop 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 -- 172.21.15.47:0/2073793649 shutdown_connections 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/2073793649 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3f840a2990 0x7f3f8413e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/2073793649 >> v2:172.21.15.179:3300/0 conn(0x7f3f84066610 0x7f3f840a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/2073793649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f84099e00 0x7f3f840a2450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 -- 172.21.15.47:0/2073793649 >> 172.21.15.47:0/2073793649 conn(0x7f3f840a3000 msgr2=0x7f3f840a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 -- 172.21.15.47:0/2073793649 shutdown_connections 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.933+0000 7f3f88b2e700 1 -- 172.21.15.47:0/2073793649 wait complete. 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 10 monclient: build_initial_monmap 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 10 monclient: monmap: 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 1 librados: starting msgr at 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 1 librados: starting objecter 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 1 Processor -- start 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 1 -- start start 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 1 librados: setting wanted keys 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 1 librados: calling monclient init 2024-04-20T02:17:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.934+0000 7f3f88b2e700 10 monclient: init 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient: _add_conns ranks=[3,0,2,4,6,7,8,1,5] 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f84066610 0x7f3f8406f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): picked mon.noname-g con 0x7f3f84066610 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 0x7f3f8406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): picked mon.noname-a con 0x7f3f840a2990 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f84070140 0x7f3f841d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): picked mon.noname-b con 0x7f3f84070140 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84098280 con 0x7f3f840a2990 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8409a9d0 con 0x7f3f84070140 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8409d0b0 con 0x7f3f84066610 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f88b2e700 10 monclient(hunting): authenticate will time out at 1758.336385s 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 0x7f3f8406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 0x7f3f8406fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47762/0 (socket says 172.21.15.47:47762) 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 -- 172.21.15.47:0/27830000 learned_addr learned my addr 172.21.15.47:0/27830000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): get_auth_request con 0x7f3f840a2990 auth_method 0 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f84070140 0x7f3f841d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f84066610 0x7f3f8406f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): get_auth_request con 0x7f3f84070140 auth_method 0 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): get_auth_request con 0x7f3f84066610 auth_method 0 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f82d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient(hunting): handle_auth_done global_id 34317 payload 824 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f84066610 msgr2=0x7f3f8406f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3f84066610 0x7f3f8406f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f84070140 msgr2=0x7f3f841d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3f84070140 0x7f3f841d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 monclient: found mon.noname-a 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.935+0000 7f3f81d9b700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78009550 con 0x7f3f840a2990 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f81d9b700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 0x7f3f8406fc00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009840 tx=0x7f3f78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 client.?.objecter ms_handle_connect 0x7f3f840a2990 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f78004390 con 0x7f3f840a2990 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: dump: 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 5 monclient: authenticate success, global_id 34317 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7801f070 con 0x7f3f840a2990 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 10 monclient: _renew_subs 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84090600 con 0x7f3f840a2990 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f78004530 con 0x7f3f840a2990 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f80d99700 4 set_mon_vals no callback set 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 10 client.34317.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f72ffd700 10 monclient: dump: 2024-04-20T02:17:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 10 monclient: _renew_subs 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f84072990 con 0x7f3f840a2990 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.936+0000 7f3f88b2e700 1 librados: init done 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f88b2e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 0x7f3f841bda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84068180 con 0x7f3f84099e00 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3f780036a0 con 0x7f3f840a2990 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 0x7f3f841bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient: get_auth_request con 0x7f3f84099e00 auth_method 0 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3f6403a2f0 0x7f3f6403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 7958+0+0 (secure 0 0 0) 0x7f3f78021070 con 0x7f3f840a2990 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 10 client.34317.objecter ms_dispatch 0x7f3f84001550 osd_map(88..88 src has 1..88) v4 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 3 client.34317.objecter handle_osd_map got epochs [88,88] > 0 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 3 client.34317.objecter handle_osd_map decoding full epoch 88 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 10 monclient(hunting): handle_auth_done global_id 34235 payload 824 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f8406bd80 con 0x7f3f84099e00 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 20 client.34317.objecter dump_active .. 0 homeless 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3f6403a2f0 0x7f3f6403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f82d9d700 10 monclient: get_auth_request con 0x7f3f6403a2f0 auth_method 0 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 10 client.34317.objecter ms_handle_connect 0x7f3f84099e00 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 10 client.34317.objecter resend_mon_ops 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f8259c700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 0x7f3f841bda10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f740119d0 tx=0x7f3f74011d90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f740036a0 con 0x7f3f84099e00 2024-04-20T02:17:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.937+0000 7f3f72ffd700 10 monclient: dump: 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 10 client.34317.objecter ms_handle_connect 0x7f3f6403a2f0 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f82d9d700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3f6403a2f0 0x7f3f6403c7b0 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c011340 tx=0x7f3f7c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3f74004240 con 0x7f3f84099e00 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 msgr2=0x7f3f841bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 0x7f3f841bda10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f740119d0 tx=0x7f3f74011d90 comp rx=0 tx=0).stop 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 msgr2=0x7f3f841bda10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.938+0000 7f3f72ffd700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f84099e00 0x7f3f841bda10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f88b2e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 0x7f3f841bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f88b2e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840904b0 con 0x7f3f8406aac0 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 0x7f3f841bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient: get_auth_request con 0x7f3f8406aac0 auth_method 0 2024-04-20T02:17:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 10 monclient(hunting): handle_auth_done global_id 34244 payload 824 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 1 -- 172.21.15.47:0/27830000 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3f840669f0 con 0x7f3f8406aac0 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f72ffd700 10 client.34317.objecter ms_handle_connect 0x7f3f8406aac0 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f8259c700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 0x7f3f841bdf50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f7401b000 tx=0x7f3f74004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f72ffd700 10 client.34317.objecter resend_mon_ops 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f7400fec0 con 0x7f3f8406aac0 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f72ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f72ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.946+0000 7f3f72ffd700 10 monclient: dump: 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f3f74029400 con 0x7f3f8406aac0 2024-04-20T02:17:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 msgr2=0x7f3f841bdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 0x7f3f841bdf50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f7401b000 tx=0x7f3f74004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 msgr2=0x7f3f841bdf50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f72ffd700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3f8406aac0 0x7f3f841bdf50 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f7401b000 tx=0x7f3f74004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:58.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:58.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:17:58.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:58.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:58.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:58.986 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:58.986 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:58.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 10 librados: watch_flush enter 2024-04-20T02:17:58.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 10 librados: watch_flush exit 2024-04-20T02:17:58.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 20 client.34317.objecter shutdown clearing up homeless session... 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 10 client.34317.objecter successfully canceled tick 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3f6403a2f0 msgr2=0x7f3f6403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3f6403a2f0 0x7f3f6403c7b0 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c011340 tx=0x7f3f7c00b040 comp rx=0 tx=0).stop 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 10 monclient: shutdown 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 msgr2=0x7f3f8406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 0x7f3f8406fc00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009840 tx=0x7f3f78004750 comp rx=0 tx=0).stop 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f82d9d700 1 -- 172.21.15.47:0/27830000 reap_dead start 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 shutdown_connections 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 --2- 172.21.15.47:0/27830000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f840a2990 0x7f3f8406fc00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 >> 172.21.15.47:0/27830000 conn(0x7f3f840a3000 msgr2=0x7f3f841bfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 shutdown_connections 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 -- 172.21.15.47:0/27830000 wait complete. 2024-04-20T02:17:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:58.947+0000 7f3f88b2e700 1 librados: shutdown 2024-04-20T02:17:58.989 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:58.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:17:58.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.162+0000 7fde7c5b7700 10 monclient: get_monmap_and_config 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.162+0000 7fde7c5b7700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.162+0000 7fde7c5b7700 10 monclient: monmap: 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:59.163846+0000 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:59.163846+0000 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:59.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.163+0000 7fde7c5b7700 1 Processor -- start 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.163+0000 7fde7c5b7700 1 -- start start 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.163+0000 7fde7c5b7700 10 monclient: init 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient: _add_conns ranks=[4,3,1,8,5,10,7,9,0,6,2] 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 0x7fde740a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): picked mon.noname-c con 0x7fde740a5300 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde740a56e0 0x7fde740a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): picked mon.noname-g con 0x7fde740a56e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fde74137ab0 addr v1:172.21.15.179:6789/0 2024-04-20T02:17:59.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde740751e0 con 0x7fde74137ab0 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde740a3140 con 0x7fde740a56e0 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde740a1bf0 con 0x7fde740a5300 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7c5b7700 10 monclient(hunting): authenticate will time out at 1758.565160s 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 0x7fde740a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 0x7fde740a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:49934/0 (socket says 172.21.15.47:49934) 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 -- 172.21.15.47:0/1912976355 learned_addr learned my addr 172.21.15.47:0/1912976355 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): get_auth_request con 0x7fde740a5300 auth_method 0 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79b52700 1 --2- 172.21.15.47:0/1912976355 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde740a56e0 0x7fde740a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79b52700 10 monclient(hunting): get_auth_request con 0x7fde740a56e0 auth_method 0 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79b52700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79b52700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79b52700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient(hunting): handle_auth_done global_id 23062 payload 274 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 -- 172.21.15.47:0/1912976355 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde740a56e0 msgr2=0x7fde740a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 --2- 172.21.15.47:0/1912976355 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde740a56e0 0x7fde740a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 -- 172.21.15.47:0/1912976355 >> v1:172.21.15.179:6789/0 conn(0x7fde74137ab0 legacy=0x7fde74137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 monclient: found mon.noname-c 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 -- 172.21.15.47:0/1912976355 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740a1fd0 con 0x7fde740a5300 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde7a353700 1 --2- 172.21.15.47:0/1912976355 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 0x7fde740a3f20 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fde70009920 tx=0x7fde70009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3988ad02120ac6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79351700 1 -- 172.21.15.47:0/1912976355 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde70009e70 con 0x7fde740a5300 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.164+0000 7fde79351700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: dump: 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.180 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 5 monclient: authenticate success, global_id 23062 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 1 -- 172.21.15.47:0/1912976355 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7000bcd0 con 0x7fde740a5300 2024-04-20T02:17:59.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 1 -- 172.21.15.47:0/1912976355 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde7000be40 con 0x7fde740a5300 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 4 set_mon_vals no callback set 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde79351700 10 monclient: dump: 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.182 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 10 monclient: shutdown 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 -- 172.21.15.47:0/1912976355 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 msgr2=0x7fde740a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/1912976355 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 0x7fde740a3f20 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fde70009920 tx=0x7fde70009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 -- 172.21.15.47:0/1912976355 shutdown_connections 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/1912976355 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde740a5300 0x7fde740a3f20 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/1912976355 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde740a56e0 0x7fde740a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 -- 172.21.15.47:0/1912976355 >> 172.21.15.47:0/1912976355 conn(0x7fde740aa710 msgr2=0x7fde740aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 -- 172.21.15.47:0/1912976355 shutdown_connections 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.165+0000 7fde7c5b7700 1 -- 172.21.15.47:0/1912976355 wait complete. 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 10 monclient: monmap: 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:59.184 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 1 librados: starting msgr at 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 1 librados: starting objecter 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 1 Processor -- start 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 1 -- start start 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 1 librados: setting wanted keys 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 1 librados: calling monclient init 2024-04-20T02:17:59.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.166+0000 7fde7c5b7700 10 monclient: init 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient: _add_conns ranks=[0,8,4,2,1,3,5,7,6] 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde740a5300 0x7fde741c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): picked mon.noname-a con 0x7fde740a5300 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde740a56e0 0x7fde74137060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): picked mon.noname-e con 0x7fde740a56e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde74137ab0 0x7fde741cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): picked mon.noname-c con 0x7fde74137ab0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74124460 con 0x7fde740a5300 2024-04-20T02:17:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde7409de40 con 0x7fde74137ab0 2024-04-20T02:17:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fde740a2290 con 0x7fde740a56e0 2024-04-20T02:17:59.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7c5b7700 10 monclient(hunting): authenticate will time out at 1758.568492s 2024-04-20T02:17:59.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde740a5300 0x7fde741c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde740a5300 0x7fde741c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47770/0 (socket says 172.21.15.47:47770) 2024-04-20T02:17:59.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 1 -- 172.21.15.47:0/3351102573 learned_addr learned my addr 172.21.15.47:0/3351102573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): get_auth_request con 0x7fde740a5300 auth_method 0 2024-04-20T02:17:59.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde740a56e0 0x7fde74137060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde74137ab0 0x7fde741cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): get_auth_request con 0x7fde74137ab0 auth_method 0 2024-04-20T02:17:59.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde7ab54700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): get_auth_request con 0x7fde740a56e0 auth_method 0 2024-04-20T02:17:59.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.167+0000 7fde79b52700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 10 monclient(hunting): handle_auth_done global_id 23071 payload 824 2024-04-20T02:17:59.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde740a56e0 msgr2=0x7fde74137060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde740a56e0 0x7fde74137060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde740a5300 msgr2=0x7fde741c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde740a5300 0x7fde741c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 1 monclient: found mon.noname-c 2024-04-20T02:17:59.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.168+0000 7fde7ab54700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70009550 con 0x7fde74137ab0 2024-04-20T02:17:59.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.169+0000 7fde7ab54700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde74137ab0 0x7fde741cc680 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00cae0 tx=0x7fde6c00cdf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.169+0000 7fde6affd700 10 client.?.objecter ms_handle_connect 0x7fde74137ab0 2024-04-20T02:17:59.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.169+0000 7fde6affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:59.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde6c003e70 con 0x7fde74137ab0 2024-04-20T02:17:59.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:59.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: dump: 2024-04-20T02:17:59.196 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.196 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.196 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.196 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.196 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.196 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.197 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.198 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c001b80 con 0x7fde74137ab0 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde6c00abf0 con 0x7fde74137ab0 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 5 monclient: authenticate success, global_id 23071 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 10 monclient: _renew_subs 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde740a45e0 con 0x7fde74137ab0 2024-04-20T02:17:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde78b50700 4 set_mon_vals no callback set 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 10 client.23071.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 10 monclient: _renew_subs 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde74124180 con 0x7fde74137ab0 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde7c5b7700 1 librados: init done 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.170+0000 7fde6affd700 10 monclient: dump: 2024-04-20T02:17:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.201 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.202 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.171+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fde6c00a780 con 0x7fde74137ab0 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.171+0000 7fde7c5b7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.171+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 0x7fde741ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.171+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.171+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74124800 con 0x7fde7406aac0 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.171+0000 7fde6affd700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde5803a320 0x7fde5803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7fde6c04aed0 con 0x7fde74137ab0 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 client.23071.objecter ms_dispatch 0x7fde74001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 3 client.23071.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 3 client.23071.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 0x7fde741ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde79b52700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde5803a320 0x7fde5803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient: get_auth_request con 0x7fde7406aac0 auth_method 0 2024-04-20T02:17:59.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde79b52700 10 monclient: get_auth_request con 0x7fde5803a320 auth_method 0 2024-04-20T02:17:59.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 20 client.23071.objecter dump_active .. 0 homeless 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 10 monclient(hunting): handle_auth_done global_id 25813 payload 824 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde740a4460 con 0x7fde7406aac0 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 client.23071.objecter ms_handle_connect 0x7fde5803a320 2024-04-20T02:17:59.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde79b52700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde5803a320 0x7fde5803c7e0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7fde74122310 tx=0x7fde64008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 client.23071.objecter ms_handle_connect 0x7fde7406aac0 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 client.23071.objecter resend_mon_ops 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde7a353700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 0x7fde741ccbc0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fde70009c70 tx=0x7fde70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde70020070 con 0x7fde7406aac0 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:17:59.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.172+0000 7fde6affd700 10 monclient: dump: 2024-04-20T02:17:59.208 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.208 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.208 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.208 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.208 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.208 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.209 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.209 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.209 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.209 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.209 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fde7000bb60 con 0x7fde7406aac0 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:59.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 msgr2=0x7fde741ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 0x7fde741ccbc0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fde70009c70 tx=0x7fde70004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 msgr2=0x7fde741ccbc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.173+0000 7fde6affd700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde7406aac0 0x7fde741ccbc0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7c5b7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 0x7fde741cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7c5b7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74068230 con 0x7fde741375a0 2024-04-20T02:17:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 0x7fde741cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient: get_auth_request con 0x7fde741375a0 auth_method 0 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 10 monclient(hunting): handle_auth_done global_id 25822 payload 824 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 1 -- 172.21.15.47:0/3351102573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde74067f00 con 0x7fde741375a0 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde7a353700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 0x7fde741cd100 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fde740a6540 tx=0x7fde70011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde6affd700 10 client.23071.objecter ms_handle_connect 0x7fde741375a0 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.183+0000 7fde6affd700 10 client.23071.objecter resend_mon_ops 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde70025440 con 0x7fde741375a0 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 10 monclient: dump: 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.217 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fde7002cab0 con 0x7fde741375a0 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 msgr2=0x7fde741cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 0x7fde741cd100 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fde740a6540 tx=0x7fde70011000 comp rx=0 tx=0).stop 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 msgr2=0x7fde741cd100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde6affd700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde741375a0 0x7fde741cd100 secure :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fde740a6540 tx=0x7fde70011000 comp rx=0 tx=0).stop 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:59.219 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.220 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.221 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.222 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.223 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:59.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.227 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.227 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.227 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:59.230 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:59.231 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.234 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:59.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.237 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:59.238 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.239 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:59.240 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.241 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:59.242 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:59.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 10 librados: watch_flush enter 2024-04-20T02:17:59.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 10 librados: watch_flush exit 2024-04-20T02:17:59.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 20 client.23071.objecter shutdown clearing up homeless session... 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 10 client.23071.objecter successfully canceled tick 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde5803a320 msgr2=0x7fde5803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde5803a320 0x7fde5803c7e0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7fde74122310 tx=0x7fde64008040 comp rx=0 tx=0).stop 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 10 monclient: shutdown 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde74137ab0 msgr2=0x7fde741cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 --2- 172.21.15.47:0/3351102573 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde74137ab0 0x7fde741cc680 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00cae0 tx=0x7fde6c00cdf0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7ab54700 1 -- 172.21.15.47:0/3351102573 reap_dead start 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7ab54700 1 -- 172.21.15.47:0/3351102573 reap_dead start 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 shutdown_connections 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 >> 172.21.15.47:0/3351102573 conn(0x7fde740aa710 msgr2=0x7fde741bfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.184+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 shutdown_connections 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.185+0000 7fde7c5b7700 1 -- 172.21.15.47:0/3351102573 wait complete. 2024-04-20T02:17:59.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.185+0000 7fde7c5b7700 1 librados: shutdown 2024-04-20T02:17:59.245 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:59.245 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:17:59.245 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:17:59.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.385+0000 7fbfce268700 10 monclient: get_monmap_and_config 2024-04-20T02:17:59.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.385+0000 7fbfce268700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.385+0000 7fbfce268700 10 monclient: monmap: 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:59.386203+0000 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:59.386203+0000 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.386 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.386 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:59.386 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:59.386 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.385+0000 7fbfce268700 1 Processor -- start 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.385+0000 7fbfce268700 1 -- start start 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.385+0000 7fbfce268700 10 monclient: init 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient: _add_conns ranks=[6,7,3,0,8,9,10,5,2,4,1] 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 0x7fbfc8135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): picked mon.noname-d con 0x7fbfc809fa80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fe60 0x7fbfc80a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): picked mon.noname-i con 0x7fbfc809fe60 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc80a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): picked mon.noname-g con 0x7fbfc80a1f10 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80a0780 con 0x7fbfc80a1f10 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc809df90 con 0x7fbfc809fa80 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc809dcb0 con 0x7fbfc809fe60 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfce268700 10 monclient(hunting): authenticate will time out at 1758.787563s 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 0x7fbfc8135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 0x7fbfc8135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50188/0 (socket says 172.21.15.47:50188) 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 1 -- 172.21.15.47:0/4025922997 learned_addr learned my addr 172.21.15.47:0/4025922997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): get_auth_request con 0x7fbfc809fa80 auth_method 0 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc77fe700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fe60 0x7fbfc80a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfcc805700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc80a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc77fe700 10 monclient(hunting): get_auth_request con 0x7fbfc809fe60 auth_method 0 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfcc805700 10 monclient(hunting): get_auth_request con 0x7fbfc80a1f10 auth_method 0 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfcc805700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfcc805700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfcc805700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_done global_id 22569 payload 274 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 1 -- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fe60 msgr2=0x7fbfc80a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.386+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fe60 0x7fbfc80a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 1 -- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 msgr2=0x7fbfc80a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc80a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 1 monclient: found mon.noname-d 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 1 -- 172.21.15.47:0/4025922997 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80751e0 con 0x7fbfc809fa80 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 0x7fbfc8135df0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009920 tx=0x7fbfb8009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=573a45da08981f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 1 -- 172.21.15.47:0/4025922997 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb8009e70 con 0x7fbfc809fa80 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:17:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: dump: 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.401 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 1 -- 172.21.15.47:0/4025922997 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800bcd0 con 0x7fbfc809fa80 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 5 monclient: authenticate success, global_id 22569 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 4 set_mon_vals no callback set 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 1 -- 172.21.15.47:0/4025922997 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb800be40 con 0x7fbfc809fa80 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfc6ffd700 10 monclient: dump: 2024-04-20T02:17:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 10 monclient: shutdown 2024-04-20T02:17:59.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 1 -- 172.21.15.47:0/4025922997 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 msgr2=0x7fbfc8135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.387+0000 7fbfce268700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 0x7fbfc8135df0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009920 tx=0x7fbfb8009c30 comp rx=0 tx=0).stop 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 -- 172.21.15.47:0/4025922997 shutdown_connections 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc80a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbfc809fa80 0x7fbfc8135df0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 --2- 172.21.15.47:0/4025922997 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fe60 0x7fbfc80a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 -- 172.21.15.47:0/4025922997 >> 172.21.15.47:0/4025922997 conn(0x7fbfc80a3000 msgr2=0x7fbfc80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 -- 172.21.15.47:0/4025922997 shutdown_connections 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 -- 172.21.15.47:0/4025922997 wait complete. 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 10 monclient: monmap: 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:59.404 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 librados: starting msgr at 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.388+0000 7fbfce268700 1 librados: starting objecter 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 Processor -- start 2024-04-20T02:17:59.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 -- start start 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 librados: setting wanted keys 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 librados: calling monclient init 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient: init 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient: _add_conns ranks=[7,0,3,4,5,2,1,6,8] 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fa80 0x7fbfc812b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): picked mon.noname-i con 0x7fbfc809fa80 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc809fe60 0x7fbfc812bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): picked mon.noname-a con 0x7fbfc809fe60 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc812c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): picked mon.noname-g con 0x7fbfc80a1f10 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8132890 con 0x7fbfc809fe60 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc812e5e0 con 0x7fbfc80a1f10 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc812eba0 con 0x7fbfc809fa80 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.389+0000 7fbfce268700 10 monclient(hunting): authenticate will time out at 1758.790938s 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc809fe60 0x7fbfc812bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc809fe60 0x7fbfc812bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47780/0 (socket says 172.21.15.47:47780) 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 -- 172.21.15.47:0/2080740394 learned_addr learned my addr 172.21.15.47:0/2080740394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): get_auth_request con 0x7fbfc809fe60 auth_method 0 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fa80 0x7fbfc812b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc812c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): get_auth_request con 0x7fbfc809fa80 auth_method 0 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): get_auth_request con 0x7fbfc80a1f10 auth_method 0 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfcc805700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient(hunting): handle_auth_done global_id 34335 payload 824 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fa80 msgr2=0x7fbfc812b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbfc809fa80 0x7fbfc812b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 msgr2=0x7fbfc812c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc80a1f10 0x7fbfc812c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 monclient: found mon.noname-a 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8009550 con 0x7fbfc809fe60 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc77fe700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc809fe60 0x7fbfc812bb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00e810 tx=0x7fbfbc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 client.?.objecter ms_handle_connect 0x7fbfc809fe60 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfbc00c070 con 0x7fbfc809fe60 2024-04-20T02:17:59.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: dump: 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc013880 con 0x7fbfc809fe60 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfce268700 5 monclient: authenticate success, global_id 34335 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfce268700 10 monclient: _renew_subs 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfce268700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc809cb50 con 0x7fbfc809fe60 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfbc013e60 con 0x7fbfc809fe60 2024-04-20T02:17:59.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc4ff9700 10 monclient: dump: 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.412 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.390+0000 7fbfc67fc700 4 set_mon_vals no callback set 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 10 client.34335.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 10 monclient: _renew_subs 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc812e300 con 0x7fbfc809fe60 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 1 librados: init done 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 0x7fbfc806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8068180 con 0x7fbfc806aac0 2024-04-20T02:17:59.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.391+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbfbc0139f0 con 0x7fbfc809fe60 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 0x7fbfc806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient: get_auth_request con 0x7fbfc806aac0 auth_method 0 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfb403a2f0 0x7fbfb403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7fbfbc04ae00 con 0x7fbfc809fe60 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 client.34335.objecter ms_dispatch 0x7fbfc8001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 3 client.34335.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 3 client.34335.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:17:59.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfcc805700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfb403a2f0 0x7fbfb403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfcc805700 10 monclient: get_auth_request con 0x7fbfb403a2f0 auth_method 0 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 20 client.34335.objecter dump_active .. 0 homeless 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_done global_id 21072 payload 824 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc809e550 con 0x7fbfc806aac0 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 client.34335.objecter ms_handle_connect 0x7fbfc806aac0 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 client.34335.objecter resend_mon_ops 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 0x7fbfc806ef30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009fd0 tx=0x7fbfb8004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb8020070 con 0x7fbfc806aac0 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 monclient: dump: 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.415 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfcc805700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfb403a2f0 0x7fbfb403c7b0 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7fbfc812e230 tx=0x7fbfc0009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.392+0000 7fbfc4ff9700 10 client.34335.objecter ms_handle_connect 0x7fbfb403a2f0 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbfb800bb60 con 0x7fbfc806aac0 2024-04-20T02:17:59.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 msgr2=0x7fbfc806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 0x7fbfc806ef30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009fd0 tx=0x7fbfb8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 msgr2=0x7fbfc806ef30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.393+0000 7fbfc4ff9700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806aac0 0x7fbfc806ef30 secure :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009fd0 tx=0x7fbfb8004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.400+0000 7fbfce268700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.400+0000 7fbfce268700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 0x7fbfc806f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.400+0000 7fbfce268700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.400+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc806a400 con 0x7fbfc806f470 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 0x7fbfc806f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient: get_auth_request con 0x7fbfc806f470 auth_method 0 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 10 monclient(hunting): handle_auth_done global_id 21081 payload 824 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 1 -- 172.21.15.47:0/2080740394 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbfc80a0930 con 0x7fbfc806f470 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc4ff9700 10 client.34335.objecter ms_handle_connect 0x7fbfc806f470 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc7fff700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 0x7fbfc806f880 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009130 tx=0x7fbfb80092c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc4ff9700 10 client.34335.objecter resend_mon_ops 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb802ac00 con 0x7fbfc806f470 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.401+0000 7fbfc4ff9700 10 monclient: dump: 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.420 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fbfb80045e0 con 0x7fbfc806f470 2024-04-20T02:17:59.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 msgr2=0x7fbfc806f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 0x7fbfc806f880 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009130 tx=0x7fbfb80092c0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 msgr2=0x7fbfc806f880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfc4ff9700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbfc806f470 0x7fbfc806f880 secure :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009130 tx=0x7fbfb80092c0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:59.422 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.423 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.425 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.426 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:59.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.428 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:59.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:59.432 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:59.435 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:59.441 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.442 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.443 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:59.444 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 10 librados: watch_flush enter 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 10 librados: watch_flush exit 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 20 client.34335.objecter shutdown clearing up homeless session... 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 10 client.34335.objecter successfully canceled tick 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfb403a2f0 msgr2=0x7fbfb403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfb403a2f0 0x7fbfb403c7b0 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7fbfc812e230 tx=0x7fbfc0009780 comp rx=0 tx=0).stop 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 10 monclient: shutdown 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc809fe60 msgr2=0x7fbfc812bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 --2- 172.21.15.47:0/2080740394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbfc809fe60 0x7fbfc812bb00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00e810 tx=0x7fbfbc00eb20 comp rx=0 tx=0).stop 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfcc805700 1 -- 172.21.15.47:0/2080740394 reap_dead start 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfcc805700 1 -- 172.21.15.47:0/2080740394 reap_dead start 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 shutdown_connections 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 >> 172.21.15.47:0/2080740394 conn(0x7fbfc80a3000 msgr2=0x7fbfc809ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 shutdown_connections 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 -- 172.21.15.47:0/2080740394 wait complete. 2024-04-20T02:17:59.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.402+0000 7fbfce268700 1 librados: shutdown 2024-04-20T02:17:59.447 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:59.447 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:17:59.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.586+0000 7fbe59297700 10 monclient: get_monmap_and_config 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.586+0000 7fbe59297700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.586+0000 7fbe59297700 10 monclient: monmap: 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:59.587300+0000 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:59.587300+0000 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:59.586 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.586+0000 7fbe59297700 1 Processor -- start 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.586+0000 7fbe59297700 1 -- start start 2024-04-20T02:17:59.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.586+0000 7fbe59297700 10 monclient: init 2024-04-20T02:17:59.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient: _add_conns ranks=[8,7,6,5,4,0,2,1,3,9,10] 2024-04-20T02:17:59.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe54071730 0x7fbe54071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): picked mon.noname-e con 0x7fbe54071730 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe540a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): picked mon.noname-i con 0x7fbe5406f020 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 0x7fbe540a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): picked mon.noname-d con 0x7fbe540a2db0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5409fea0 con 0x7fbe540a2db0 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5409e7b0 con 0x7fbe5406f020 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5406d190 con 0x7fbe54071730 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe59297700 10 monclient(hunting): authenticate will time out at 1758.988388s 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 0x7fbe540a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe54071730 0x7fbe54071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 0x7fbe540a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50190/0 (socket says 172.21.15.47:50190) 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 -- 172.21.15.47:0/2936121517 learned_addr learned my addr 172.21.15.47:0/2936121517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): get_auth_request con 0x7fbe540a2db0 auth_method 0 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): get_auth_request con 0x7fbe54071730 auth_method 0 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe540a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): get_auth_request con 0x7fbe5406f020 auth_method 0 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe527fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient(hunting): handle_auth_done global_id 22578 payload 274 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 -- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe54071730 msgr2=0x7fbe54071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe54071730 0x7fbe54071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 -- 172.21.15.47:0/2936121517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 msgr2=0x7fbe540a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe540a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 monclient: found mon.noname-d 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe537fe700 1 -- 172.21.15.47:0/2936121517 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540751e0 con 0x7fbe540a2db0 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.587+0000 7fbe52ffd700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe54071730 0x7fbe54071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe537fe700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 0x7fbe540a3210 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009e80 tx=0x7fbe4c00b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=357edb4ee0e58c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 1 -- 172.21.15.47:0/2936121517 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4c00e070 con 0x7fbe540a2db0 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: dump: 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.602 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 5 monclient: authenticate success, global_id 22578 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 1 -- 172.21.15.47:0/2936121517 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c0099b0 con 0x7fbe540a2db0 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 1 -- 172.21.15.47:0/2936121517 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4c00bd80 con 0x7fbe540a2db0 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 4 set_mon_vals no callback set 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe51ffb700 10 monclient: dump: 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.604 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 10 monclient: shutdown 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 -- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 msgr2=0x7fbe540a3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 0x7fbe540a3210 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009e80 tx=0x7fbe4c00b330 comp rx=0 tx=0).stop 2024-04-20T02:17:59.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 -- 172.21.15.47:0/2936121517 shutdown_connections 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbe540a2db0 0x7fbe540a3210 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe54071730 0x7fbe54071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 --2- 172.21.15.47:0/2936121517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe540a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.588+0000 7fbe59297700 1 -- 172.21.15.47:0/2936121517 >> 172.21.15.47:0/2936121517 conn(0x7fbe540a3f90 msgr2=0x7fbe5406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.589+0000 7fbe59297700 1 -- 172.21.15.47:0/2936121517 shutdown_connections 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.589+0000 7fbe59297700 1 -- 172.21.15.47:0/2936121517 wait complete. 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.589+0000 7fbe59297700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.589+0000 7fbe59297700 10 monclient: monmap: 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:59.606 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.589+0000 7fbe59297700 1 librados: starting msgr at 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.589+0000 7fbe59297700 1 librados: starting objecter 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 Processor -- start 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 -- start start 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 librados: setting wanted keys 2024-04-20T02:17:59.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 librados: calling monclient init 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient: init 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient: _add_conns ranks=[7,2,8,1,3,0,4,5,6] 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe541c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): picked mon.noname-i con 0x7fbe5406f020 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbe54071730 0x7fbe541c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): picked mon.noname-b con 0x7fbe54071730 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 0x7fbe54120de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): picked mon.noname-e con 0x7fbe540a2db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54124400 con 0x7fbe54071730 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54123510 con 0x7fbe5406f020 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5409f1b0 con 0x7fbe540a2db0 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe59297700 10 monclient(hunting): authenticate will time out at 1758.991783s 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe537fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 0x7fbe54120de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbe54071730 0x7fbe541c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbe54071730 0x7fbe541c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39478/0 (socket says 172.21.15.47:39478) 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe537fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 0x7fbe54120de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39916/0 (socket says 172.21.15.47:39916) 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 1 -- 172.21.15.47:0/1546651517 learned_addr learned my addr 172.21.15.47:0/1546651517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 10 monclient(hunting): get_auth_request con 0x7fbe54071730 auth_method 0 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe527fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe537fe700 10 monclient(hunting): get_auth_request con 0x7fbe540a2db0 auth_method 0 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe537fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe537fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe537fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.590+0000 7fbe52ffd700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe541c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): get_auth_request con 0x7fbe5406f020 auth_method 0 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe527fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe527fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe527fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 10 monclient(hunting): handle_auth_done global_id 34271 payload 824 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 msgr2=0x7fbe541c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbe5406f020 0x7fbe541c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbe54071730 msgr2=0x7fbe541c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbe54071730 0x7fbe541c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 monclient: found mon.noname-e 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440094a0 con 0x7fbe540a2db0 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe537fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 0x7fbe54120de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c015000 tx=0x7fbe4c004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 client.?.objecter ms_handle_connect 0x7fbe540a2db0 2024-04-20T02:17:59.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4c0092e0 con 0x7fbe540a2db0 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: dump: 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.612 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c003e50 con 0x7fbe540a2db0 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 5 monclient: authenticate success, global_id 34271 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 10 monclient: _renew_subs 2024-04-20T02:17:59.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c009b30 con 0x7fbe540a2db0 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4c0046c0 con 0x7fbe540a2db0 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe517fa700 4 set_mon_vals no callback set 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe3b7fe700 10 monclient: dump: 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.614 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 10 client.34271.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 10 monclient: _renew_subs 2024-04-20T02:17:59.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe54124ca0 con 0x7fbe540a2db0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.591+0000 7fbe59297700 1 librados: init done 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbe4c004820 con 0x7fbe540a2db0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe59297700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe59297700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 0x7fbe54121320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54068280 con 0x7fbe5406aac0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe3b7fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbe4003a2e0 0x7fbe4003c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 0x7fbe54121320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7fbe4c04c430 con 0x7fbe540a2db0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe3b7fe700 10 client.34271.objecter ms_dispatch 0x7fbe54001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe3b7fe700 3 client.34271.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe3b7fe700 3 client.34271.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient: get_auth_request con 0x7fbe5406aac0 auth_method 0 2024-04-20T02:17:59.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.592+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 20 client.34271.objecter dump_active .. 0 homeless 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe527fc700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbe4003a2e0 0x7fbe4003c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe527fc700 10 monclient: get_auth_request con 0x7fbe4003a2e0 auth_method 0 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_done global_id 34241 payload 824 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe52ffd700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe5406c3a0 con 0x7fbe5406aac0 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe52ffd700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 0x7fbe54121320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011b50 tx=0x7fbe44011f10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 client.34271.objecter ms_handle_connect 0x7fbe5406aac0 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 client.34271.objecter resend_mon_ops 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 client.34271.objecter ms_handle_connect 0x7fbe4003a2e0 2024-04-20T02:17:59.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe527fc700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbe4003a2e0 0x7fbe4003c7a0 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7fbe54122540 tx=0x7fbe48009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe44014440 con 0x7fbe5406aac0 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 monclient: dump: 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.618 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbe4400fac0 con 0x7fbe5406aac0 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 msgr2=0x7fbe54121320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 0x7fbe54121320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011b50 tx=0x7fbe44011f10 comp rx=0 tx=0).stop 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 msgr2=0x7fbe54121320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.593+0000 7fbe3b7fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe5406aac0 0x7fbe54121320 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011b50 tx=0x7fbe44011f10 comp rx=0 tx=0).stop 2024-04-20T02:17:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.601+0000 7fbe59297700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:59.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.601+0000 7fbe59297700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 0x7fbe54121860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.601+0000 7fbe59297700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.601+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5406a400 con 0x7fbe541250c0 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.601+0000 7fbe52ffd700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 0x7fbe54121860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient: get_auth_request con 0x7fbe541250c0 auth_method 0 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 10 monclient(hunting): handle_auth_done global_id 34250 payload 824 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 1 -- 172.21.15.47:0/1546651517 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbe5409dac0 con 0x7fbe541250c0 2024-04-20T02:17:59.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe52ffd700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 0x7fbe54121860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011b50 tx=0x7fbe44004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe3b7fe700 10 client.34271.objecter ms_handle_connect 0x7fbe541250c0 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe3b7fe700 10 client.34271.objecter resend_mon_ops 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4402c8a0 con 0x7fbe541250c0 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe3b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.602+0000 7fbe3b7fe700 10 monclient: dump: 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.623 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fbe4402c430 con 0x7fbe541250c0 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 msgr2=0x7fbe54121860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 0x7fbe54121860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011b50 tx=0x7fbe44004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 msgr2=0x7fbe54121860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe3b7fe700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbe541250c0 0x7fbe54121860 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011b50 tx=0x7fbe44004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:59.625 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.626 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:59.627 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:59.628 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.629 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.630 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.631 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.636 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:59.637 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.639 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.644 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.645 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:59.646 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 10 librados: watch_flush enter 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 10 librados: watch_flush exit 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 20 client.34271.objecter shutdown clearing up homeless session... 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 10 client.34271.objecter successfully canceled tick 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbe4003a2e0 msgr2=0x7fbe4003c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbe4003a2e0 0x7fbe4003c7a0 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7fbe54122540 tx=0x7fbe48009210 comp rx=0 tx=0).stop 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 10 monclient: shutdown 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 msgr2=0x7fbe54120de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe537fe700 1 -- 172.21.15.47:0/1546651517 reap_dead start 2024-04-20T02:17:59.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 0x7fbe54120de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c015000 tx=0x7fbe4c004ad0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 shutdown_connections 2024-04-20T02:17:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 --2- 172.21.15.47:0/1546651517 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbe540a2db0 0x7fbe54120de0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 >> 172.21.15.47:0/1546651517 conn(0x7fbe540a3f90 msgr2=0x7fbe540a54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 shutdown_connections 2024-04-20T02:17:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 -- 172.21.15.47:0/1546651517 wait complete. 2024-04-20T02:17:59.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.603+0000 7fbe59297700 1 librados: shutdown 2024-04-20T02:17:59.649 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:59.649 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:17:59.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:17:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.788+0000 7fc2e10e9700 10 monclient: get_monmap_and_config 2024-04-20T02:17:59.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.788+0000 7fc2e10e9700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.788+0000 7fc2e10e9700 10 monclient: monmap: 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:59.789349+0000 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:59.789349+0000 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.788 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:59.789 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:59.789 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.788+0000 7fc2e10e9700 1 Processor -- start 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.788+0000 7fc2e10e9700 1 -- start start 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.788+0000 7fc2e10e9700 10 monclient: init 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient: _add_conns ranks=[4,10,7,0,1,2,9,6,3,5,8] 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): picked mon.noname-c con 0x7fc2dc0a3960 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc2dc0a3d40 0x7fc2dc0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): picked mon.noname-f con 0x7fc2dc0a3d40 addr v2:172.21.15.179:3300/0 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc134700 0x7fc2dc134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): picked mon.noname-i con 0x7fc2dc134700 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc06ca20 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc06e110 con 0x7fc2dc0a3960 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc06e3f0 con 0x7fc2dc134700 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2e10e9700 10 monclient(hunting): authenticate will time out at 1759.190447s 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:49944/0 (socket says 172.21.15.47:49944) 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 -- 172.21.15.47:0/135392792 learned_addr learned my addr 172.21.15.47:0/135392792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): get_auth_request con 0x7fc2dc0a3960 auth_method 0 2024-04-20T02:17:59.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2d259c700 1 --2- 172.21.15.47:0/135392792 >> v2:172.21.15.179:3300/0 conn(0x7fc2dc0a3d40 0x7fc2dc0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2db59e700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc134700 0x7fc2dc134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2d259c700 10 monclient(hunting): get_auth_request con 0x7fc2dc0a3d40 auth_method 0 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2d259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2d259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2d259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2db59e700 10 monclient(hunting): get_auth_request con 0x7fc2dc134700 auth_method 0 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2db59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2db59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2db59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_done global_id 23080 payload 274 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 -- 172.21.15.47:0/135392792 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc134700 msgr2=0x7fc2dc134bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc134700 0x7fc2dc134bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 -- 172.21.15.47:0/135392792 >> v2:172.21.15.179:3300/0 conn(0x7fc2dc0a3d40 msgr2=0x7fc2dc0a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/135392792 >> v2:172.21.15.179:3300/0 conn(0x7fc2dc0a3d40 0x7fc2dc0a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 monclient: found mon.noname-c 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2dad9d700 1 -- 172.21.15.47:0/135392792 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0751e0 con 0x7fc2dc0a3960 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2db59e700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc134700 0x7fc2dc134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.789+0000 7fc2d259c700 1 --2- 172.21.15.47:0/135392792 >> v2:172.21.15.179:3300/0 conn(0x7fc2dc0a3d40 0x7fc2dc0a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc070590 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098f0 tx=0x7fc2cc009c00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=162b898eaffa8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 1 -- 172.21.15.47:0/135392792 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2cc00bb10 con 0x7fc2dc0a3960 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: dump: 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.803 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 1 -- 172.21.15.47:0/135392792 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc00bc70 con 0x7fc2dc0a3960 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 5 monclient: authenticate success, global_id 23080 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 10 monclient: get_monmap_and_config success 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 1 -- 172.21.15.47:0/135392792 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2cc00bde0 con 0x7fc2dc0a3960 2024-04-20T02:17:59.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 4 set_mon_vals no callback set 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2da59c700 10 monclient: dump: 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.805 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 10 monclient: shutdown 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 -- 172.21.15.47:0/135392792 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 msgr2=0x7fc2dc070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc070590 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098f0 tx=0x7fc2cc009c00 comp rx=0 tx=0).stop 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 -- 172.21.15.47:0/135392792 shutdown_connections 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc134700 0x7fc2dc134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/135392792 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc070590 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/135392792 >> v2:172.21.15.179:3300/0 conn(0x7fc2dc0a3d40 0x7fc2dc0a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.790+0000 7fc2e10e9700 1 -- 172.21.15.47:0/135392792 >> 172.21.15.47:0/135392792 conn(0x7fc2dc0a8610 msgr2=0x7fc2dc0a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.791+0000 7fc2e10e9700 1 -- 172.21.15.47:0/135392792 shutdown_connections 2024-04-20T02:17:59.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.791+0000 7fc2e10e9700 1 -- 172.21.15.47:0/135392792 wait complete. 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.791+0000 7fc2e10e9700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.791+0000 7fc2e10e9700 10 monclient: monmap: 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.807 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 librados: starting msgr at 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 librados: starting objecter 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 Processor -- start 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 -- start start 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 librados: setting wanted keys 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 librados: calling monclient init 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient: init 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient: _reopen_session rank -1 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient: _add_conns ranks=[5,4,8,2,0,7,1,3,6] 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc0a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): picked mon.noname-h con 0x7fc2dc0a3960 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3d40 0x7fc2dc133a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): picked mon.noname-c con 0x7fc2dc0a3d40 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2dc134700 0x7fc2dc1cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): picked mon.noname-e con 0x7fc2dc134700 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc121120 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc11ea60 con 0x7fc2dc0a3960 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc071f80 con 0x7fc2dc134700 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): _renew_subs 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.792+0000 7fc2e10e9700 10 monclient(hunting): authenticate will time out at 1759.193920s 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2dc134700 0x7fc2dc1cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2dc134700 0x7fc2dc1cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39920/0 (socket says 172.21.15.47:39920) 2024-04-20T02:17:59.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 1 -- 172.21.15.47:0/953522492 learned_addr learned my addr 172.21.15.47:0/953522492 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): get_auth_request con 0x7fc2dc134700 auth_method 0 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3d40 0x7fc2dc133a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): get_auth_request con 0x7fc2dc0a3d40 auth_method 0 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc0a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): get_auth_request con 0x7fc2dc0a3960 auth_method 0 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient(hunting): handle_auth_done global_id 23089 payload 824 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2dc134700 msgr2=0x7fc2dc1cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2dc134700 0x7fc2dc1cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc2dc0a3960 msgr2=0x7fc2dc0a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc2dc0a3960 0x7fc2dc0a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:17:59.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient: _finish_hunting 0 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 monclient: found mon.noname-c 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc009550 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2db59e700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2dc134700 0x7fc2dc1cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d259c700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3d40 0x7fc2dc133a40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400e810 tx=0x7fc2c400eb20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 client.?.objecter ms_handle_connect 0x7fc2dc0a3d40 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2c400c070 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: dump: 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.812 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: _finish_auth 0 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:17:59.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2e10e9700 5 monclient: authenticate success, global_id 23089 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c4013880 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2e10e9700 10 monclient: _renew_subs 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2e10e9700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc120b60 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2c4013e60 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d9d9b700 4 set_mon_vals no callback set 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 10 client.23089.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.793+0000 7fc2d3fff700 10 monclient: dump: 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.814 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 10 monclient: _renew_subs 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc11f780 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 1 librados: init done 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc2c40139f0 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 0x7fc2dc1cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc068380 con 0x7fc2dc133f80 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.794+0000 7fc2d3fff700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc2bc03a2f0 0x7fc2bc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7fc2c404adc0 con 0x7fc2dc0a3d40 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 client.23089.objecter ms_dispatch 0x7fc2dc001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 3 client.23089.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 3 client.23089.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 0x7fc2dc1cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient: get_auth_request con 0x7fc2dc133f80 auth_method 0 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2db59e700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc2bc03a2f0 0x7fc2bc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 20 client.23089.objecter dump_active .. 0 homeless 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2db59e700 10 monclient: get_auth_request con 0x7fc2bc03a2f0 auth_method 0 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_done global_id 21004 payload 824 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2dc067fa0 con 0x7fc2dc133f80 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 client.23089.objecter ms_handle_connect 0x7fc2dc133f80 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 client.23089.objecter resend_mon_ops 2024-04-20T02:17:59.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 0x7fc2dc1cc690 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098f0 tx=0x7fc2cc004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 client.23089.objecter ms_handle_connect 0x7fc2bc03a2f0 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2db59e700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc2bc03a2f0 0x7fc2bc03c7b0 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc120400 tx=0x7fc2d4009870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2cc020070 con 0x7fc2dc133f80 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.795+0000 7fc2d3fff700 10 monclient: dump: 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.818 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc2cc004250 con 0x7fc2dc133f80 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 msgr2=0x7fc2dc1cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 0x7fc2dc1cc690 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098f0 tx=0x7fc2cc004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 msgr2=0x7fc2dc1cc690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.796+0000 7fc2d3fff700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc133f80 0x7fc2dc1cc690 secure :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098f0 tx=0x7fc2cc004750 comp rx=0 tx=0).stop 2024-04-20T02:17:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.803+0000 7fc2e10e9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:17:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.803+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 0x7fc2dc1ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:17:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.803+0000 7fc2e10e9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:17:59.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.803+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc120cf0 con 0x7fc2dc06aac0 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 0x7fc2dc1ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient: get_auth_request con 0x7fc2dc06aac0 auth_method 0 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 10 monclient(hunting): handle_auth_done global_id 21013 payload 824 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 1 -- 172.21.15.47:0/953522492 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc2dc070200 con 0x7fc2dc06aac0 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2dad9d700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 0x7fc2dc1ccbd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009840 tx=0x7fc2cc011000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2d3fff700 10 client.23089.objecter ms_handle_connect 0x7fc2dc06aac0 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2d3fff700 10 client.23089.objecter resend_mon_ops 2024-04-20T02:17:59.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2cc004600 con 0x7fc2dc06aac0 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2d3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2d3fff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.804+0000 7fc2d3fff700 10 monclient: dump: 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:17:59.823 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fc2cc02b830 con 0x7fc2dc06aac0 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 msgr2=0x7fc2dc1ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 0x7fc2dc1ccbd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009840 tx=0x7fc2cc011000 comp rx=0 tx=0).stop 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 msgr2=0x7fc2dc1ccbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2d3fff700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc2dc06aac0 0x7fc2dc1ccbd0 secure :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009840 tx=0x7fc2cc011000 comp rx=0 tx=0).stop 2024-04-20T02:17:59.824 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 18, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:17:59.825 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.826 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.827 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:17:59.828 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:17:59.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:17:59.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:17:59.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.837 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:17:59.838 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.840 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:17:59.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:17:59.843 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:17:59.844 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:17:59.845 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 10 librados: watch_flush enter 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 10 librados: watch_flush exit 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 20 client.23089.objecter shutdown clearing up homeless session... 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 10 client.23089.objecter successfully canceled tick 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc2bc03a2f0 msgr2=0x7fc2bc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc2bc03a2f0 0x7fc2bc03c7b0 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc120400 tx=0x7fc2d4009870 comp rx=0 tx=0).stop 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 10 monclient: shutdown 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3d40 msgr2=0x7fc2dc133a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:17:59.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 --2- 172.21.15.47:0/953522492 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2dc0a3d40 0x7fc2dc133a40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400e810 tx=0x7fc2c400eb20 comp rx=0 tx=0).stop 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2db59e700 1 -- 172.21.15.47:0/953522492 reap_dead start 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2db59e700 1 -- 172.21.15.47:0/953522492 reap_dead start 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 shutdown_connections 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 >> 172.21.15.47:0/953522492 conn(0x7fc2dc0a8610 msgr2=0x7fc2dc070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 shutdown_connections 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 -- 172.21.15.47:0/953522492 wait complete. 2024-04-20T02:17:59.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.805+0000 7fc2e10e9700 1 librados: shutdown 2024-04-20T02:17:59.848 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:17:59.848 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h', 'i', 'd', 'c'] 2024-04-20T02:17:59.849 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-20T02:17:59.849 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-04-20T02:17:59.849 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h store 2024-04-20T02:17:59.849 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h sync_force --yes-i-really-mean-it 2024-04-20T02:17:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.988+0000 7ff7cc22e700 10 monclient: get_monmap_and_config 2024-04-20T02:17:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.988+0000 7ff7cc22e700 10 monclient: build_initial_monmap 2024-04-20T02:17:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.988+0000 7ff7cc22e700 10 monclient: monmap: 2024-04-20T02:17:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:17:59.989853+0000 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:17:59.989853+0000 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:17:59.989 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:17:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:17:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:17:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:17:59.990 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:17:59.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.989+0000 7ff7cc22e700 1 Processor -- start 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.989+0000 7ff7cc22e700 1 -- start start 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.989+0000 7ff7cc22e700 10 monclient: init 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient: _add_conns ranks=[7,0,4,2,9,10,5,6,8,3,1] 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c40a4150 0x7ff7c40a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): picked mon.noname-i con 0x7ff7c40a4150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff7c40a2e90 addr v1:172.21.15.47:6789/0 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c40a32d0 0x7ff7c413c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): picked mon.noname-c con 0x7ff7c40a32d0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c40751e0 con 0x7ff7c40a2e90 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c409f4f0 con 0x7ff7c40a32d0 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40a4530 con 0x7ff7c40a4150 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7cc22e700 10 monclient(hunting): authenticate will time out at 1759.391261s 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c97c9700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7c40a2e90 0x7ff7c413c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48236/0 (socket says 172.21.15.47:48236) 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c97c9700 1 -- 172.21.15.47:0/3605620955 learned_addr learned my addr 172.21.15.47:0/3605620955 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c40a32d0 0x7ff7c413c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): get_auth_request con 0x7ff7c40a32d0 auth_method 0 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c40a4150 0x7ff7c40a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): get_auth_request con 0x7ff7c40a4150 auth_method 0 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c8fc8700 1 -- 172.21.15.47:0/3605620955 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722465926 0 0) 0x7ff7c40751e0 con 0x7ff7c40a2e90 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c8fc8700 10 monclient(hunting): _negotiate 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c8fc8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c8fc8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c8fc8700 10 monclient(hunting): my global_id is 34344 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c8fc8700 1 -- 172.21.15.47:0/3605620955 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b00035b0 con 0x7ff7c40a2e90 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_done global_id 23098 payload 274 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 -- 172.21.15.47:0/3605620955 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c40a4150 msgr2=0x7ff7c40a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c40a4150 0x7ff7c40a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 -- 172.21.15.47:0/3605620955 >> v1:172.21.15.47:6789/0 conn(0x7ff7c40a2e90 legacy=0x7ff7c413c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 monclient: found mon.noname-c 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 -- 172.21.15.47:0/3605620955 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40a3780 con 0x7ff7c40a32d0 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.990+0000 7ff7ca7cb700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c40a32d0 0x7ff7c413c9e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009dc0 tx=0x7ff7bc00b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9d461f9bd87463f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 1 -- 172.21.15.47:0/3605620955 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7bc00e070 con 0x7ff7c40a32d0 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: dump: 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.005 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 5 monclient: authenticate success, global_id 23098 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 1 -- 172.21.15.47:0/3605620955 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc012430 con 0x7ff7c40a32d0 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 1 -- 172.21.15.47:0/3605620955 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7bc012a10 con 0x7ff7c40a32d0 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 4 set_mon_vals no callback set 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7c8fc8700 10 monclient: dump: 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.007 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 10 monclient: shutdown 2024-04-20T02:18:00.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 -- 172.21.15.47:0/3605620955 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c40a32d0 msgr2=0x7ff7c413c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c40a32d0 0x7ff7c413c9e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009dc0 tx=0x7ff7bc00b330 comp rx=0 tx=0).stop 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 -- 172.21.15.47:0/3605620955 shutdown_connections 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c40a32d0 0x7ff7c413c9e0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/3605620955 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c40a4150 0x7ff7c40a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 -- 172.21.15.47:0/3605620955 >> 172.21.15.47:0/3605620955 conn(0x7ff7c40a7570 msgr2=0x7ff7c40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 -- 172.21.15.47:0/3605620955 shutdown_connections 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.991+0000 7ff7cc22e700 1 -- 172.21.15.47:0/3605620955 wait complete. 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 10 monclient: monmap: 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.010 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 1 librados: starting msgr at 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 1 librados: starting objecter 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 1 Processor -- start 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 1 -- start start 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 1 librados: setting wanted keys 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 1 librados: calling monclient init 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.992+0000 7ff7cc22e700 10 monclient: init 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient: _add_conns ranks=[2,3,0,1,4,7,5,8,6] 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 0x7ff7c41d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): picked mon.noname-b con 0x7ff7c40a2e90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c40a32d0 0x7ff7c41d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): picked mon.noname-g con 0x7ff7c40a32d0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7c40a4150 0x7ff7c41d1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): picked mon.noname-a con 0x7ff7c40a4150 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41c4970 con 0x7ff7c40a4150 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41b1570 con 0x7ff7c40a2e90 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c409dc20 con 0x7ff7c40a32d0 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7cc22e700 10 monclient(hunting): authenticate will time out at 1759.394514s 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 0x7ff7c41d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 0x7ff7c41d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39492/0 (socket says 172.21.15.47:39492) 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 -- 172.21.15.47:0/1079720316 learned_addr learned my addr 172.21.15.47:0/1079720316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): get_auth_request con 0x7ff7c40a2e90 auth_method 0 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7c40a4150 0x7ff7c41d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): get_auth_request con 0x7ff7c40a4150 auth_method 0 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c97c9700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c40a32d0 0x7ff7c41d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7ca7cb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c97c9700 10 monclient(hunting): get_auth_request con 0x7ff7c40a32d0 auth_method 0 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c97c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c97c9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c97c9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient(hunting): handle_auth_done global_id 23213 payload 824 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c40a32d0 msgr2=0x7ff7c41d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c40a32d0 0x7ff7c41d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7c40a4150 msgr2=0x7ff7c41d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7c40a4150 0x7ff7c41d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 monclient: found mon.noname-b 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:18:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.993+0000 7ff7c9fca700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc009a70 con 0x7ff7c40a2e90 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7c9fca700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 0x7ff7c41d06d0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9b0 tx=0x7ff7c000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 client.?.objecter ms_handle_connect 0x7ff7c40a2e90 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7c0009710 con 0x7ff7c40a2e90 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: dump: 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 5 monclient: authenticate success, global_id 23213 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c000ccc0 con 0x7ff7c40a2e90 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7c000ce30 con 0x7ff7c40a2e90 2024-04-20T02:18:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7bbfff700 4 set_mon_vals no callback set 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7ba7fc700 10 monclient: dump: 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.017 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 10 monclient: _renew_subs 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c41b1290 con 0x7ff7c40a2e90 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 10 client.23213.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 10 monclient: _renew_subs 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c4120e30 con 0x7ff7c40a2e90 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.994+0000 7ff7cc22e700 1 librados: init done 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff7c000c850 con 0x7ff7c40a2e90 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7cc22e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 0x7ff7c41c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4068310 con 0x7ff7c41c38d0 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7a803a2f0 0x7ff7a803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7ff7c004b3c0 con 0x7ff7c40a2e90 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 10 client.23213.objecter ms_dispatch 0x7ff7c4001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 3 client.23213.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 3 client.23213.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ba7fc700 20 client.23213.objecter dump_active .. 0 homeless 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 0x7ff7c41c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient: get_auth_request con 0x7ff7c41c38d0 auth_method 0 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ca7cb700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7a803a2f0 0x7ff7a803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7ca7cb700 10 monclient: get_auth_request con 0x7ff7a803a2f0 auth_method 0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.995+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_done global_id 34268 payload 824 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7c97c9700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7c4067d30 con 0x7ff7c41c38d0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 client.23213.objecter ms_handle_connect 0x7ff7a803a2f0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ca7cb700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7a803a2f0 0x7ff7a803c7b0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc014040 tx=0x7ff7bc011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 client.23213.objecter ms_handle_connect 0x7ff7c41c38d0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 client.23213.objecter resend_mon_ops 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7c97c9700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 0x7ff7c41c3cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7c41202f0 tx=0x7ff7b400e860 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b400ee80 con 0x7ff7c41c38d0 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:18:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 monclient: dump: 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff7b4005b50 con 0x7ff7c41c38d0 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 msgr2=0x7ff7c41c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 0x7ff7c41c3cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7c41202f0 tx=0x7ff7b400e860 comp rx=0 tx=0).stop 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 msgr2=0x7ff7c41c3cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:17:59.996+0000 7ff7ba7fc700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c38d0 0x7ff7c41c3cb0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7c41202f0 tx=0x7ff7b400e860 comp rx=0 tx=0).stop 2024-04-20T02:18:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.005+0000 7ff7cc22e700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:18:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.005+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 0x7ff7c406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7cc22e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c406bf10 con 0x7ff7c41c41f0 2024-04-20T02:18:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 0x7ff7c406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient: get_auth_request con 0x7ff7c41c41f0 auth_method 0 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 10 monclient(hunting): handle_auth_done global_id 34277 payload 824 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 1 -- 172.21.15.47:0/1079720316 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7ff7c4067320 con 0x7ff7c41c41f0 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7ba7fc700 10 client.23213.objecter ms_handle_connect 0x7ff7c41c41f0 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7ba7fc700 10 client.23213.objecter resend_mon_ops 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7c97c9700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 0x7ff7c406fb40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4013000 tx=0x7ff7b40010a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4007730 con 0x7ff7c41c41f0 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.006+0000 7ff7ba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:18:00.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 10 monclient: dump: 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7ff7b401c6d0 con 0x7ff7c41c41f0 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 msgr2=0x7ff7c406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 0x7ff7c406fb40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4013000 tx=0x7ff7b40010a0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 msgr2=0x7ff7c406fb40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ba7fc700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c41c41f0 0x7ff7c406fb40 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4013000 tx=0x7ff7b40010a0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:18:00.028 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:00.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 10 librados: watch_flush enter 2024-04-20T02:18:00.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 10 librados: watch_flush exit 2024-04-20T02:18:00.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 20 client.23213.objecter shutdown clearing up homeless session... 2024-04-20T02:18:00.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 10 client.23213.objecter successfully canceled tick 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7a803a2f0 msgr2=0x7ff7a803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7a803a2f0 0x7ff7a803c7b0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc014040 tx=0x7ff7bc011000 comp rx=0 tx=0).stop 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 10 monclient: shutdown 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 msgr2=0x7ff7c41d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 0x7ff7c41d06d0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9b0 tx=0x7ff7c000ecc0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7ca7cb700 1 -- 172.21.15.47:0/1079720316 reap_dead start 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 shutdown_connections 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 --2- 172.21.15.47:0/1079720316 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7c40a2e90 0x7ff7c41d06d0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 >> 172.21.15.47:0/1079720316 conn(0x7ff7c40a7570 msgr2=0x7ff7c406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 shutdown_connections 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.007+0000 7ff7cc22e700 1 -- 172.21.15.47:0/1079720316 wait complete. 2024-04-20T02:18:00.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.008+0000 7ff7cc22e700 1 librados: shutdown 2024-04-20T02:18:00.031 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:18:00.032 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-04-20T02:18:00.032 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:18:00.044 INFO:tasks.ceph.mon.h:Stopped 2024-04-20T02:18:00.044 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-04-20T02:18:00.044 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:18:00.044 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-04-20T02:18:00.045 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:18:00.097 INFO:tasks.ceph.mon.i:Stopped 2024-04-20T02:18:00.097 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-04-20T02:18:00.097 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d store 2024-04-20T02:18:00.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d sync_force --yes-i-really-mean-it 2024-04-20T02:18:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.240+0000 7fd1b3e7b700 10 monclient: get_monmap_and_config 2024-04-20T02:18:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.240+0000 7fd1b3e7b700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.240+0000 7fd1b3e7b700 10 monclient: monmap: 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:00.241821+0000 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:00.241821+0000 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.241 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.241+0000 7fd1b3e7b700 1 Processor -- start 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.241+0000 7fd1b3e7b700 1 -- start start 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.241+0000 7fd1b3e7b700 10 monclient: init 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.241+0000 7fd1b3e7b700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.241+0000 7fd1b3e7b700 10 monclient: _add_conns ranks=[9,2,10,3,1,6,0,7,4,8,5] 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.241+0000 7fd1b3e7b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1ac0a3a20 0x7fd1ac0a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): picked mon.noname-a con 0x7fd1ac0a3a20 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): picked mon.noname-b con 0x7fd1ac0a2350 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd1ac135740 0x7fd1ac135bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): picked mon.noname-f con 0x7fd1ac135740 addr v2:172.21.15.179:3300/0 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac09e230 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac0a3e00 con 0x7fd1ac135740 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac09c610 con 0x7fd1ac0a2350 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 10 monclient(hunting): authenticate will time out at 1759.643072s 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1ac0a3a20 0x7fd1ac0a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0a27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39498/0 (socket says 172.21.15.47:39498) 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd1ac0a3a20 0x7fd1ac0a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47800/0 (socket says 172.21.15.47:47800) 2024-04-20T02:18:00.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 1 -- 172.21.15.47:0/778527148 learned_addr learned my addr 172.21.15.47:0/778527148 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): get_auth_request con 0x7fd1ac0a2350 auth_method 0 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): get_auth_request con 0x7fd1ac0a3a20 auth_method 0 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b2418700 1 --2- 172.21.15.47:0/778527148 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac135740 0x7fd1ac135bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b2418700 10 monclient(hunting): get_auth_request con 0x7fd1ac135740 auth_method 0 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b2418700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b2418700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b2418700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_done global_id 34362 payload 274 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 -- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 msgr2=0x7fd1ac0a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0a27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 -- 172.21.15.47:0/778527148 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac135740 msgr2=0x7fd1ac135bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/778527148 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac135740 0x7fd1ac135bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 monclient: found mon.noname-a 2024-04-20T02:18:00.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 -- 172.21.15.47:0/778527148 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0751e0 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1416700 1 --2- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac0a1e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0a42b0 tx=0x7fd19c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625a4abf225b0557 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 1 -- 172.21.15.47:0/778527148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd19c011aa0 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 10 monclient: dump: 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.257 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.258 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b0c15700 1 -- 172.21.15.47:0/778527148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c011c00 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.242+0000 7fd1b3e7b700 5 monclient: authenticate success, global_id 34362 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b0c15700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b0c15700 1 -- 172.21.15.47:0/778527148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd19c011d70 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 4 set_mon_vals no callback set 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b0c15700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b0c15700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b0c15700 10 monclient: dump: 2024-04-20T02:18:00.259 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 10 monclient: shutdown 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 msgr2=0x7fd1ac0a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac0a1e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0a42b0 tx=0x7fd19c009b20 comp rx=0 tx=0).stop 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/778527148 shutdown_connections 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/778527148 >> v2:172.21.15.179:3300/0 conn(0x7fd1ac135740 0x7fd1ac135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac0a1e10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/778527148 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/778527148 >> 172.21.15.47:0/778527148 conn(0x7fd1ac0a2f00 msgr2=0x7fd1ac0a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/778527148 shutdown_connections 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.243+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/778527148 wait complete. 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 10 monclient: monmap: 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:00.262 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 1 librados: starting msgr at 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 1 librados: starting objecter 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 1 Processor -- start 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 1 -- start start 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 1 librados: setting wanted keys 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 1 librados: calling monclient init 2024-04-20T02:18:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.244+0000 7fd1b3e7b700 10 monclient: init 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient: _add_conns ranks=[2,0,3,8,7,6,5,4,1] 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): picked mon.noname-b con 0x7fd1ac0a2350 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac1c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): picked mon.noname-a con 0x7fd1ac0a3a20 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac135740 0x7fd1ac1c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): picked mon.noname-g con 0x7fd1ac135740 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac094c50 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0704e0 con 0x7fd1ac0a2350 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac071440 con 0x7fd1ac135740 2024-04-20T02:18:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b3e7b700 10 monclient(hunting): authenticate will time out at 1759.646378s 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39512/0 (socket says 172.21.15.47:39512) 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 1 -- 172.21.15.47:0/3355792457 learned_addr learned my addr 172.21.15.47:0/3355792457 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): get_auth_request con 0x7fd1ac0a2350 auth_method 0 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac1c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): get_auth_request con 0x7fd1ac0a3a20 auth_method 0 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac135740 0x7fd1ac1c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): get_auth_request con 0x7fd1ac135740 auth_method 0 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b2418700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient(hunting): handle_auth_done global_id 34371 payload 824 2024-04-20T02:18:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac135740 msgr2=0x7fd1ac1c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd1ac135740 0x7fd1ac1c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 msgr2=0x7fd1ac0726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 monclient: found mon.noname-a 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1416700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8009550 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.245+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd1ac0a2350 0x7fd1ac0726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b1416700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac1c7000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac06d6d0 tx=0x7fd1a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 client.?.objecter ms_handle_connect 0x7fd1ac0a3a20 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1a80036a0 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: dump: 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a8003840 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 5 monclient: authenticate success, global_id 34371 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 10 monclient: _renew_subs 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c0094a0 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1a8003e20 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a27fc700 10 monclient: dump: 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1a3fff700 4 set_mon_vals no callback set 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 10 client.34371.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 10 monclient: _renew_subs 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac071a00 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.246+0000 7fd1b3e7b700 1 librados: init done 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b3e7b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 0x7fd1ac1ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac068180 con 0x7fd1ac1c9fd0 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd1a80039b0 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 0x7fd1ac1ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient: get_auth_request con 0x7fd1ac1c9fd0 auth_method 0 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd19003a320 0x7fd19003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7fd1a804b4f0 con 0x7fd1ac0a3a20 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 10 client.34371.objecter ms_dispatch 0x7fd1ac001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 3 client.34371.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 3 client.34371.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_done global_id 22587 payload 824 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1ac09d900 con 0x7fd1ac1c9fd0 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b2418700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd19003a320 0x7fd19003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 20 client.34371.objecter dump_active .. 0 homeless 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b2418700 10 monclient: get_auth_request con 0x7fd19003a320 auth_method 0 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 10 client.34371.objecter ms_handle_connect 0x7fd1ac1c9fd0 2024-04-20T02:18:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 10 client.34371.objecter resend_mon_ops 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 0x7fd1ac1ca3b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009790 tx=0x7fd19c004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd19c020070 con 0x7fd1ac1c9fd0 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.247+0000 7fd1a27fc700 10 monclient: dump: 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 10 client.34371.objecter ms_handle_connect 0x7fd19003a320 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1b2418700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd19003a320 0x7fd19003c7e0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4003a20 tx=0x7fd1a4009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd19c006c60 con 0x7fd1ac1c9fd0 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 msgr2=0x7fd1ac1ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 0x7fd1ac1ca3b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009790 tx=0x7fd19c004750 comp rx=0 tx=0).stop 2024-04-20T02:18:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 msgr2=0x7fd1ac1ca3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.248+0000 7fd1a27fc700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1c9fd0 0x7fd1ac1ca3b0 secure :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009790 tx=0x7fd19c004750 comp rx=0 tx=0).stop 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b3e7b700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 0x7fd1ac1ba900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b3e7b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac06bfd0 con 0x7fd1ac1ca8f0 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 0x7fd1ac1ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 10 monclient: get_auth_request con 0x7fd1ac1ca8f0 auth_method 0 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.257+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1b1c17700 10 monclient(hunting): handle_auth_done global_id 22596 payload 824 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1b1c17700 1 -- 172.21.15.47:0/3355792457 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fd1ac09dbe0 con 0x7fd1ac1ca8f0 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 client.34371.objecter ms_handle_connect 0x7fd1ac1ca8f0 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 client.34371.objecter resend_mon_ops 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1b1c17700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 0x7fd1ac1ba900 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009b60 tx=0x7fd19c011000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd19c01bab0 con 0x7fd1ac1ca8f0 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 monclient: dump: 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fd19c01bc10 con 0x7fd1ac1ca8f0 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 msgr2=0x7fd1ac1ba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 0x7fd1ac1ba900 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009b60 tx=0x7fd19c011000 comp rx=0 tx=0).stop 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 msgr2=0x7fd1ac1ba900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.258+0000 7fd1a27fc700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd1ac1ca8f0 0x7fd1ac1ba900 secure :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009b60 tx=0x7fd19c011000 comp rx=0 tx=0).stop 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:18:00.282 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:00.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 10 librados: watch_flush enter 2024-04-20T02:18:00.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 10 librados: watch_flush exit 2024-04-20T02:18:00.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 20 client.34371.objecter shutdown clearing up homeless session... 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 10 client.34371.objecter successfully canceled tick 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd19003a320 msgr2=0x7fd19003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd19003a320 0x7fd19003c7e0 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4003a20 tx=0x7fd1a4009780 comp rx=0 tx=0).stop 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 10 monclient: shutdown 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 msgr2=0x7fd1ac1c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 --2- 172.21.15.47:0/3355792457 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1ac0a3a20 0x7fd1ac1c7000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac06d6d0 tx=0x7fd1a800ec80 comp rx=0 tx=0).stop 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b2418700 1 -- 172.21.15.47:0/3355792457 reap_dead start 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b2418700 1 -- 172.21.15.47:0/3355792457 reap_dead start 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 shutdown_connections 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 >> 172.21.15.47:0/3355792457 conn(0x7fd1ac0a2f00 msgr2=0x7fd1ac098a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 shutdown_connections 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 -- 172.21.15.47:0/3355792457 wait complete. 2024-04-20T02:18:00.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.259+0000 7fd1b3e7b700 1 librados: shutdown 2024-04-20T02:18:00.286 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-04-20T02:18:00.286 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-04-20T02:18:00.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:18:00.298 INFO:tasks.ceph.mon.d:Stopped 2024-04-20T02:18:00.299 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-20T02:18:00.299 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c store 2024-04-20T02:18:00.299 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c sync_force --yes-i-really-mean-it 2024-04-20T02:18:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.479+0000 7fc5042a2700 10 monclient: get_monmap_and_config 2024-04-20T02:18:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.479+0000 7fc5042a2700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.479+0000 7fc5042a2700 10 monclient: monmap: 2024-04-20T02:18:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.479 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:00.481061+0000 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:00.481061+0000 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.480 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:00.481 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:00.481 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.480+0000 7fc5042a2700 1 Processor -- start 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.480+0000 7fc5042a2700 1 -- start start 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.480+0000 7fc5042a2700 10 monclient: init 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient: _add_conns ranks=[9,1,6,10,7,2,8,4,5,3,0] 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4fc0a0be0 0x7fc4fc0a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): picked mon.noname-a con 0x7fc4fc0a0be0 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc4fc0a2950 addr v1:172.21.15.179:6789/0 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): picked mon.noname-d con 0x7fc4fc0a34e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc0751e0 con 0x7fc4fc0a2950 2024-04-20T02:18:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc09dfe0 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc067860 con 0x7fc4fc0a34e0 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc5042a2700 10 monclient(hunting): authenticate will time out at 1759.882358s 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50283f700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc4fc0a34e0 msgr2=0x7fc4fc13fb50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50283f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc13fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4fc0a0be0 0x7fc4fc0a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc4fc0a0be0 0x7fc4fc0a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47832/0 (socket says 172.21.15.47:47832) 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 -- 172.21.15.47:0/3498491285 learned_addr learned my addr 172.21.15.47:0/3498491285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): get_auth_request con 0x7fc4fc0a0be0 auth_method 0 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient(hunting): handle_auth_done global_id 34380 payload 274 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 -- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc4fc0a34e0 msgr2=0x7fc4fc13fb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 --2- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc13fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 -- 172.21.15.47:0/3498491285 >> v1:172.21.15.179:6789/0 conn(0x7fc4fc0a2950 legacy=0x7fc4fc0a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 monclient: found mon.noname-a 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 -- 172.21.15.47:0/3498491285 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc067d20 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.481+0000 7fc50203e700 1 --2- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4fc0a0be0 0x7fc4fc0a0fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80098f0 tx=0x7fc4f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0075d82071f189 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 1 -- 172.21.15.47:0/3498491285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f800bb10 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: dump: 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.501 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 1 -- 172.21.15.47:0/3498491285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800bc70 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 5 monclient: authenticate success, global_id 34380 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 1 -- 172.21.15.47:0/3498491285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f800bde0 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 4 set_mon_vals no callback set 2024-04-20T02:18:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc50103c700 10 monclient: dump: 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.503 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 10 monclient: shutdown 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 -- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4fc0a0be0 msgr2=0x7fc4fc0a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 --2- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4fc0a0be0 0x7fc4fc0a0fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80098f0 tx=0x7fc4f8009c00 comp rx=0 tx=0).stop 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 -- 172.21.15.47:0/3498491285 shutdown_connections 2024-04-20T02:18:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 --2- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc13fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 --2- 172.21.15.47:0/3498491285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4fc0a0be0 0x7fc4fc0a0fc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 -- 172.21.15.47:0/3498491285 >> 172.21.15.47:0/3498491285 conn(0x7fc4fc0a7570 msgr2=0x7fc4fc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 -- 172.21.15.47:0/3498491285 shutdown_connections 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.482+0000 7fc5042a2700 1 -- 172.21.15.47:0/3498491285 wait complete. 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 10 monclient: monmap: 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 1 librados: starting msgr at 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 1 librados: starting objecter 2024-04-20T02:18:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 1 Processor -- start 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 1 -- start start 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 1 librados: setting wanted keys 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 1 librados: calling monclient init 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.483+0000 7fc5042a2700 10 monclient: init 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient: _add_conns ranks=[8,1,7,6,5,0,3,2,4] 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc4fc0a0be0 0x7fc4fc1cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): picked mon.noname-e con 0x7fc4fc0a0be0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4fc0a2950 0x7fc4fc1bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): picked mon.noname-f con 0x7fc4fc0a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc1bd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): picked mon.noname-i con 0x7fc4fc0a34e0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1aa100 con 0x7fc4fc0a2950 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc09e4b0 con 0x7fc4fc0a34e0 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1a82f0 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc5042a2700 10 monclient(hunting): authenticate will time out at 1759.885384s 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50283f700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc4fc0a34e0 msgr2=0x7fc4fc1bd220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc4fc0a0be0 0x7fc4fc1cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50283f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc1bd220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc4fc0a0be0 0x7fc4fc1cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39926/0 (socket says 172.21.15.47:39926) 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 -- 172.21.15.47:0/2860716332 learned_addr learned my addr 172.21.15.47:0/2860716332 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): get_auth_request con 0x7fc4fc0a0be0 auth_method 0 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4fc0a2950 0x7fc4fc1bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): get_auth_request con 0x7fc4fc0a2950 auth_method 0 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient(hunting): handle_auth_done global_id 34289 payload 824 2024-04-20T02:18:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc4fc0a34e0 msgr2=0x7fc4fc1bd220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc4fc0a34e0 0x7fc4fc1bd220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4fc0a2950 msgr2=0x7fc4fc1bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4fc0a2950 0x7fc4fc1bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 monclient: found mon.noname-e 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.484+0000 7fc50203e700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8009550 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc50203e700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc4fc0a0be0 0x7fc4fc1cc420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b0e0 tx=0x7fc4f8004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 client.?.objecter ms_handle_connect 0x7fc4fc0a0be0 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f8004350 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: dump: 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 5 monclient: authenticate success, global_id 34289 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f80044d0 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 10 monclient: _renew_subs 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f8024470 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc072210 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc50083b700 4 set_mon_vals no callback set 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc4f2ffd700 10 monclient: dump: 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 10 client.34289.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 10 monclient: _renew_subs 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc1cce80 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.485+0000 7fc5042a2700 1 librados: init done 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc5042a2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc5042a2700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 0x7fc4fc02bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1cd010 con 0x7fc4fc1c96f0 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc4f8021070 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 0x7fc4fc02bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient: get_auth_request con 0x7fc4fc1c96f0 auth_method 0 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4e003a2f0 0x7fc4e003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7fc4f804c670 con 0x7fc4fc0a0be0 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 10 client.34289.objecter ms_dispatch 0x7fc4fc001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 10 monclient(hunting): handle_auth_done global_id 23107 payload 824 2024-04-20T02:18:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 3 client.34289.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 3 client.34289.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4fc09f2e0 con 0x7fc4fc1c96f0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50283f700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4e003a2f0 0x7fc4e003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50283f700 10 monclient: get_auth_request con 0x7fc4e003a2f0 auth_method 0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc50183d700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 0x7fc4fc02bbc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc1a7ef0 tx=0x7fc4ec00fb00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 20 client.34289.objecter dump_active .. 0 homeless 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 10 client.34289.objecter ms_handle_connect 0x7fc4fc1c96f0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 10 client.34289.objecter resend_mon_ops 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4ec006cd0 con 0x7fc4fc1c96f0 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.486+0000 7fc4f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 10 monclient: dump: 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc50283f700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4e003a2f0 0x7fc4e003c7b0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4003d40 tx=0x7fc4f40091f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 10 client.34289.objecter ms_handle_connect 0x7fc4e003a2f0 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc4ec006e30 con 0x7fc4fc1c96f0 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 msgr2=0x7fc4fc02bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 0x7fc4fc02bbc0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc1a7ef0 tx=0x7fc4ec00fb00 comp rx=0 tx=0).stop 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 msgr2=0x7fc4fc02bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.487+0000 7fc4f2ffd700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c96f0 0x7fc4fc02bbc0 secure :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc1a7ef0 tx=0x7fc4ec00fb00 comp rx=0 tx=0).stop 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc5042a2700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc5042a2700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 0x7fc4fc02c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc5042a2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc068430 con 0x7fc4fc1c9ad0 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 0x7fc4fc02c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient: get_auth_request con 0x7fc4fc1c9ad0 auth_method 0 2024-04-20T02:18:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 10 monclient(hunting): handle_auth_done global_id 23116 payload 824 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 1 -- 172.21.15.47:0/2860716332 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fc4fc09f5c0 con 0x7fc4fc1c9ad0 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc4f2ffd700 10 client.34289.objecter ms_handle_connect 0x7fc4fc1c9ad0 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc4f2ffd700 10 client.34289.objecter resend_mon_ops 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.496+0000 7fc50183d700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 0x7fc4fc02c100 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec016000 tx=0x7fc4ec000ed0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4ec022070 con 0x7fc4fc1c9ad0 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 10 monclient: dump: 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.521 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fc4ec0076f0 con 0x7fc4fc1c9ad0 2024-04-20T02:18:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 msgr2=0x7fc4fc02c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 0x7fc4fc02c100 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec016000 tx=0x7fc4ec000ed0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 msgr2=0x7fc4fc02c100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc4f2ffd700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4fc1c9ad0 0x7fc4fc02c100 secure :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec016000 tx=0x7fc4ec000ed0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:18:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:18:00.523 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 10 librados: watch_flush enter 2024-04-20T02:18:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 10 librados: watch_flush exit 2024-04-20T02:18:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 20 client.34289.objecter shutdown clearing up homeless session... 2024-04-20T02:18:00.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 10 client.34289.objecter successfully canceled tick 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4e003a2f0 msgr2=0x7fc4e003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc4e003a2f0 0x7fc4e003c7b0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4003d40 tx=0x7fc4f40091f0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 10 monclient: shutdown 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc4fc0a0be0 msgr2=0x7fc4fc1cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 1 --2- 172.21.15.47:0/2860716332 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc4fc0a0be0 0x7fc4fc1cc420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b0e0 tx=0x7fc4f8004750 comp rx=0 tx=0).stop 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc50283f700 1 -- 172.21.15.47:0/2860716332 reap_dead start 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc50283f700 1 -- 172.21.15.47:0/2860716332 reap_dead start 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 shutdown_connections 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.497+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 >> 172.21.15.47:0/2860716332 conn(0x7fc4fc0a7570 msgr2=0x7fc4fc0a59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.498+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 shutdown_connections 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.498+0000 7fc5042a2700 1 -- 172.21.15.47:0/2860716332 wait complete. 2024-04-20T02:18:00.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.498+0000 7fc5042a2700 1 librados: shutdown 2024-04-20T02:18:00.525 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:18:00.526 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T02:18:00.526 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:18:00.538 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T02:18:00.538 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-04-20T02:18:00.538 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.716+0000 7f245398b700 10 monclient: get_monmap_and_config 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.716+0000 7f245398b700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.716+0000 7f245398b700 10 monclient: monmap: 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:00.717699+0000 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:00.717699+0000 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:00.716 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:00.717 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 Processor -- start 2024-04-20T02:18:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 -- start start 2024-04-20T02:18:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient: init 2024-04-20T02:18:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient: _add_conns ranks=[7,9,3,8,6,5,10,4,0,1,2] 2024-04-20T02:18:00.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f244c09d2f0 0x7f244c135d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): picked mon.noname-i con 0x7f244c09d2f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244c09d6d0 0x7f244c0a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): picked mon.noname-a con 0x7f244c09d6d0 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c0a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): picked mon.noname-g con 0x7f244c09db30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c0a0c80 con 0x7f244c09d6d0 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0a0a00 con 0x7f244c09db30 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0a0e80 con 0x7f244c09d2f0 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.717+0000 7f245398b700 10 monclient(hunting): authenticate will time out at 1760.118951s 2024-04-20T02:18:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244c09d6d0 0x7f244c0a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451727700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f244c09d2f0 msgr2=0x7f244c135d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451727700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f244c09d2f0 0x7f244c135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f244c09d6d0 0x7f244c0a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47860/0 (socket says 172.21.15.47:47860) 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 -- 172.21.15.47:0/3514760523 learned_addr learned my addr 172.21.15.47:0/3514760523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): get_auth_request con 0x7f244c09d6d0 auth_method 0 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c0a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): get_auth_request con 0x7f244c09db30 auth_method 0 2024-04-20T02:18:00.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2451f28700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient(hunting): handle_auth_done global_id 34389 payload 274 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 -- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f244c09d2f0 msgr2=0x7f244c135d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f244c09d2f0 0x7f244c135d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 -- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 msgr2=0x7f244c0a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c0a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 monclient: found mon.noname-a 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 -- 172.21.15.47:0/3514760523 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0751e0 con 0x7f244c09d6d0 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2450f26700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d6d0 0x7f244c0a2800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f243c0098f0 tx=0x7f243c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba938a7eee87d248 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 1 -- 172.21.15.47:0/3514760523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f243c011aa0 con 0x7f244c09d6d0 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: dump: 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 1 -- 172.21.15.47:0/3514760523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c011c00 con 0x7f244c09d6d0 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f245398b700 5 monclient: authenticate success, global_id 34389 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f245398b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f245398b700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 1 -- 172.21.15.47:0/3514760523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f243c011d70 con 0x7f244c09d6d0 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f245398b700 4 set_mon_vals no callback set 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.718+0000 7f2443fff700 10 monclient: dump: 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.734 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 10 monclient: shutdown 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 -- 172.21.15.47:0/3514760523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d6d0 msgr2=0x7f244c0a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d6d0 0x7f244c0a2800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f243c0098f0 tx=0x7f243c009c00 comp rx=0 tx=0).stop 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 -- 172.21.15.47:0/3514760523 shutdown_connections 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c0a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f244c09d2f0 0x7f244c135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 --2- 172.21.15.47:0/3514760523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d6d0 0x7f244c0a2800 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 -- 172.21.15.47:0/3514760523 >> 172.21.15.47:0/3514760523 conn(0x7f244c0a7420 msgr2=0x7f244c0a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 -- 172.21.15.47:0/3514760523 shutdown_connections 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.719+0000 7f245398b700 1 -- 172.21.15.47:0/3514760523 wait complete. 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.720+0000 7f245398b700 10 monclient: build_initial_monmap 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.720+0000 7f245398b700 10 monclient: monmap: 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.720+0000 7f245398b700 1 librados: starting msgr at 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.720+0000 7f245398b700 1 librados: starting objecter 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.720+0000 7f245398b700 1 Processor -- start 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 -- start start 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 librados: setting wanted keys 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 librados: calling monclient init 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient: init 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:00.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient: _add_conns ranks=[0,2,3,1,7,8,4,5,6] 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 0x7f244c1ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): picked mon.noname-a con 0x7f244c09d2f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f244c09d6d0 0x7f244c133f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): picked mon.noname-b con 0x7f244c09d6d0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c134480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): picked mon.noname-g con 0x7f244c09db30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c1caeb0 con 0x7f244c09d2f0 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c067150 con 0x7f244c09d6d0 2024-04-20T02:18:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0726d0 con 0x7f244c09db30 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f245398b700 10 monclient(hunting): authenticate will time out at 1760.122815s 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 0x7f244c1ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 0x7f244c1ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47864/0 (socket says 172.21.15.47:47864) 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 1 -- 172.21.15.47:0/708913005 learned_addr learned my addr 172.21.15.47:0/708913005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 10 monclient(hunting): get_auth_request con 0x7f244c09d2f0 auth_method 0 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2451727700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2450f26700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f244c09d6d0 0x7f244c133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2450f26700 10 monclient(hunting): get_auth_request con 0x7f244c09d6d0 auth_method 0 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2450f26700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2450f26700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.721+0000 7f2450f26700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2450f26700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2450f26700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2450f26700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451f28700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451f28700 10 monclient(hunting): get_auth_request con 0x7f244c09db30 auth_method 0 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451f28700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451f28700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451f28700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 10 monclient(hunting): handle_auth_done global_id 34398 payload 824 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 -- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 msgr2=0x7f244c134480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c134480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 -- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f244c09d6d0 msgr2=0x7f244c133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f244c09d6d0 0x7f244c133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 10 monclient: _finish_hunting 0 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 monclient: found mon.noname-a 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 -- 172.21.15.47:0/708913005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c009550 con 0x7f244c09d2f0 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2450f26700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f244c09d6d0 0x7f244c133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2451727700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 0x7f244c1ca1e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f244800b780 tx=0x7f244800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 client.?.objecter ms_handle_connect 0x7f244c09d2f0 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2448003c50 con 0x7f244c09d2f0 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: dump: 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.742 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: _finish_auth 0 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f245398b700 5 monclient: authenticate success, global_id 34398 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2448003dd0 con 0x7f244c09d2f0 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24480103a0 con 0x7f244c09d2f0 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f24437fe700 4 set_mon_vals no callback set 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f2441ffb700 10 monclient: dump: 2024-04-20T02:18:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f245398b700 10 monclient: _renew_subs 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f245398b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.722+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c1216f0 con 0x7f244c09d2f0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f245398b700 10 client.34398.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f245398b700 10 monclient: _renew_subs 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f245398b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c11f390 con 0x7f244c09d2f0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f245398b700 1 librados: init done 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2448005070 con 0x7f244c09d2f0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f2441ffb700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f243003a2f0 0x7f243003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7f244804a100 con 0x7f244c09d2f0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f2441ffb700 10 client.34398.objecter ms_dispatch 0x7f244c001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f2441ffb700 3 client.34398.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.723+0000 7f2441ffb700 3 client.34398.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f245398b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f245398b700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f245398b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c121090 con 0x7f244c09d2f0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f2441ffb700 20 client.34398.objecter dump_active .. 0 homeless 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f2450f26700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f243003a2f0 0x7f243003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f2450f26700 10 monclient: get_auth_request con 0x7f243003a2f0 auth_method 0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f2450f26700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f243003a2f0 0x7f243003c7b0 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f243c0098f0 tx=0x7f243c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.724+0000 7f2441ffb700 10 client.34398.objecter ms_handle_connect 0x7f243003a2f0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.729+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2448016020 con 0x7f244c09d2f0 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.729+0000 7f2441ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:00.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:00.729+0000 7f2441ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:01.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.038+0000 7f245398b700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:18:01.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.038+0000 7f245398b700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:18:01.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.038+0000 7f245398b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:01.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.038+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f244c1212c0 con 0x7f244c09d2f0 2024-04-20T02:18:01.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.039+0000 7f2441ffb700 1 -- 172.21.15.47:0/708913005 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f244800d640 con 0x7f244c09d2f0 2024-04-20T02:18:01.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.039+0000 7f2441ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:18:01.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.039+0000 7f2441ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:01.039 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":18,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:18:01.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 10 librados: watch_flush enter 2024-04-20T02:18:01.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 10 librados: watch_flush exit 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 20 client.34398.objecter shutdown clearing up homeless session... 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 10 client.34398.objecter successfully canceled tick 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f243003a2f0 msgr2=0x7f243003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f243003a2f0 0x7f243003c7b0 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f243c0098f0 tx=0x7f243c008030 comp rx=0 tx=0).stop 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 10 monclient: shutdown 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 msgr2=0x7f244c1ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 0x7f244c1ca1e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f244800b780 tx=0x7f244800ba90 comp rx=0 tx=0).stop 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 shutdown_connections 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f243003a2f0 0x7f243003c7b0 unknown :-1 s=CLOSED pgs=3057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f244c09d2f0 0x7f244c1ca1e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f244c09db30 0x7f244c134480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 --2- 172.21.15.47:0/708913005 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f244c09d6d0 0x7f244c133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:01.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 >> 172.21.15.47:0/708913005 conn(0x7f244c0a7420 msgr2=0x7f244c1bd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:01.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 shutdown_connections 2024-04-20T02:18:01.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 -- 172.21.15.47:0/708913005 wait complete. 2024-04-20T02:18:01.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:01.041+0000 7f245398b700 1 librados: shutdown 2024-04-20T02:18:04.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:18:04.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.201+0000 7f98c11aa700 10 monclient: get_monmap_and_config 2024-04-20T02:18:04.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.201+0000 7f98c11aa700 10 monclient: build_initial_monmap 2024-04-20T02:18:04.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.201+0000 7f98c11aa700 10 monclient: monmap: 2024-04-20T02:18:04.201 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:04.202336+0000 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:04.202336+0000 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:04.202 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.201+0000 7f98c11aa700 1 Processor -- start 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.201+0000 7f98c11aa700 1 -- start start 2024-04-20T02:18:04.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.201+0000 7f98c11aa700 10 monclient: init 2024-04-20T02:18:04.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:04.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient: _add_conns ranks=[4,5,8,0,10,9,2,3,7,6,1] 2024-04-20T02:18:04.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f98bc0a0ad0 0x7f98bc0a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): picked mon.noname-c con 0x7f98bc0a0ad0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a2800 0x7f98bc0a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): picked mon.noname-h con 0x7f98bc0a2800 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 0x7f98bc13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): picked mon.noname-e con 0x7f98bc0a31c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc09ee40 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0a3620 con 0x7f98bc0a2800 2024-04-20T02:18:04.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc09ddb0 con 0x7f98bc0a31c0 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98c11aa700 10 monclient(hunting): authenticate will time out at 1763.603562s 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bad9d700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f98bc0a0ad0 msgr2=0x7f98bc0a0eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bad9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f98bc0a0ad0 0x7f98bc0a0eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 0x7f98bc13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98ba59c700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a2800 msgr2=0x7f98bc0a2c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98ba59c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a2800 0x7f98bc0a2c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 0x7f98bc13e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:46282/0 (socket says 172.21.15.47:46282) 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 1 -- 172.21.15.47:0/3796443280 learned_addr learned my addr 172.21.15.47:0/3796443280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:04.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): get_auth_request con 0x7f98bc0a31c0 auth_method 0 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.202+0000 7f98bb59e700 10 monclient(hunting): handle_auth_done global_id 34298 payload 274 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 -- 172.21.15.47:0/3796443280 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a2800 msgr2=0x7f98bc0a2c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a2800 0x7f98bc0a2c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 -- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f98bc0a0ad0 msgr2=0x7f98bc0a0eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f98bc0a0ad0 0x7f98bc0a0eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 10 monclient: _finish_hunting 0 2024-04-20T02:18:04.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 monclient: found mon.noname-e 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 -- 172.21.15.47:0/3796443280 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc0751e0 con 0x7f98bc0a31c0 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98bb59e700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 0x7f98bc13e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009e80 tx=0x7f98b400b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a8b0bf4534fa532b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 1 -- 172.21.15.47:0/3796443280 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f98b400e070 con 0x7f98bc0a31c0 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: dump: 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:04.217 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:04.218 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:04.220 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: _finish_auth 0 2024-04-20T02:18:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:18:04.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 1 -- 172.21.15.47:0/3796443280 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b400be30 con 0x7f98bc0a31c0 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98c11aa700 5 monclient: authenticate success, global_id 34298 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98c11aa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98c11aa700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 1 -- 172.21.15.47:0/3796443280 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f98b40123f0 con 0x7f98bc0a31c0 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98c11aa700 4 set_mon_vals no callback set 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.203+0000 7f98b9d9b700 10 monclient: dump: 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:04.221 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 10 monclient: shutdown 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 -- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 msgr2=0x7f98bc13e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:04.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 0x7f98bc13e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009e80 tx=0x7f98b400b330 comp rx=0 tx=0).stop 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 -- 172.21.15.47:0/3796443280 shutdown_connections 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f98bc0a31c0 0x7f98bc13e9f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f98bc0a0ad0 0x7f98bc0a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3796443280 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a2800 0x7f98bc0a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 -- 172.21.15.47:0/3796443280 >> 172.21.15.47:0/3796443280 conn(0x7f98bc0a7420 msgr2=0x7f98bc0a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 -- 172.21.15.47:0/3796443280 shutdown_connections 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.204+0000 7f98c11aa700 1 -- 172.21.15.47:0/3796443280 wait complete. 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 10 monclient: build_initial_monmap 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 10 monclient: monmap: 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:04.223 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 1 librados: starting msgr at 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 1 librados: starting objecter 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 1 Processor -- start 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 1 -- start start 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 1 librados: setting wanted keys 2024-04-20T02:18:04.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 1 librados: calling monclient init 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.205+0000 7f98c11aa700 10 monclient: init 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient: _add_conns ranks=[2,1,5,8,0,7,3,4,6] 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 0x7f98bc1cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): picked mon.noname-b con 0x7f98bc0a0ad0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98bc0a2800 0x7f98bc1bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): picked mon.noname-f con 0x7f98bc0a2800 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a31c0 0x7f98bc1bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): picked mon.noname-h con 0x7f98bc0a31c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc1a9f90 con 0x7f98bc0a2800 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc072130 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc1a8180 con 0x7f98bc0a31c0 2024-04-20T02:18:04.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98c11aa700 10 monclient(hunting): authenticate will time out at 1763.607291s 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 0x7f98bc1cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bb59e700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a31c0 msgr2=0x7f98bc1bd0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bb59e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a31c0 0x7f98bc1bd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 0x7f98bc1cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:51340/0 (socket says 172.21.15.47:51340) 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 -- 172.21.15.47:0/3104163814 learned_addr learned my addr 172.21.15.47:0/3104163814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): get_auth_request con 0x7f98bc0a0ad0 auth_method 0 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98bc0a2800 0x7f98bc1bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:04.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): get_auth_request con 0x7f98bc0a2800 auth_method 0 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98ba59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient(hunting): handle_auth_done global_id 23249 payload 824 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 -- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a31c0 msgr2=0x7f98bc1bd0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a31c0 0x7f98bc1bd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 -- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98bc0a2800 msgr2=0x7f98bc1bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98bc0a2800 0x7f98bc1bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:18:04.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 monclient: found mon.noname-b 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 -- 172.21.15.47:0/3104163814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b4009b30 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98bad9d700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 0x7f98bc1cc2f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b550 tx=0x7f98ac00b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.206+0000 7f98a37fe700 10 client.?.objecter ms_handle_connect 0x7f98bc0a0ad0 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f98ac003c50 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: dump: 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:04.228 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: _finish_auth 0 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac003df0 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 5 monclient: authenticate success, global_id 23249 2024-04-20T02:18:04.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 10 monclient: _renew_subs 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc09e510 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f98ac0103a0 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 10 client.23249.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98a37fe700 10 monclient: dump: 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:04.230 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98b959a700 4 set_mon_vals no callback set 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 10 monclient: _renew_subs 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc1ccd70 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.207+0000 7f98c11aa700 1 librados: init done 2024-04-20T02:18:04.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f98ac005070 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f98a803a2f0 0x7f98a803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8723+0+0 (secure 0 0 0) 0x7f98ac04a100 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 10 client.23249.objecter ms_dispatch 0x7f98bc001550 osd_map(89..89 src has 1..89) v4 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 3 client.23249.objecter handle_osd_map got epochs [89,89] > 0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 3 client.23249.objecter handle_osd_map decoding full epoch 89 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98a37fe700 20 client.23249.objecter dump_active .. 0 homeless 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98c11aa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98ba59c700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f98a803a2f0 0x7f98a803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98c11aa700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98c11aa700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc09ffc0 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.208+0000 7f98ba59c700 10 monclient: get_auth_request con 0x7f98a803a2f0 auth_method 0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.209+0000 7f98ba59c700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f98a803a2f0 0x7f98a803c7b0 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009420 tx=0x7f98b00093b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.209+0000 7f98a37fe700 10 client.23249.objecter ms_handle_connect 0x7f98a803a2f0 2024-04-20T02:18:04.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.214+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f98ac016020 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.214+0000 7f98a37fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:04.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.214+0000 7f98a37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98c11aa700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98c11aa700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98c11aa700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f98bc1aa1d0 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98a37fe700 1 -- 172.21.15.47:0/3104163814 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f98ac00d640 con 0x7f98bc0a0ad0 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98a37fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.524+0000 7f98a37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:04.524 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":18,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 10 librados: watch_flush enter 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 10 librados: watch_flush exit 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 20 client.23249.objecter shutdown clearing up homeless session... 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 10 client.23249.objecter successfully canceled tick 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f98a803a2f0 msgr2=0x7f98a803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f98a803a2f0 0x7f98a803c7b0 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009420 tx=0x7f98b00093b0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 10 monclient: shutdown 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 msgr2=0x7f98bc1cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.526+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 0x7f98bc1cc2f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00b550 tx=0x7f98ac00b860 comp rx=0 tx=0).stop 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 shutdown_connections 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f98bc0a31c0 0x7f98bc1bd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f98bc0a0ad0 0x7f98bc1cc2f0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f98a803a2f0 0x7f98a803c7b0 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 --2- 172.21.15.47:0/3104163814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98bc0a2800 0x7f98bc1bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 >> 172.21.15.47:0/3104163814 conn(0x7f98bc0a7420 msgr2=0x7f98bc0a5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 shutdown_connections 2024-04-20T02:18:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 -- 172.21.15.47:0/3104163814 wait complete. 2024-04-20T02:18:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:04.527+0000 7f98c11aa700 1 librados: shutdown 2024-04-20T02:18:07.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:18:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.684+0000 7fd447804700 10 monclient: get_monmap_and_config 2024-04-20T02:18:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.684+0000 7fd447804700 10 monclient: build_initial_monmap 2024-04-20T02:18:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.684+0000 7fd447804700 10 monclient: monmap: 2024-04-20T02:18:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:07.685909+0000 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:07.685909+0000 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:07.685 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:07.686 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:07.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.685+0000 7fd447804700 1 Processor -- start 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.685+0000 7fd447804700 1 -- start start 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.685+0000 7fd447804700 10 monclient: init 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient: _add_conns ranks=[4,0,7,10,5,1,6,8,9,3,2] 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 0x7fd4400a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): picked mon.noname-c con 0x7fd4400a5080 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:18:07.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd4400a1a80 addr v1:172.21.15.47:6789/0 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 0x7fd44013da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): picked mon.noname-i con 0x7fd4400a3f90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4400751e0 con 0x7fd4400a1a80 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400a5460 con 0x7fd4400a5080 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd440070b10 con 0x7fd4400a3f90 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd447804700 10 monclient(hunting): authenticate will time out at 1767.087205s 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd4455a0700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 msgr2=0x7fd4400a1540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd4455a0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 0x7fd4400a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd444d9f700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd4400a1a80 0x7fd4400a3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54874/0 (socket says 172.21.15.47:54874) 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 learned_addr learned my addr 172.21.15.47:0/2266215208 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:18:07.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 msgr2=0x7fd44013da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:07.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.686+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 0x7fd44013da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.887+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 msgr2=0x7fd4400a1540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.887+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 0x7fd4400a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.887+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 msgr2=0x7fd44013da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:07.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:07.887+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 0x7fd44013da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:08.288+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 msgr2=0x7fd4400a1540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:08.288+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 0x7fd4400a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:08.288+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 msgr2=0x7fd44013da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:08.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:08.288+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 0x7fd44013da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:09.089+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 msgr2=0x7fd44013da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:09.089+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 msgr2=0x7fd4400a1540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:09.089+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 0x7fd44013da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:09.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:09.089+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 0x7fd4400a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): tick 2024-04-20T02:18:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:18:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:18:10.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 msgr2=0x7fd44013da50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd4400a3f90 0x7fd44013da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 msgr2=0x7fd4400a1540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd4400a5080 0x7fd4400a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 >> v1:172.21.15.47:6789/0 conn(0x7fd4400a1a80 legacy=0x7fd4400a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient: _add_conns ranks=[9,3,10,1,6,5,2,8] 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): picked mon.noname-a con 0x7fd4240038d0 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): picked mon.noname-g con 0x7fd424006320 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): picked mon.noname-f con 0x7fd424008e70 addr v2:172.21.15.179:3300/0 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4400a5460 con 0x7fd4240038d0 2024-04-20T02:18:10.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd440070b10 con 0x7fd424008e70 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42400ba80 con 0x7fd424006320 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd436ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd4240038d0 auth_method 0 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:10.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 10 monclient(hunting): get_auth_request con 0x7fd424008e70 auth_method 0 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 10 monclient(hunting): get_auth_request con 0x7fd424006320 auth_method 0 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:10.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:18:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.686+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:10.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.687+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd4240038d0 auth_method 0 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:10.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 10 monclient(hunting): get_auth_request con 0x7fd424008e70 auth_method 0 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.887+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 10 monclient(hunting): get_auth_request con 0x7fd424006320 auth_method 0 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:10.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:10.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:10.888+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:11.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd4240038d0 auth_method 0 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:11.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 10 monclient(hunting): get_auth_request con 0x7fd424008e70 auth_method 0 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:11.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.288+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 10 monclient(hunting): get_auth_request con 0x7fd424006320 auth_method 0 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:11.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:11.289+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd4240038d0 auth_method 0 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:12.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 10 monclient(hunting): get_auth_request con 0x7fd424008e70 auth_method 0 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 10 monclient(hunting): get_auth_request con 0x7fd424006320 auth_method 0 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:12.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:12.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:12.090+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): tick 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 msgr2=0x7fd4240087b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd424006320 0x7fd4240087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 msgr2=0x7fd42400b330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.179:3300/0 conn(0x7fd424008e70 0x7fd42400b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:13.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 msgr2=0x7fd424005d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> v2:172.21.15.47:3300/0 conn(0x7fd4240038d0 0x7fd424005d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient: _add_conns ranks=[8,6,5,2,1] 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 reap_dead start 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): picked mon.noname-e con 0x7fd42400bfc0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 reap_dead start 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 0x7fd4240067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): picked mon.noname-d con 0x7fd424003d10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 0x7fd42400b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): picked mon.noname-h con 0x7fd424006ce0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400a5460 con 0x7fd424006ce0 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd440070b10 con 0x7fd424003d10 2024-04-20T02:18:13.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 1 -- 172.21.15.47:0/2266215208 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd42400ba80 con 0x7fd42400bfc0 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd436ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 msgr2=0x7fd4240067a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 0x7fd4240067a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 msgr2=0x7fd42400b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 0x7fd42400b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd42400bfc0 auth_method 0 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 msgr2=0x7fd42400e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 msgr2=0x7fd42400e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:13.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.687+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:13.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 msgr2=0x7fd4240067a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 0x7fd4240067a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 msgr2=0x7fd42400b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 0x7fd42400b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd42400bfc0 auth_method 0 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 msgr2=0x7fd42400e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 msgr2=0x7fd42400e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:13.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:13.888+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:14.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd444d9f700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 msgr2=0x7fd4240067a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:14.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 0x7fd4240067a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:14.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd445da1700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 msgr2=0x7fd42400b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd445da1700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 0x7fd42400b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd42400bfc0 auth_method 0 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient(hunting): handle_auth_done global_id 34307 payload 274 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 msgr2=0x7fd4240067a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 0x7fd4240067a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 msgr2=0x7fd42400b260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 0x7fd42400b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient: _finish_hunting 0 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 monclient: found mon.noname-e 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 -- 172.21.15.47:0/2266215208 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440136370 con 0x7fd42400bfc0 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009e10 tx=0x7fd43c005040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=72c84a732c001c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 1 -- 172.21.15.47:0/2266215208 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd43c01a8c0 con 0x7fd42400bfc0 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: dump: 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.293 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: _finish_auth 0 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: _check_auth_tickets 2024-04-20T02:18:14.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 1 -- 172.21.15.47:0/2266215208 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c011040 con 0x7fd42400bfc0 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 5 monclient: authenticate success, global_id 34307 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 1 -- 172.21.15.47:0/2266215208 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd43c0050b0 con 0x7fd42400bfc0 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 4 set_mon_vals no callback set 2024-04-20T02:18:14.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd437fff700 10 monclient: dump: 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.296 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 10 monclient: shutdown 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 1 -- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 msgr2=0x7fd42400e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.289+0000 7fd447804700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009e10 tx=0x7fd43c005040 comp rx=0 tx=0).stop 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 -- 172.21.15.47:0/2266215208 shutdown_connections 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd424006ce0 0x7fd42400b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd424003d10 0x7fd4240067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 --2- 172.21.15.47:0/2266215208 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd42400bfc0 0x7fd42400e3b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 -- 172.21.15.47:0/2266215208 >> 172.21.15.47:0/2266215208 conn(0x7fd4400a84a0 msgr2=0x7fd4400a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 -- 172.21.15.47:0/2266215208 shutdown_connections 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.290+0000 7fd447804700 1 -- 172.21.15.47:0/2266215208 wait complete. 2024-04-20T02:18:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 10 monclient: build_initial_monmap 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 10 monclient: monmap: 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 1 librados: starting msgr at 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 1 librados: starting objecter 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 1 Processor -- start 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 1 -- start start 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 1 librados: setting wanted keys 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 1 librados: calling monclient init 2024-04-20T02:18:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.291+0000 7fd447804700 10 monclient: init 2024-04-20T02:18:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient: _add_conns ranks=[8,0,4,5,3,2,6,7,1] 2024-04-20T02:18:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 0x7fd4401c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): picked mon.noname-e con 0x7fd424003d10 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd424006ce0 0x7fd4401349b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): picked mon.noname-a con 0x7fd424006ce0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd42400bfc0 0x7fd440134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): picked mon.noname-c con 0x7fd42400bfc0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd440121ad0 con 0x7fd424006ce0 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4401209a0 con 0x7fd42400bfc0 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd440071260 con 0x7fd424003d10 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:14.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd447804700 10 monclient(hunting): authenticate will time out at 1773.693141s 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd445da1700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd42400bfc0 msgr2=0x7fd440134ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd445da1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd42400bfc0 0x7fd440134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 0x7fd4401c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 0x7fd4401c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39682/0 (socket says 172.21.15.47:39682) 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 -- 172.21.15.47:0/2967184502 learned_addr learned my addr 172.21.15.47:0/2967184502 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): get_auth_request con 0x7fd424003d10 auth_method 0 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd424006ce0 0x7fd4401349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): get_auth_request con 0x7fd424006ce0 auth_method 0 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd444d9f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient(hunting): handle_auth_done global_id 34316 payload 824 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 -- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd42400bfc0 msgr2=0x7fd440134ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd42400bfc0 0x7fd440134ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 -- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd424006ce0 msgr2=0x7fd4401349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd424006ce0 0x7fd4401349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient: _finish_hunting 0 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 monclient: found mon.noname-e 2024-04-20T02:18:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 -- 172.21.15.47:0/2967184502 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c009a70 con 0x7fd424003d10 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd4455a0700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 0x7fd4401c7f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00ba60 tx=0x7fd43c005d30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd435ffb700 10 client.?.objecter ms_handle_connect 0x7fd424003d10 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.292+0000 7fd435ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd43c01ae70 con 0x7fd424003d10 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: dump: 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: _finish_auth 0 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c022a70 con 0x7fd424003d10 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 5 monclient: authenticate success, global_id 34316 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 10 monclient: _renew_subs 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4400a21e0 con 0x7fd424003d10 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd43c022be0 con 0x7fd424003d10 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd435ffb700 10 monclient: dump: 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd4377fe700 4 set_mon_vals no callback set 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 10 client.34316.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 10 monclient: _renew_subs 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44011ffa0 con 0x7fd424003d10 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.293+0000 7fd447804700 1 librados: init done 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd43c022d40 con 0x7fd424003d10 2024-04-20T02:18:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd447804700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd447804700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd447804700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd424040740 0x7fd424042c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd440121fd0 con 0x7fd424003d10 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7905+0+0 (secure 0 0 0) 0x7fd43c04ddc0 con 0x7fd424003d10 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 10 client.34316.objecter ms_dispatch 0x7fd440001550 osd_map(90..90 src has 1..90) v4 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 3 client.34316.objecter handle_osd_map got epochs [90,90] > 0 2024-04-20T02:18:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 3 client.34316.objecter handle_osd_map decoding full epoch 90 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd435ffb700 20 client.34316.objecter dump_active .. 0 homeless 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd424040740 0x7fd424042c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.294+0000 7fd444d9f700 10 monclient: get_auth_request con 0x7fd424040740 auth_method 0 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.295+0000 7fd435ffb700 10 client.34316.objecter ms_handle_connect 0x7fd424040740 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.295+0000 7fd444d9f700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd424040740 0x7fd424042c00 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7fd44009e9c0 tx=0x7fd430001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.300+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd440121fd0 con 0x7fd424003d10 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.300+0000 7fd435ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.300+0000 7fd435ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.609+0000 7fd447804700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.609+0000 7fd447804700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.609+0000 7fd447804700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.609+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd440121fd0 con 0x7fd424003d10 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.610+0000 7fd435ffb700 1 -- 172.21.15.47:0/2967184502 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3151 (secure 0 0 0) 0x7fd440121fd0 con 0x7fd424003d10 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.610+0000 7fd435ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:18:14.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.610+0000 7fd435ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:14.611 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":22,"quorum":[0,1,2,3,8],"quorum_names":["a","f","b","g","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:18:14.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 10 librados: watch_flush enter 2024-04-20T02:18:14.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 10 librados: watch_flush exit 2024-04-20T02:18:14.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 20 client.34316.objecter shutdown clearing up homeless session... 2024-04-20T02:18:14.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 10 client.34316.objecter successfully canceled tick 2024-04-20T02:18:14.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd424040740 msgr2=0x7fd424042c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd424040740 0x7fd424042c00 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7fd44009e9c0 tx=0x7fd430001120 comp rx=0 tx=0).stop 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 10 monclient: shutdown 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 msgr2=0x7fd4401c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 0x7fd4401c7f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00ba60 tx=0x7fd43c005d30 comp rx=0 tx=0).stop 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 shutdown_connections 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd424040740 0x7fd424042c00 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd42400bfc0 0x7fd440134ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd424003d10 0x7fd4401c7f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.612+0000 7fd447804700 1 --2- 172.21.15.47:0/2967184502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd424006ce0 0x7fd4401349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.613+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 >> 172.21.15.47:0/2967184502 conn(0x7fd4400a84a0 msgr2=0x7fd44006cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.613+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 shutdown_connections 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.613+0000 7fd447804700 1 -- 172.21.15.47:0/2967184502 wait complete. 2024-04-20T02:18:14.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.613+0000 7fd447804700 1 librados: shutdown 2024-04-20T02:18:14.626 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-04-20T02:18:14.626 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-04-20T02:18:14.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.764+0000 7f7bf1f7f700 10 monclient: get_monmap_and_config 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.764+0000 7f7bf1f7f700 10 monclient: build_initial_monmap 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.764+0000 7f7bf1f7f700 10 monclient: monmap: 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:14.764490+0000 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:14.764490+0000 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:14.763 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:14.764 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 Processor -- start 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 -- start start 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient: init 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient: _add_conns ranks=[1,2,8,10,3,9,7,4,6,5,0] 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7bec137d90 addr v1:172.21.15.179:6789/0 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 0x7f7bec13cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): picked mon.noname-b con 0x7f7bec0a09e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec13f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): picked mon.noname-e con 0x7f7bec13d1c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec0751e0 con 0x7f7bec137d90 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec09fa40 con 0x7f7bec0a09e0 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec09dfc0 con 0x7f7bec13d1c0 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient(hunting): authenticate will time out at 1774.165768s 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 0x7f7bec13cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec13f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 0x7f7bec13cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53468/0 (socket says 172.21.15.47:53468) 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec13f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39692/0 (socket says 172.21.15.47:39692) 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 -- 172.21.15.47:0/1362794763 learned_addr learned my addr 172.21.15.47:0/1362794763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): get_auth_request con 0x7f7bec0a09e0 auth_method 0 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): get_auth_request con 0x7f7bec13d1c0 auth_method 0 2024-04-20T02:18:14.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient(hunting): handle_auth_done global_id 23258 payload 274 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beb7fe700 1 -- 172.21.15.47:0/1362794763 >> v1:172.21.15.179:6789/0 conn(0x7f7bec137d90 legacy=0x7f7bec138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beb7fe700 1 -- 172.21.15.47:0/1362794763 >> v1:172.21.15.179:6789/0 conn(0x7f7bec137d90 legacy=0x7f7bec138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beb7fe700 1 --1- 172.21.15.47:0/1362794763 >> v1:172.21.15.179:6789/0 conn(0x7f7bec137d90 0x7f7bec138170 :-1 s=OPENED pgs=2328 cs=1 l=1).handle_message read tag failed 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beb7fe700 1 --1- 172.21.15.47:0/1362794763 >> v1:172.21.15.179:6789/0 conn(0x7f7bec137d90 0x7f7bec138170 :-1 s=OPENED pgs=2328 cs=1 l=1).fault on lossy channel, failing 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 -- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 msgr2=0x7f7bec13f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 --2- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec13f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 -- 172.21.15.47:0/1362794763 >> v1:172.21.15.179:6789/0 conn(0x7f7bec137d90 legacy=0x7f7bec138170 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 monclient: found mon.noname-b 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 -- 172.21.15.47:0/1362794763 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec138880 con 0x7f7bec0a09e0 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bebfff700 1 --2- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: ms_handle_reset stray mon v1:172.21.15.179:6789/0 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7beaffd700 1 --2- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 0x7f7bec13cc80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009870 tx=0x7f7be0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ac20634057964 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 1 -- 172.21.15.47:0/1362794763 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7be00044a0 con 0x7f7bec0a09e0 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: dump: 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.778 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: _finish_auth 0 2024-04-20T02:18:14.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 5 monclient: authenticate success, global_id 23258 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 1 -- 172.21.15.47:0/1362794763 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be00036a0 con 0x7f7bec0a09e0 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bf1f7f700 4 set_mon_vals no callback set 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 1 -- 172.21.15.47:0/1362794763 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7be0003c80 con 0x7f7bec0a09e0 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.765+0000 7f7bea7fc700 10 monclient: dump: 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.780 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 10 monclient: shutdown 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 msgr2=0x7f7bec13cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 0x7f7bec13cc80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009870 tx=0x7f7be0009b80 comp rx=0 tx=0).stop 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/1362794763 shutdown_connections 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/1362794763 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec0a09e0 0x7f7bec13cc80 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/1362794763 >> 172.21.15.47:0/1362794763 conn(0x7f7bec0aa8b0 msgr2=0x7f7bec0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/1362794763 shutdown_connections 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.766+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/1362794763 wait complete. 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.767+0000 7f7bf1f7f700 10 monclient: build_initial_monmap 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.767+0000 7f7bf1f7f700 10 monclient: monmap: 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:14.782 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.767+0000 7f7bf1f7f700 1 librados: starting msgr at 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.767+0000 7f7bf1f7f700 1 librados: starting objecter 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.767+0000 7f7bf1f7f700 1 Processor -- start 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 -- start start 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 librados: setting wanted keys 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 librados: calling monclient init 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient: init 2024-04-20T02:18:14.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient: _add_conns ranks=[6,2,8,7,3,5,0,1,4] 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7bec0a09e0 0x7f7bec09b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): picked mon.noname-d con 0x7f7bec0a09e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 0x7f7bec09bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): picked mon.noname-b con 0x7f7bec137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beb7fe700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7bec0a09e0 msgr2=0x7f7bec09b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec09c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): picked mon.noname-e con 0x7f7bec13d1c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beb7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7bec0a09e0 0x7f7bec09b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0a2690 con 0x7f7bec137d90 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0a2970 con 0x7f7bec0a09e0 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0a2f30 con 0x7f7bec13d1c0 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bf1f7f700 10 monclient(hunting): authenticate will time out at 1774.169800s 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 0x7f7bec09bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 0x7f7bec09bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53476/0 (socket says 172.21.15.47:53476) 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 1 -- 172.21.15.47:0/963548038 learned_addr learned my addr 172.21.15.47:0/963548038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 10 monclient(hunting): get_auth_request con 0x7f7bec137d90 auth_method 0 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bebfff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec09c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bebfff700 10 monclient(hunting): get_auth_request con 0x7f7bec13d1c0 auth_method 0 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bebfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bebfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7bebfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.768+0000 7f7beaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bebfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bebfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bebfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 10 monclient(hunting): handle_auth_done global_id 23267 payload 824 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 msgr2=0x7f7bec09c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7bec13d1c0 0x7f7bec09c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7bec0a09e0 msgr2=0x7f7bec09b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7bec0a09e0 0x7f7bec09b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 monclient: found mon.noname-b 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:18:14.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be4009a70 con 0x7f7bec137d90 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7beaffd700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 0x7f7bec09bc60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009130 tx=0x7f7be0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 client.?.objecter ms_handle_connect 0x7f7bec137d90 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7be0003c50 con 0x7f7bec137d90 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: dump: 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.787 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.788 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: _finish_auth 0 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0003df0 con 0x7f7bec137d90 2024-04-20T02:18:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 5 monclient: authenticate success, global_id 23267 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 10 monclient: _renew_subs 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be0009550 con 0x7f7bec137d90 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7be0024470 con 0x7f7bec137d90 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bd3fff700 10 monclient: dump: 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.790 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7be9ffb700 4 set_mon_vals no callback set 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 10 client.23267.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 10 monclient: _renew_subs 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:14.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec09ceb0 con 0x7f7bec137d90 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.769+0000 7f7bf1f7f700 1 librados: init done 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bf1f7f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 0x7f7bec12e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec068180 con 0x7f7bec0a36c0 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7be0021070 con 0x7f7bec137d90 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 0x7f7bec12e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient: get_auth_request con 0x7f7bec0a36c0 auth_method 0 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bd803a2f0 0x7f7bd803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7905+0+0 (secure 0 0 0) 0x7f7be004c250 con 0x7f7bec137d90 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bd3fff700 10 client.23267.objecter ms_dispatch 0x7f7bec001550 osd_map(90..90 src has 1..90) v4 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bd3fff700 3 client.23267.objecter handle_osd_map got epochs [90,90] > 0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7bd3fff700 3 client.23267.objecter handle_osd_map decoding full epoch 90 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_done global_id 34416 payload 824 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.770+0000 7f7beb7fe700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bec0a7bc0 con 0x7f7bec0a36c0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 20 client.23267.objecter dump_active .. 0 homeless 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 client.23267.objecter ms_handle_connect 0x7f7bec0a36c0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 client.23267.objecter resend_mon_ops 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7beb7fe700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 0x7f7bec12e9b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c470 tx=0x7f7bdc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bebfff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bd803a2f0 0x7f7bd803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bebfff700 10 monclient: get_auth_request con 0x7f7bd803a2f0 auth_method 0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bdc0070d0 con 0x7f7bec0a36c0 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 monclient: dump: 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.794 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.795 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7bdc005a60 con 0x7f7bec0a36c0 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 msgr2=0x7f7bec12e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 0x7f7bec12e9b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c470 tx=0x7f7bdc00c830 comp rx=0 tx=0).stop 2024-04-20T02:18:14.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 msgr2=0x7f7bec12e9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec0a36c0 0x7f7bec12e9b0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c470 tx=0x7f7bdc00c830 comp rx=0 tx=0).stop 2024-04-20T02:18:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bd3fff700 10 client.23267.objecter ms_handle_connect 0x7f7bd803a2f0 2024-04-20T02:18:14.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.771+0000 7f7bebfff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bd803a2f0 0x7f7bd803c7b0 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7f7bec1350a0 tx=0x7f7be400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:18:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 0x7f7bec12eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec06a400 con 0x7f7bec09d510 2024-04-20T02:18:14.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 0x7f7bec12eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient: get_auth_request con 0x7f7bec09d510 auth_method 0 2024-04-20T02:18:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:14.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 10 monclient(hunting): handle_auth_done global_id 34425 payload 824 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 1 -- 172.21.15.47:0/963548038 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7bec0a0000 con 0x7f7bec09d510 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 client.23267.objecter ms_handle_connect 0x7f7bec09d510 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7beb7fe700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 0x7f7bec12eef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c470 tx=0x7f7bdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 client.23267.objecter resend_mon_ops 2024-04-20T02:18:14.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bdc01f070 con 0x7f7bec09d510 2024-04-20T02:18:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 monclient: dump: 2024-04-20T02:18:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.804 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:14.805 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:14.806 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:14.807 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f7bdc0192c0 con 0x7f7bec09d510 2024-04-20T02:18:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:18:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 msgr2=0x7f7bec12eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 0x7f7bec12eef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c470 tx=0x7f7bdc000bc0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 msgr2=0x7f7bec12eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:14.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bd3fff700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bec09d510 0x7f7bec12eef0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c470 tx=0x7f7bdc000bc0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.808 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:18:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:18:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:18:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 22, 2024-04-20T02:18:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:18:14.809 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:18:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:18:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:18:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:18:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:18:14.810 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:14.811 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:14.813 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:14.814 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:18:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:18:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:14.815 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:14.816 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:18:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:18:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:18:14.817 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:18:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:18:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:18:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:18:14.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:18:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:18:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:18:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:18:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:18:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:18:14.819 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:18:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:14.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:14.821 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:18:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:18:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:18:14.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:18:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:18:14.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:18:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.824 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:18:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:18:14.825 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:18:14.826 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:18:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:18:14.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:18:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:18:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:18:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.829 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:18:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:18:14.830 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:18:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:18:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:18:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:18:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.833 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:18:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:18:14.834 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:18:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:18:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:18:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:18:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:18:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:18:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:18:14.840 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:18:14.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.845 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:18:14.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:18:14.847 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:14.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:18:14.851 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:14.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 10 librados: watch_flush enter 2024-04-20T02:18:14.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 10 librados: watch_flush exit 2024-04-20T02:18:14.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 20 client.23267.objecter shutdown clearing up homeless session... 2024-04-20T02:18:14.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 10 client.23267.objecter successfully canceled tick 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bd803a2f0 msgr2=0x7f7bd803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7bd803a2f0 0x7f7bd803c7b0 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7f7bec1350a0 tx=0x7f7be400b040 comp rx=0 tx=0).stop 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 10 monclient: shutdown 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 msgr2=0x7f7bec09bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 0x7f7bec09bc60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009130 tx=0x7f7be0004750 comp rx=0 tx=0).stop 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bebfff700 1 -- 172.21.15.47:0/963548038 reap_dead start 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 shutdown_connections 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 --2- 172.21.15.47:0/963548038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7bec137d90 0x7f7bec09bc60 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.780+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 >> 172.21.15.47:0/963548038 conn(0x7f7bec0aa8b0 msgr2=0x7f7bec1352e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.781+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 shutdown_connections 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.781+0000 7f7bf1f7f700 1 -- 172.21.15.47:0/963548038 wait complete. 2024-04-20T02:18:14.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.781+0000 7f7bf1f7f700 1 librados: shutdown 2024-04-20T02:18:14.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:18:14.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon b 2024-04-20T02:18:14.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.996+0000 7f434e0d6700 10 monclient: get_monmap_and_config 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.996+0000 7f434e0d6700 10 monclient: build_initial_monmap 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.996+0000 7f434e0d6700 10 monclient: monmap: 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:14.996535+0000 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:14.996535+0000 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:14.995 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:14.996 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 Processor -- start 2024-04-20T02:18:15.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 -- start start 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient: init 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient: _add_conns ranks=[0,5,9,7,4,1,8,10,2,3,6] 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f434809fa80 addr v1:172.21.15.47:6789/0 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f434809fe60 0x7f43480a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): picked mon.noname-h con 0x7f434809fe60 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f434813ac00 0x7f434813cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): picked mon.noname-a con 0x7f434813ac00 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480751e0 con 0x7f434809fa80 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43480a0780 con 0x7f434813ac00 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f434809df90 con 0x7f434809fe60 2024-04-20T02:18:15.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f434e0d6700 10 monclient(hunting): authenticate will time out at 1774.397879s 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f43477fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f434809fa80 0x7f4348135df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39194/0 (socket says 172.21.15.47:39194) 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f43477fe700 1 -- 172.21.15.47:0/3602944831 learned_addr learned my addr 172.21.15.47:0/3602944831 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.997+0000 7f4346ffd700 1 -- 172.21.15.47:0/3602944831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f434809fe60 msgr2=0x7f43480a0240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4346ffd700 1 --2- 172.21.15.47:0/3602944831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f434809fe60 0x7f43480a0240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 --2- 172.21.15.47:0/3602944831 >> v2:172.21.15.47:3300/0 conn(0x7f434813ac00 0x7f434813cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): get_auth_request con 0x7f434813ac00 auth_method 0 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 1 -- 172.21.15.47:0/3602944831 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024702218 0 0) 0x7f43480751e0 con 0x7f434809fa80 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient(hunting): _negotiate 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient(hunting): my global_id is 34443 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 1 -- 172.21.15.47:0/3602944831 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43300035b0 con 0x7f434809fa80 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient(hunting): handle_auth_done global_id 34434 payload 274 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 -- 172.21.15.47:0/3602944831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f434809fe60 msgr2=0x7f43480a0240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 --2- 172.21.15.47:0/3602944831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f434809fe60 0x7f43480a0240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 -- 172.21.15.47:0/3602944831 >> v1:172.21.15.47:6789/0 conn(0x7f434809fa80 legacy=0x7f4348135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient: _finish_hunting 0 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 monclient: found mon.noname-a 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:15.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 -- 172.21.15.47:0/3602944831 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434809dcb0 con 0x7f434813ac00 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f4347fff700 1 --2- 172.21.15.47:0/3602944831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434813ac00 0x7f434813cff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4340009d90 tx=0x7f434000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793f96ce968f07ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 1 -- 172.21.15.47:0/3602944831 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f434000e070 con 0x7f434813ac00 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: dump: 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.012 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: _finish_auth 0 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: _check_auth_tickets 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 1 -- 172.21.15.47:0/3602944831 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340012430 con 0x7f434813ac00 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f434e0d6700 5 monclient: authenticate success, global_id 34434 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f434e0d6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:15.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f434e0d6700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f434e0d6700 4 set_mon_vals no callback set 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 1 -- 172.21.15.47:0/3602944831 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4340012a10 con 0x7f434813ac00 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.998+0000 7f43467fc700 10 monclient: dump: 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.014 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 10 monclient: shutdown 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 -- 172.21.15.47:0/3602944831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434813ac00 msgr2=0x7f434813cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3602944831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434813ac00 0x7f434813cff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4340009d90 tx=0x7f434000b330 comp rx=0 tx=0).stop 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 -- 172.21.15.47:0/3602944831 shutdown_connections 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3602944831 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434813ac00 0x7f434813cff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3602944831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f434809fe60 0x7f43480a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 -- 172.21.15.47:0/3602944831 >> 172.21.15.47:0/3602944831 conn(0x7f43480a3000 msgr2=0x7f43480a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 -- 172.21.15.47:0/3602944831 shutdown_connections 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:14.999+0000 7f434e0d6700 1 -- 172.21.15.47:0/3602944831 wait complete. 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.000+0000 7f434e0d6700 10 monclient: build_initial_monmap 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.000+0000 7f434e0d6700 10 monclient: monmap: 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:15.017 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.000+0000 7f434e0d6700 1 librados: starting msgr at 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.000+0000 7f434e0d6700 1 librados: starting objecter 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.000+0000 7f434e0d6700 1 Processor -- start 2024-04-20T02:18:15.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 -- start start 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 librados: setting wanted keys 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 librados: calling monclient init 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient: init 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient: _add_conns ranks=[0,8,6,7,4,5,1,3,2] 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434809fa80 0x7f43481ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): picked mon.noname-a con 0x7f434809fa80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f434809fe60 0x7f43481ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): picked mon.noname-e con 0x7f434809fe60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f434813ac00 0x7f43481cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): picked mon.noname-d con 0x7f434813ac00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348070b50 con 0x7f434809fa80 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4348071870 con 0x7f434813ac00 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4348071e30 con 0x7f434809fe60 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f434e0d6700 10 monclient(hunting): authenticate will time out at 1774.401886s 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f4347fff700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f434813ac00 msgr2=0x7f43481cac70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f4347fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f434813ac00 0x7f43481cac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.001+0000 7f43477fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434809fa80 0x7f43481ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f434809fe60 0x7f43481ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434809fa80 0x7f43481ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52918/0 (socket says 172.21.15.47:52918) 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 -- 172.21.15.47:0/3513388719 learned_addr learned my addr 172.21.15.47:0/3513388719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): get_auth_request con 0x7f434809fa80 auth_method 0 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): get_auth_request con 0x7f434809fe60 auth_method 0 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient(hunting): handle_auth_done global_id 34452 payload 824 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f434809fe60 msgr2=0x7f43481ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f434809fe60 0x7f43481ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f434813ac00 msgr2=0x7f43481cac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f434813ac00 0x7f43481cac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient: _finish_hunting 0 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 monclient: found mon.noname-a 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009a70 con 0x7f434809fa80 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4346ffd700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f434809fe60 0x7f43481ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f43477fe700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434809fa80 0x7f43481ca1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f433800c4d0 tx=0x7f433800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 client.?.objecter ms_handle_connect 0x7f434809fa80 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43380070d0 con 0x7f434809fa80 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: dump: 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.023 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: _finish_auth 0 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 5 monclient: authenticate success, global_id 34452 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4338007230 con 0x7f434809fa80 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 10 monclient: _renew_subs 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4348071b50 con 0x7f434809fa80 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4338007810 con 0x7f434809fa80 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f4345ffb700 4 set_mon_vals no callback set 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 10 client.34452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f432ffff700 10 monclient: dump: 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.025 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 10 monclient: _renew_subs 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4348072af0 con 0x7f434809fa80 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.002+0000 7f434e0d6700 1 librados: init done 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f434e0d6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 0x7f4348095ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f43380073a0 con 0x7f434809fa80 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434813a2c0 con 0x7f434806aac0 2024-04-20T02:18:15.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 0x7f4348095ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient: get_auth_request con 0x7f434806aac0 auth_method 0 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f433403a2f0 0x7f433403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 7905+0+0 (secure 0 0 0) 0x7f4338049e00 con 0x7f434809fa80 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 10 client.34452.objecter ms_dispatch 0x7f4348001550 osd_map(90..90 src has 1..90) v4 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 3 client.34452.objecter handle_osd_map got epochs [90,90] > 0 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 3 client.34452.objecter handle_osd_map decoding full epoch 90 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.003+0000 7f432ffff700 20 client.34452.objecter dump_active .. 0 homeless 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f4346ffd700 10 monclient(hunting): handle_auth_done global_id 23276 payload 824 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f4346ffd700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43480719e0 con 0x7f434806aac0 2024-04-20T02:18:15.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f4346ffd700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 0x7f4348095ff0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f434806d2e0 tx=0x7f433c005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 client.34452.objecter ms_handle_connect 0x7f434806aac0 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 client.34452.objecter resend_mon_ops 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f4347fff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f433403a2f0 0x7f433403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f4347fff700 10 monclient: get_auth_request con 0x7f433403a2f0 auth_method 0 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f433c00ec10 con 0x7f434806aac0 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 monclient: dump: 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.028 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f433c00ed70 con 0x7f434806aac0 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 msgr2=0x7f4348095ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 0x7f4348095ff0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f434806d2e0 tx=0x7f433c005980 comp rx=0 tx=0).stop 2024-04-20T02:18:15.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 msgr2=0x7f4348095ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f434806aac0 0x7f4348095ff0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f432ffff700 10 client.34452.objecter ms_handle_connect 0x7f433403a2f0 2024-04-20T02:18:15.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.004+0000 7f4347fff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f433403a2f0 0x7f433403c7b0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7f43480a0bc0 tx=0x7f4340011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f434e0d6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 0x7f4348096530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f434e0d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4348068180 con 0x7f4348070290 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 0x7f4348096530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient: get_auth_request con 0x7f4348070290 auth_method 0 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.012+0000 7f4346ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f4346ffd700 10 monclient(hunting): handle_auth_done global_id 23285 payload 824 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f4346ffd700 1 -- 172.21.15.47:0/3513388719 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f434809e550 con 0x7f4348070290 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f4346ffd700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 0x7f4348096530 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f433c018000 tx=0x7f433c005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 client.34452.objecter ms_handle_connect 0x7f4348070290 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 client.34452.objecter resend_mon_ops 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f433c005ce0 con 0x7f4348070290 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 monclient: dump: 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.033 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f433c005e40 con 0x7f4348070290 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:18:15.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 msgr2=0x7f4348096530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 0x7f4348096530 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f433c018000 tx=0x7f433c005a60 comp rx=0 tx=0).stop 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 msgr2=0x7f4348096530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.013+0000 7f432ffff700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4348070290 0x7f4348096530 secure :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f433c018000 tx=0x7f433c005a60 comp rx=0 tx=0).stop 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 22, 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:18:15.035 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:15.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:15.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:18:15.038 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:15.039 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:18:15.040 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:18:15.041 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.043 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:18:15.045 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:18:15.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.049 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:18:15.051 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:18:15.052 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.054 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.055 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:18:15.056 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 10 librados: watch_flush enter 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 10 librados: watch_flush exit 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 20 client.34452.objecter shutdown clearing up homeless session... 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 10 client.34452.objecter successfully canceled tick 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f433403a2f0 msgr2=0x7f433403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f433403a2f0 0x7f433403c7b0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7f43480a0bc0 tx=0x7f4340011000 comp rx=0 tx=0).stop 2024-04-20T02:18:15.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 10 monclient: shutdown 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434809fa80 msgr2=0x7f43481ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 --2- 172.21.15.47:0/3513388719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f434809fa80 0x7f43481ca1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f433800c4d0 tx=0x7f433800c7e0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f4347fff700 1 -- 172.21.15.47:0/3513388719 reap_dead start 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f4347fff700 1 -- 172.21.15.47:0/3513388719 reap_dead start 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 shutdown_connections 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 >> 172.21.15.47:0/3513388719 conn(0x7f43480a3000 msgr2=0x7f4348099290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 shutdown_connections 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 -- 172.21.15.47:0/3513388719 wait complete. 2024-04-20T02:18:15.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.014+0000 7f434e0d6700 1 librados: shutdown 2024-04-20T02:18:15.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:18:15.060 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-04-20T02:18:15.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:18:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.206+0000 7f2ecee09700 10 monclient: get_monmap_and_config 2024-04-20T02:18:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.206+0000 7f2ecee09700 10 monclient: build_initial_monmap 2024-04-20T02:18:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.206+0000 7f2ecee09700 10 monclient: monmap: 2024-04-20T02:18:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.205 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:15.206835+0000 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:15.206835+0000 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:15.206 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:15.207 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:15.207 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.207+0000 7f2ecee09700 1 Processor -- start 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.207+0000 7f2ecee09700 1 -- start start 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.207+0000 7f2ecee09700 10 monclient: init 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient: _add_conns ranks=[2,10,5,9,8,6,4,3,1,0,7] 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ec80a5c90 0x7f2ec80a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): picked mon.noname-b con 0x7f2ec80a5c90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ec80a0eb0 0x7f2ec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): picked mon.noname-f con 0x7f2ec80a0eb0 addr v2:172.21.15.179:3300/0 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2ec80a65b0 0x7f2ec813c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): picked mon.noname-h con 0x7f2ec80a65b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec809fd20 con 0x7f2ec80a0eb0 2024-04-20T02:18:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80a7bc0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec809f760 con 0x7f2ec80a65b0 2024-04-20T02:18:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:15.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ecee09700 10 monclient(hunting): authenticate will time out at 1774.608705s 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.208+0000 7f2ec7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ec80a0eb0 0x7f2ec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2ec80a0eb0 0x7f2ec80a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:59496/0 (socket says 172.21.15.47:59496) 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 1 -- 172.21.15.47:0/3821776239 learned_addr learned my addr 172.21.15.47:0/3821776239 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): get_auth_request con 0x7f2ec80a0eb0 auth_method 0 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ecd3a6700 1 -- 172.21.15.47:0/3821776239 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2ec80a65b0 msgr2=0x7f2ec813c210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:15.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.209+0000 7f2ecd3a6700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2ec80a65b0 0x7f2ec813c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ec80a5c90 0x7f2ec80a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): get_auth_request con 0x7f2ec80a5c90 auth_method 0 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient(hunting): handle_auth_done global_id 23294 payload 274 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 -- 172.21.15.47:0/3821776239 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2ec80a65b0 msgr2=0x7f2ec813c210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2ec80a65b0 0x7f2ec813c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 -- 172.21.15.47:0/3821776239 >> v2:172.21.15.179:3300/0 conn(0x7f2ec80a0eb0 msgr2=0x7f2ec80a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 --2- 172.21.15.47:0/3821776239 >> v2:172.21.15.179:3300/0 conn(0x7f2ec80a0eb0 0x7f2ec80a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient: _finish_hunting 0 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 monclient: found mon.noname-b 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.210+0000 7f2eccba5700 1 -- 172.21.15.47:0/3821776239 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80751e0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2eccba5700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ec80a5c90 0x7f2ec80a0970 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009840 tx=0x7f2ebc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2a6d485a311598 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 1 -- 172.21.15.47:0/3821776239 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ebc011aa0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: dump: 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.214 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: _finish_auth 0 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 1 -- 172.21.15.47:0/3821776239 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc011c00 con 0x7f2ec80a5c90 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ecee09700 5 monclient: authenticate success, global_id 23294 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ecee09700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ecee09700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ecee09700 4 set_mon_vals no callback set 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 1 -- 172.21.15.47:0/3821776239 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ebc011d70 con 0x7f2ec80a5c90 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.211+0000 7f2ec77fe700 10 monclient: dump: 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.216 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 10 monclient: shutdown 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 -- 172.21.15.47:0/3821776239 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ec80a5c90 msgr2=0x7f2ec80a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ec80a5c90 0x7f2ec80a0970 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009840 tx=0x7f2ebc009b50 comp rx=0 tx=0).stop 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 -- 172.21.15.47:0/3821776239 shutdown_connections 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2ec80a65b0 0x7f2ec813c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 --2- 172.21.15.47:0/3821776239 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2ec80a5c90 0x7f2ec80a0970 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 --2- 172.21.15.47:0/3821776239 >> v2:172.21.15.179:3300/0 conn(0x7f2ec80a0eb0 0x7f2ec80a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.212+0000 7f2ecee09700 1 -- 172.21.15.47:0/3821776239 >> 172.21.15.47:0/3821776239 conn(0x7f2ec80aa8b0 msgr2=0x7f2ec80a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.213+0000 7f2ecee09700 1 -- 172.21.15.47:0/3821776239 shutdown_connections 2024-04-20T02:18:15.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.214+0000 7f2ecee09700 1 -- 172.21.15.47:0/3821776239 wait complete. 2024-04-20T02:18:15.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 10 monclient: build_initial_monmap 2024-04-20T02:18:15.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 10 monclient: monmap: 2024-04-20T02:18:15.223 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:15.223 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:15.223 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:15.223 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:15.223 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:15.225 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:15.225 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:15.225 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 1 librados: starting msgr at 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 1 librados: starting objecter 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 1 Processor -- start 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 1 -- start start 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 1 librados: setting wanted keys 2024-04-20T02:18:15.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 1 librados: calling monclient init 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.215+0000 7f2ecee09700 10 monclient: init 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient: _add_conns ranks=[8,1,6,3,0,5,2,7,4] 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec80a5c90 0x7f2ec81c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): picked mon.noname-e con 0x7f2ec80a5c90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ec80a65b0 0x7f2ec81c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): picked mon.noname-f con 0x7f2ec80a65b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ec81cd930 0x7f2ec81addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): picked mon.noname-d con 0x7f2ec81cd930 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec81c3e80 con 0x7f2ec80a65b0 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec809d960 con 0x7f2ec81cd930 2024-04-20T02:18:15.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec8072880 con 0x7f2ec80a5c90 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2ecee09700 10 monclient(hunting): authenticate will time out at 1774.616569s 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec80a5c90 0x7f2ec81c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec80a5c90 0x7f2ec81c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39708/0 (socket says 172.21.15.47:39708) 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 1 -- 172.21.15.47:0/1863997803 learned_addr learned my addr 172.21.15.47:0/1863997803 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 10 monclient(hunting): get_auth_request con 0x7f2ec80a5c90 auth_method 0 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.216+0000 7f2eccba5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ecd3a6700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ec81cd930 msgr2=0x7f2ec81addf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ecd3a6700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ec81cd930 0x7f2ec81addf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ec7fff700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ec80a65b0 0x7f2ec81c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ec7fff700 10 monclient(hunting): get_auth_request con 0x7f2ec80a65b0 auth_method 0 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ec7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ec7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 10 monclient(hunting): handle_auth_done global_id 34352 payload 824 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ec81cd930 msgr2=0x7f2ec81addf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2ec81cd930 0x7f2ec81addf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ec80a65b0 msgr2=0x7f2ec81c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ec80a65b0 0x7f2ec81c3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 10 monclient: _finish_hunting 0 2024-04-20T02:18:15.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 monclient: found mon.noname-e 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0094a0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.217+0000 7f2eccba5700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec80a5c90 0x7f2ec81c3400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc000c00 tx=0x7f2ebc004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 client.?.objecter ms_handle_connect 0x7f2ec80a5c90 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ebc003b10 con 0x7f2ec80a5c90 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: dump: 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.230 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: _finish_auth 0 2024-04-20T02:18:15.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: _check_auth_tickets 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc01f070 con 0x7f2ec80a5c90 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ecee09700 5 monclient: authenticate success, global_id 34352 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ecee09700 10 monclient: _renew_subs 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ecee09700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec81b0520 con 0x7f2ec80a5c90 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2ebc003cd0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec57fa700 10 monclient: dump: 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.232 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ecee09700 10 client.34352.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:15.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.219+0000 7f2ec67fc700 4 set_mon_vals no callback set 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.220+0000 7f2ecee09700 10 monclient: _renew_subs 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.220+0000 7f2ecee09700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.220+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec81b0ae0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.220+0000 7f2ecee09700 1 librados: init done 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2ebc0036a0 con 0x7f2ec80a5c90 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ecee09700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ecee09700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 0x7f2ec81ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec8068180 con 0x7f2ec806aac0 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec57fa700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2eb403a2f0 0x7f2eb403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 0x7f2ec81ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient: get_auth_request con 0x7f2ec806aac0 auth_method 0 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 7323+0+0 (secure 0 0 0) 0x7f2ebc021070 con 0x7f2ec80a5c90 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec57fa700 10 client.34352.objecter ms_dispatch 0x7f2ec8001550 osd_map(91..91 src has 1..91) v4 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec57fa700 3 client.34352.objecter handle_osd_map got epochs [91,91] > 0 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec57fa700 3 client.34352.objecter handle_osd_map decoding full epoch 91 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ecd3a6700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2eb403a2f0 0x7f2eb403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.221+0000 7f2ecd3a6700 10 monclient: get_auth_request con 0x7f2eb403a2f0 auth_method 0 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 20 client.34352.objecter dump_active .. 0 homeless 2024-04-20T02:18:15.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_done global_id 34361 payload 824 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec7fff700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ec8067fb0 con 0x7f2ec806aac0 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 10 client.34352.objecter ms_handle_connect 0x7f2ec806aac0 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 10 client.34352.objecter resend_mon_ops 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec7fff700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 0x7f2ec81ae330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80a7760 tx=0x7f2eb8011980 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2eb8015ae0 con 0x7f2ec806aac0 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 10 monclient: dump: 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.236 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ecd3a6700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2eb403a2f0 0x7f2eb403c7b0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0003cf0 tx=0x7f2ec00091f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.222+0000 7f2ec57fa700 10 client.34352.objecter ms_handle_connect 0x7f2eb403a2f0 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2eb8015c40 con 0x7f2ec806aac0 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 msgr2=0x7f2ec81ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 0x7f2ec81ae330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80a7760 tx=0x7f2eb8011980 comp rx=0 tx=0).stop 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 msgr2=0x7f2ec81ae330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:15.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.223+0000 7f2ec57fa700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec806aac0 0x7f2ec81ae330 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80a7760 tx=0x7f2eb8011980 comp rx=0 tx=0).stop 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.236+0000 7f2ecee09700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.236+0000 7f2ecee09700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 0x7f2ec80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.236+0000 7f2ecee09700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.236+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec806a400 con 0x7f2ec81ae870 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 0x7f2ec80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient: get_auth_request con 0x7f2ec81ae870 auth_method 0 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 10 monclient(hunting): handle_auth_done global_id 34370 payload 824 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 1 -- 172.21.15.47:0/1863997803 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2ec80a6a60 con 0x7f2ec81ae870 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec57fa700 10 client.34352.objecter ms_handle_connect 0x7f2ec81ae870 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec57fa700 10 client.34352.objecter resend_mon_ops 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.237+0000 7f2ec7fff700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 0x7f2ec80a3690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2eb801b000 tx=0x7f2eb8004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2eb8025070 con 0x7f2ec81ae870 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 10 monclient: dump: 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.241 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.242 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f2eb8006490 con 0x7f2ec81ae870 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 msgr2=0x7f2ec80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 0x7f2ec80a3690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2eb801b000 tx=0x7f2eb8004750 comp rx=0 tx=0).stop 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 msgr2=0x7f2ec80a3690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.238+0000 7f2ec57fa700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec81ae870 0x7f2ec80a3690 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2eb801b000 tx=0x7f2eb8004750 comp rx=0 tx=0).stop 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:15.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 22, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:18:15.244 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.245 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.246 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:18:15.247 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:15.248 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.249 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.250 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.251 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:18:15.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.254 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.255 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:18:15.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.257 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.258 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:18:15.259 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:18:15.260 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.262 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.263 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:15.264 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:18:15.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:18:15.266 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 10 librados: watch_flush enter 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 10 librados: watch_flush exit 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 20 client.34352.objecter shutdown clearing up homeless session... 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 10 client.34352.objecter successfully canceled tick 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2eb403a2f0 msgr2=0x7f2eb403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2eb403a2f0 0x7f2eb403c7b0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0003cf0 tx=0x7f2ec00091f0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 10 monclient: shutdown 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:15.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec80a5c90 msgr2=0x7f2ec81c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 --2- 172.21.15.47:0/1863997803 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2ec80a5c90 0x7f2ec81c3400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc000c00 tx=0x7f2ebc004750 comp rx=0 tx=0).stop 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecd3a6700 1 -- 172.21.15.47:0/1863997803 reap_dead start 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecd3a6700 1 -- 172.21.15.47:0/1863997803 reap_dead start 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 shutdown_connections 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 >> 172.21.15.47:0/1863997803 conn(0x7f2ec80aa8b0 msgr2=0x7f2ec809ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 shutdown_connections 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 -- 172.21.15.47:0/1863997803 wait complete. 2024-04-20T02:18:15.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.239+0000 7f2ecee09700 1 librados: shutdown 2024-04-20T02:18:15.269 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:18:15.269 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon f 2024-04-20T02:18:15.270 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:18:15.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.408+0000 7f667dbba700 10 monclient: get_monmap_and_config 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.408+0000 7f667dbba700 10 monclient: build_initial_monmap 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.408+0000 7f667dbba700 10 monclient: monmap: 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:15.408427+0000 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:15.408427+0000 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:15.407 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:15.408 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 Processor -- start 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 -- start start 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient: init 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient: _add_conns ranks=[10,3,9,8,5,6,4,1,7,0,2] 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f66780a44f0 0x7f6678071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): picked mon.noname-f con 0x7f66780a44f0 addr v2:172.21.15.179:3300/0 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6678071e60 0x7f66780a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): picked mon.noname-g con 0x7f6678071e60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66780a2e20 0x7f66780a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): picked mon.noname-a con 0x7f66780a2e20 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66780a0390 con 0x7f66780a2e20 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66780a48d0 con 0x7f66780a44f0 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667809e930 con 0x7f6678071e60 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f667dbba700 10 monclient(hunting): authenticate will time out at 1774.809664s 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66780a2e20 0x7f66780a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66780a2e20 0x7f66780a32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52928/0 (socket says 172.21.15.47:52928) 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 1 -- 172.21.15.47:0/2938236163 learned_addr learned my addr 172.21.15.47:0/2938236163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): get_auth_request con 0x7f66780a2e20 auth_method 0 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6677fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f6676ffd700 1 --2- 172.21.15.47:0/2938236163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6678071e60 0x7f66780a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.409+0000 7f66777fe700 1 --2- 172.21.15.47:0/2938236163 >> v2:172.21.15.179:3300/0 conn(0x7f66780a44f0 0x7f6678071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66777fe700 10 monclient(hunting): get_auth_request con 0x7f66780a44f0 auth_method 0 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66777fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66777fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66777fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6676ffd700 10 monclient(hunting): get_auth_request con 0x7f6678071e60 auth_method 0 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6676ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:15.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6676ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6676ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 10 monclient(hunting): handle_auth_done global_id 34461 payload 274 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 -- 172.21.15.47:0/2938236163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6678071e60 msgr2=0x7f66780a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 --2- 172.21.15.47:0/2938236163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6678071e60 0x7f66780a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 -- 172.21.15.47:0/2938236163 >> v2:172.21.15.179:3300/0 conn(0x7f66780a44f0 msgr2=0x7f6678071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 --2- 172.21.15.47:0/2938236163 >> v2:172.21.15.179:3300/0 conn(0x7f66780a44f0 0x7f6678071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 10 monclient: _finish_hunting 0 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 monclient: found mon.noname-a 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 -- 172.21.15.47:0/2938236163 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66780751e0 con 0x7f66780a2e20 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66777fe700 1 --2- 172.21.15.47:0/2938236163 >> v2:172.21.15.179:3300/0 conn(0x7f66780a44f0 0x7f6678071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f6677fff700 1 --2- 172.21.15.47:0/2938236163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66780a2e20 0x7f66780a32d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e50 tx=0x7f667000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefe4c02f66ae6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:15.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 1 -- 172.21.15.47:0/2938236163 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f667000e070 con 0x7f66780a2e20 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: dump: 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.419 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: _finish_auth 0 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: _check_auth_tickets 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f667dbba700 5 monclient: authenticate success, global_id 34461 2024-04-20T02:18:15.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 1 -- 172.21.15.47:0/2938236163 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66700099b0 con 0x7f66780a2e20 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f667dbba700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 1 -- 172.21.15.47:0/2938236163 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f667000bdf0 con 0x7f66780a2e20 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f667dbba700 4 set_mon_vals no callback set 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.410+0000 7f66767fc700 10 monclient: dump: 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:15.421 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:15.422 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 10 monclient: shutdown 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 -- 172.21.15.47:0/2938236163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66780a2e20 msgr2=0x7f66780a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 --2- 172.21.15.47:0/2938236163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66780a2e20 0x7f66780a32d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6670009e50 tx=0x7f667000b330 comp rx=0 tx=0).stop 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 -- 172.21.15.47:0/2938236163 shutdown_connections 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 --2- 172.21.15.47:0/2938236163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66780a2e20 0x7f66780a32d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 --2- 172.21.15.47:0/2938236163 >> v2:172.21.15.179:3300/0 conn(0x7f66780a44f0 0x7f6678071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 --2- 172.21.15.47:0/2938236163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6678071e60 0x7f66780a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 -- 172.21.15.47:0/2938236163 >> 172.21.15.47:0/2938236163 conn(0x7f66780a39d0 msgr2=0x7f66780a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 -- 172.21.15.47:0/2938236163 shutdown_connections 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 1 -- 172.21.15.47:0/2938236163 wait complete. 2024-04-20T02:18:15.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 10 monclient: build_initial_monmap 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.411+0000 7f667dbba700 10 monclient: monmap: 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:15.424 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 1 librados: starting msgr at 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 1 librados: starting objecter 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 1 Processor -- start 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 1 -- start start 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 1 librados: setting wanted keys 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 1 librados: calling monclient init 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.412+0000 7f667dbba700 10 monclient: init 2024-04-20T02:18:15.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient: _add_conns ranks=[7,5,4,1,2,3,8,6,0] 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 0x7f66781ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): picked mon.noname-i con 0x7f6678071e60 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 0x7f66781cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): picked mon.noname-h con 0x7f66780a2e20 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 0x7f6678122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): picked mon.noname-c con 0x7f66780a44f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6678126970 con 0x7f66780a44f0 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6678126bc0 con 0x7f66780a2e20 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f667809f3c0 con 0x7f6678071e60 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:15.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f667dbba700 10 monclient(hunting): authenticate will time out at 1774.813153s 2024-04-20T02:18:15.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f6677fff700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 msgr2=0x7f6678122ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:15.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f6677fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 0x7f6678122ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f6676ffd700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 msgr2=0x7f66781cad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:15.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f66777fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 msgr2=0x7f66781ca7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:15.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f6676ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 0x7f66781cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.413+0000 7f66777fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 0x7f66781ca7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.614+0000 7f6677fff700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 msgr2=0x7f6678122ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.614+0000 7f6677fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 0x7f6678122ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.614+0000 7f6676ffd700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 msgr2=0x7f66781cad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.614+0000 7f6676ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 0x7f66781cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.614+0000 7f66777fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 msgr2=0x7f66781ca7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:15.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:15.614+0000 7f66777fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 0x7f66781ca7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:18:16.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.015+0000 7f6677fff700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 msgr2=0x7f6678122ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:16.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.015+0000 7f6677fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 0x7f6678122ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:16.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.015+0000 7f6676ffd700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 msgr2=0x7f66781cad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:16.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.015+0000 7f6676ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 0x7f66781cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:16.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.015+0000 7f66777fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 msgr2=0x7f66781ca7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:16.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.015+0000 7f66777fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 0x7f66781ca7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:18:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.816+0000 7f6677fff700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 msgr2=0x7f6678122ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.816+0000 7f6677fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 0x7f6678122ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.816+0000 7f6676ffd700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 msgr2=0x7f66781cad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:18:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.816+0000 7f66777fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 msgr2=0x7f66781ca7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:18:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.816+0000 7f66777fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 0x7f66781ca7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:16.816+0000 7f6676ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 0x7f66781cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:18:18.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): tick 2024-04-20T02:18:18.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:18:18.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 monclient(hunting): continuing hunt 2024-04-20T02:18:18.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:18:18.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 msgr2=0x7f66781ca7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:18.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6678071e60 0x7f66781ca7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 msgr2=0x7f66781cad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f66780a2e20 0x7f66781cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 msgr2=0x7f6678122ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f66780a44f0 0x7f6678122ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient: _add_conns ranks=[2,3,6,8,0,1] 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 0x7f6658005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): picked mon.noname-b con 0x7f6658003890 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f66580062c0 0x7f6658008750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): picked mon.noname-g con 0x7f66580062c0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6658008df0 0x7f665800b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): picked mon.noname-d con 0x7f6658008df0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6678126970 con 0x7f6658003890 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6678126bc0 con 0x7f66580062c0 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f667809f3c0 con 0x7f6658008df0 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f665effd700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6677fff700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6658008df0 msgr2=0x7f665800b2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6677fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6658008df0 0x7f665800b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 0x7f6658005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 0x7f6658005cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53508/0 (socket says 172.21.15.47:53508) 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f66580062c0 0x7f6658008750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 -- 172.21.15.47:0/2351669057 learned_addr learned my addr 172.21.15.47:0/2351669057 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): get_auth_request con 0x7f6658003890 auth_method 0 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): get_auth_request con 0x7f66580062c0 auth_method 0 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient(hunting): handle_auth_done global_id 23303 payload 824 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6658008df0 msgr2=0x7f665800b2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6658008df0 0x7f665800b2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f66580062c0 msgr2=0x7f6658008750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f66580062c0 0x7f6658008750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient: _finish_hunting 0 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 monclient: found mon.noname-b 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6677fff700 1 -- 172.21.15.47:0/2351669057 reap_dead start 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f66777fe700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009b30 con 0x7f6658003890 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.413+0000 7f6676ffd700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f66580062c0 0x7f6658008750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f66777fe700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 0x7f6658005cc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f666800cb30 tx=0x7f666800ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 client.?.objecter ms_handle_connect 0x7f6658003890 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f666800acc0 con 0x7f6658003890 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:18:18.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: dump: 2024-04-20T02:18:18.423 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.423 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.423 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.423 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: _finish_auth 0 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:18:18.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666800ae20 con 0x7f6658003890 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6668011860 con 0x7f6658003890 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 5 monclient: authenticate success, global_id 23303 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 10 monclient: _renew_subs 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6678124fe0 con 0x7f6658003890 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f6675ffb700 4 set_mon_vals no callback set 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f665ffff700 10 monclient: dump: 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.425 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 10 client.23303.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 10 monclient: _renew_subs 2024-04-20T02:18:18.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6678125420 con 0x7f6658003890 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.414+0000 7f667dbba700 1 librados: init done 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f667dbba700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 0x7f66781ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667806f0e0 con 0x7f667806aac0 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f66680113f0 con 0x7f6658003890 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 0x7f66781ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient: get_auth_request con 0x7f667806aac0 auth_method 0 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f665ffff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f666403a1c0 0x7f666403c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 7229+0+0 (secure 0 0 0) 0x7f666804a770 con 0x7f6658003890 2024-04-20T02:18:18.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f665ffff700 10 client.23303.objecter ms_dispatch 0x7f6678001550 osd_map(94..94 src has 1..94) v4 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f665ffff700 3 client.23303.objecter handle_osd_map got epochs [94,94] > 0 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f665ffff700 3 client.23303.objecter handle_osd_map decoding full epoch 94 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.415+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f6677fff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f666403a1c0 0x7f666403c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f6677fff700 10 monclient: get_auth_request con 0x7f666403a1c0 auth_method 0 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 20 client.23303.objecter dump_active .. 0 homeless 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f6676ffd700 10 monclient(hunting): handle_auth_done global_id 25939 payload 824 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f6676ffd700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66780a0020 con 0x7f667806aac0 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 client.23303.objecter ms_handle_connect 0x7f667806aac0 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f6676ffd700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 0x7f66781ca7c0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f66580037b0 tx=0x7f666c0118b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 client.23303.objecter resend_mon_ops 2024-04-20T02:18:18.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f666c0044a0 con 0x7f667806aac0 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f6677fff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f666403a1c0 0x7f666403c680 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7f66781ca790 tx=0x7f6670009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 monclient: dump: 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.429 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 client.23303.objecter ms_handle_connect 0x7f666403a1c0 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f666c004030 con 0x7f667806aac0 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 msgr2=0x7f66781ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 0x7f66781ca7c0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f66580037b0 tx=0x7f666c0118b0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 msgr2=0x7f66781ca7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:18.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.416+0000 7f665ffff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 0x7f66781ca7c0 secure :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f66580037b0 tx=0x7f666c0118b0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f667dbba700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:18:18.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 0x7f667811f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f667dbba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6678126d50 con 0x7f66781cad00 2024-04-20T02:18:18.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 0x7f667811f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient: get_auth_request con 0x7f66781cad00 auth_method 0 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 10 monclient(hunting): handle_auth_done global_id 25948 payload 824 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.426+0000 7f6676ffd700 1 -- 172.21.15.47:0/2351669057 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66781267b0 con 0x7f66781cad00 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 client.23303.objecter ms_handle_connect 0x7f66781cad00 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 client.23303.objecter resend_mon_ops 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f6676ffd700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 0x7f667811f530 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f666c004000 tx=0x7f666c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f666c015e30 con 0x7f66781cad00 2024-04-20T02:18:18.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 monclient: dump: 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.435 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9311 (secure 0 0 0) 0x7f666c0159c0 con 0x7f66781cad00 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 msgr2=0x7f667811f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 0x7f667811f530 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f666c004000 tx=0x7f666c004750 comp rx=0 tx=0).stop 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 msgr2=0x7f667811f530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f665ffff700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 0x7f667811f530 secure :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f666c004000 tx=0x7f666c004750 comp rx=0 tx=0).stop 2024-04-20T02:18:18.436 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 22, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:18:18.437 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:18:18.438 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:18.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:18:18.440 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:18.441 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:18:18.443 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:18:18.444 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.446 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.449 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:18:18.452 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.455 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.456 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:18:18.457 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:18.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f667dbba700 10 librados: watch_flush enter 2024-04-20T02:18:18.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f667dbba700 10 librados: watch_flush exit 2024-04-20T02:18:18.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f667dbba700 20 client.23303.objecter shutdown clearing up homeless session... 2024-04-20T02:18:18.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.427+0000 7f667dbba700 10 client.23303.objecter successfully canceled tick 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f666403a1c0 msgr2=0x7f666403c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f666403a1c0 0x7f666403c680 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7f66781ca790 tx=0x7f6670009000 comp rx=0 tx=0).stop 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 10 monclient: shutdown 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 msgr2=0x7f6658005cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 0x7f6658005cc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f666800cb30 tx=0x7f666800ce40 comp rx=0 tx=0).stop 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 shutdown_connections 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f666403a1c0 0x7f666403c680 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6658003890 0x7f6658005cc0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f667806aac0 0x7f66781ca7c0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 --2- 172.21.15.47:0/2351669057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66781cad00 0x7f667811f530 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 >> 172.21.15.47:0/2351669057 conn(0x7f66780a39d0 msgr2=0x7f6678066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:18.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 shutdown_connections 2024-04-20T02:18:18.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 -- 172.21.15.47:0/2351669057 wait complete. 2024-04-20T02:18:18.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.428+0000 7f667dbba700 1 librados: shutdown 2024-04-20T02:18:18.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:18:18.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon g 2024-04-20T02:18:18.461 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:18:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.604+0000 7f3e930f6700 10 monclient: get_monmap_and_config 2024-04-20T02:18:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.604+0000 7f3e930f6700 10 monclient: build_initial_monmap 2024-04-20T02:18:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.604+0000 7f3e930f6700 10 monclient: monmap: 2024-04-20T02:18:18.603 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:18:18.605063+0000 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:18:18.605063+0000 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:18:18.604 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:18:18.605 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.605+0000 7f3e930f6700 1 Processor -- start 2024-04-20T02:18:18.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.605+0000 7f3e930f6700 1 -- start start 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.605+0000 7f3e930f6700 10 monclient: init 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient: _add_conns ranks=[9,4,1,3,2,8,5,0,7,10,6] 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3e8c09f690 0x7f3e8c0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): picked mon.noname-a con 0x7f3e8c09f690 addr v2:172.21.15.47:3300/0 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3e8c0a3ea0 0x7f3e8c0a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): picked mon.noname-c con 0x7f3e8c0a3ea0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3e8c1357f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e8bfff700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3e8c0a3ea0 msgr2=0x7f3e8c0a4320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c0751e0 con 0x7f3e8c1357f0 2024-04-20T02:18:18.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c070ce0 con 0x7f3e8c09f690 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c072780 con 0x7f3e8c0a3ea0 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e8bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3e8c0a3ea0 0x7f3e8c0a4320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e930f6700 10 monclient(hunting): authenticate will time out at 1778.006836s 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3e8c09f690 0x7f3e8c0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3e8c09f690 0x7f3e8c0a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52940/0 (socket says 172.21.15.47:52940) 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 1 -- 172.21.15.47:0/3661665229 learned_addr learned my addr 172.21.15.47:0/3661665229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 10 monclient(hunting): get_auth_request con 0x7f3e8c09f690 auth_method 0 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.606+0000 7f3e90e92700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 10 monclient(hunting): handle_auth_done global_id 34506 payload 274 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 1 -- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3e8c0a3ea0 msgr2=0x7f3e8c0a4320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 1 --2- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3e8c0a3ea0 0x7f3e8c0a4320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 1 -- 172.21.15.47:0/3661665229 >> v1:172.21.15.179:6789/0 conn(0x7f3e8c1357f0 legacy=0x7f3e8c135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 10 monclient: _finish_hunting 0 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 1 monclient: found mon.noname-a 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 1 -- 172.21.15.47:0/3661665229 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c09fa70 con 0x7f3e8c09f690 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e90e92700 1 --2- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e8c09f690 0x7f3e8c0a3960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0098f0 tx=0x7f3e7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b090f5cd4e7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 1 -- 172.21.15.47:0/3661665229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e7c00bb10 con 0x7f3e8c09f690 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:18:18.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: dump: 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.620 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e930f6700 5 monclient: authenticate success, global_id 34506 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 1 -- 172.21.15.47:0/3661665229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c00bc70 con 0x7f3e8c09f690 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e930f6700 10 monclient: get_monmap_and_config success 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 1 -- 172.21.15.47:0/3661665229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e7c00bde0 con 0x7f3e8c09f690 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e930f6700 4 set_mon_vals no callback set 2024-04-20T02:18:18.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.607+0000 7f3e8b7fe700 10 monclient: dump: 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.622 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.623 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 10 monclient: shutdown 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 -- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e8c09f690 msgr2=0x7f3e8c0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 --2- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e8c09f690 0x7f3e8c0a3960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0098f0 tx=0x7f3e7c009c00 comp rx=0 tx=0).stop 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 -- 172.21.15.47:0/3661665229 shutdown_connections 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 --2- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e8c09f690 0x7f3e8c0a3960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 --2- 172.21.15.47:0/3661665229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3e8c0a3ea0 0x7f3e8c0a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 -- 172.21.15.47:0/3661665229 >> 172.21.15.47:0/3661665229 conn(0x7f3e8c0a8610 msgr2=0x7f3e8c0a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 -- 172.21.15.47:0/3661665229 shutdown_connections 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 1 -- 172.21.15.47:0/3661665229 wait complete. 2024-04-20T02:18:18.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 10 monclient: build_initial_monmap 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.608+0000 7f3e930f6700 10 monclient: monmap: 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:18:18.625 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 librados: starting msgr at 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 librados: starting objecter 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 Processor -- start 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 -- start start 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 librados: setting wanted keys 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 librados: calling monclient init 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient: init 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient: _add_conns ranks=[1,8,3,4,6,2,5,7,0] 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e8c09f690 0x7f3e8c1c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient(hunting): picked mon.noname-f con 0x7f3e8c09f690 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3e8c0a3ea0 0x7f3e8c134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient(hunting): picked mon.noname-e con 0x7f3e8c0a3ea0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1357f0 0x7f3e8c135240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient(hunting): picked mon.noname-g con 0x7f3e8c1357f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.609+0000 7f3e930f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c124490 con 0x7f3e8c09f690 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e930f6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c09edc0 con 0x7f3e8c1357f0 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e930f6700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c06e2b0 con 0x7f3e8c0a3ea0 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e930f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e930f6700 10 monclient(hunting): authenticate will time out at 1778.010021s 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3e8c0a3ea0 0x7f3e8c134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3e8c0a3ea0 0x7f3e8c134d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:39744/0 (socket says 172.21.15.47:39744) 2024-04-20T02:18:18.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e8c09f690 0x7f3e8c1c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 1 -- 172.21.15.47:0/160891933 learned_addr learned my addr 172.21.15.47:0/160891933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): get_auth_request con 0x7f3e8c0a3ea0 auth_method 0 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): get_auth_request con 0x7f3e8c09f690 auth_method 0 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1357f0 0x7f3e8c135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): get_auth_request con 0x7f3e8c1357f0 auth_method 0 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e91693700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient(hunting): handle_auth_done global_id 25966 payload 824 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3e8c0a3ea0 msgr2=0x7f3e8c134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3e8c0a3ea0 0x7f3e8c134d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1357f0 msgr2=0x7f3e8c135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1357f0 0x7f3e8c135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient: _finish_hunting 0 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 monclient: found mon.noname-f 2024-04-20T02:18:18.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c009550 con 0x7f3e8c09f690 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e90e92700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e8c09f690 0x7f3e8c1c8db0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c016000 tx=0x7f3e7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e897fa700 10 client.?.objecter ms_handle_connect 0x7f3e8c09f690 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e897fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e7c0044b0 con 0x7f3e8c09f690 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e897fa700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.610+0000 7f3e897fa700 10 monclient: dump: 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 10 monclient: _finish_auth 0 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 10 monclient: _check_auth_tickets 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 5 monclient: authenticate success, global_id 25966 2024-04-20T02:18:18.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c01f070 con 0x7f3e8c09f690 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 monclient: _renew_subs 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c123e40 con 0x7f3e8c09f690 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e7c003b10 con 0x7f3e8c09f690 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e8affd700 4 set_mon_vals no callback set 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 client.25966.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 10 monclient: dump: 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.632 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 monclient: _renew_subs 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c1c9830 con 0x7f3e8c09f690 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 1 librados: init done 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 0x7f3e8c1cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c068380 con 0x7f3e8c06aac0 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.611+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3e7c021030 con 0x7f3e8c09f690 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 0x7f3e8c1cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient: get_auth_request con 0x7f3e8c06aac0 auth_method 0 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 0x7f3e6c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 7229+0+0 (secure 0 0 0) 0x7f3e7c04bde0 con 0x7f3e8c09f690 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 client.25966.objecter ms_dispatch 0x7f3e8c001550 osd_map(94..94 src has 1..94) v4 2024-04-20T02:18:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 3 client.25966.objecter handle_osd_map got epochs [94,94] > 0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 3 client.25966.objecter handle_osd_map decoding full epoch 94 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_done global_id 21162 payload 824 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e8c09e030 con 0x7f3e8c06aac0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 0x7f3e6c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 10 monclient: get_auth_request con 0x7f3e6c03a2f0 auth_method 0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 20 client.25966.objecter dump_active .. 0 homeless 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 client.25966.objecter ms_handle_connect 0x7f3e8c06aac0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 client.25966.objecter resend_mon_ops 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e8bfff700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 0x7f3e8c1cc2d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c1be960 tx=0x7f3e74009a80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e74012600 con 0x7f3e8c06aac0 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 monclient: dump: 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.635 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 10 client.25966.objecter ms_handle_connect 0x7f3e6c03a2f0 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 0x7f3e6c03c7b0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f3e84002200 tx=0x7f3e84009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 msgr2=0x7f3e6c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 msgr2=0x7f3e6c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 0x7f3e6c03c7b0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f3e84002200 tx=0x7f3e84009040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e91693700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 0x7f3e6c03c7b0 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f3e84002200 tx=0x7f3e84009040 comp rx=0 tx=0).stop 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 msgr2=0x7f3e6c03c7b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.612+0000 7f3e897fa700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e6c03a2f0 0x7f3e6c03c7b0 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3e74005aa0 con 0x7f3e8c06aac0 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 msgr2=0x7f3e8c1cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 0x7f3e8c1cc2d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c1be960 tx=0x7f3e74009a80 comp rx=0 tx=0).stop 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 msgr2=0x7f3e8c1cc2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:18.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.613+0000 7f3e897fa700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c06aac0 0x7f3e8c1cc2d0 secure :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c1be960 tx=0x7f3e74009a80 comp rx=0 tx=0).stop 2024-04-20T02:18:18.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.620+0000 7f3e930f6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:18:18.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.620+0000 7f3e930f6700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 0x7f3e8c1ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:18:18.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.620+0000 7f3e930f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.620+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c06a400 con 0x7f3e8c1cc810 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 0x7f3e8c1ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient: get_auth_request con 0x7f3e8c1cc810 auth_method 0 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 10 monclient(hunting): handle_auth_done global_id 21171 payload 824 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 1 -- 172.21.15.47:0/160891933 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3e8c0721c0 con 0x7f3e8c1cc810 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e8bfff700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 0x7f3e8c1ccc20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005a00 tx=0x7f3e74005a30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e897fa700 10 client.25966.objecter ms_handle_connect 0x7f3e8c1cc810 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e897fa700 10 client.25966.objecter resend_mon_ops 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e74023070 con 0x7f3e8c1cc810 2024-04-20T02:18:18.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e897fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.621+0000 7f3e897fa700 10 monclient: dump: 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:18:18.641 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9479 (secure 0 0 0) 0x7f3e74012e00 con 0x7f3e8c1cc810 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 msgr2=0x7f3e8c1ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 0x7f3e8c1ccc20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005a00 tx=0x7f3e74005a30 comp rx=0 tx=0).stop 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 msgr2=0x7f3e8c1ccc20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e897fa700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e8c1cc810 0x7f3e8c1ccc20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e91693700 1 -- 172.21.15.47:0/160891933 reap_dead start 2024-04-20T02:18:18.642 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 22, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:18:18.643 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:18.644 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:18:18.645 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:18:18.646 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:18:18.648 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:18:18.651 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.654 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.657 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.660 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:18.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.662 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:18:18.663 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 10 librados: watch_flush enter 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 10 librados: watch_flush exit 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 20 client.25966.objecter shutdown clearing up homeless session... 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 10 client.25966.objecter successfully canceled tick 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 10 monclient: shutdown 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e8c09f690 msgr2=0x7f3e8c1c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e8c09f690 0x7f3e8c1c8db0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c016000 tx=0x7f3e7c004750 comp rx=0 tx=0).stop 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 shutdown_connections 2024-04-20T02:18:18.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 --2- 172.21.15.47:0/160891933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e8c09f690 0x7f3e8c1c8db0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:18:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 >> 172.21.15.47:0/160891933 conn(0x7f3e8c0a8610 msgr2=0x7f3e8c09fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:18:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 shutdown_connections 2024-04-20T02:18:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 -- 172.21.15.47:0/160891933 wait complete. 2024-04-20T02:18:18.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:18:18.622+0000 7f3e930f6700 1 librados: shutdown 2024-04-20T02:18:18.666 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:18:18.666 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:18:25.063 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosS api_aio: Running main() from gmock_main.cc 2024-04-20T02:18:25.063 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-04-20T02:18:25.063 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [----------] Global test environment set-up. 2024-04-20T02:18:25.063 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-04-20T02:18:25.063 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-04-20T02:18:25.063 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.TooBig (3018 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (6033 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (2926 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3020 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (2983 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3095 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3039 ms) 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-04-20T02:18:25.064 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2879 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3117 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2938 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3095 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2932 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2973 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3182 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.Flush (2793 ms) 2024-04-20T02:18:25.065 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3001 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2995 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3001 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3026 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (6995 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3006 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3065 ms) 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-04-20T02:18:25.066 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3003 ms) 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (18103 ms) 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (5813 ms) 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2999 ms) 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [----------] 26 tests from LibRadosAio (103031 ms total) 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: 2024-04-20T02:18:25.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-04-20T02:18:33.073 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite8017 ms total) 2024-04-20T02:18:33.073 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (99 ms) 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (99 ms total) 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (20 ms) 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (5 ms) 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (25 ms total) 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:18:33.074 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (20 ms) 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (23 ms total) 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (204 ms) 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (2 ms) 2024-04-20T02:18:33.075 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (206 ms total) 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1026 ms) 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1026 ms total) 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (110960 ms total) 2024-04-20T02:18:33.076 INFO:tasks.workunit.client.0.smithi179.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout:napshotsECPP.SnapRemovePP (2036 ms) 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2031 ms) 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1911 ms) 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9003 ms total) 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4032 ms) 2024-04-20T02:18:34.092 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3994 ms) 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3976 ms) 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12002 ms total) 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (111950 ms total) 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-04-20T02:18:34.093 INFO:tasks.workunit.client.0.smithi179.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-04-20T02:18:34.094 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.094 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46889 2024-04-20T02:18:34.094 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46889 2024-04-20T02:18:34.094 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46854 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46854 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46838 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46838 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47077 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47077 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47061 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47061 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47253 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47253 2024-04-20T02:18:34.095 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:18:34.096 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46806 2024-04-20T02:18:34.096 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46806 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (3049 ms) 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (21076 ms) 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6002 ms) 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-04-20T02:18:49.067 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2890 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3086 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2991 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2938 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3194 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2797 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3009 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2995 ms) 2024-04-20T02:18:49.068 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2995 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (10017 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2998 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3073 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3004 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (18072 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2840 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (6005 ms) 2024-04-20T02:18:49.069 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3024 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2983 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3017 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2997 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2985 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3008 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3004 ms) 2024-04-20T02:18:49.070 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-04-20T02:19:48.667 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:19:48.667 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-04-20T02:19:48.667 INFO:tasks.ceph.mon.h:Restarting daemon 2024-04-20T02:19:48.668 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-04-20T02:19:48.671 INFO:tasks.ceph.mon.h:Started 2024-04-20T02:19:48.671 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:19:48.671 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-04-20T02:19:48.672 INFO:tasks.ceph.mon.i:Restarting daemon 2024-04-20T02:19:48.672 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-04-20T02:19:48.674 INFO:tasks.ceph.mon.i:Started 2024-04-20T02:19:48.674 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-04-20T02:19:48.674 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-04-20T02:19:48.674 INFO:tasks.ceph.mon.d:Restarting daemon 2024-04-20T02:19:48.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-04-20T02:19:48.677 INFO:tasks.ceph.mon.d:Started 2024-04-20T02:19:48.677 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:19:48.677 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-20T02:19:48.677 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T02:19:48.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T02:19:48.679 INFO:tasks.ceph.mon.c:Started 2024-04-20T02:19:48.679 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:19:48.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:19:48.798 INFO:tasks.ceph.mon.h.smithi179.stderr:2024-04-20T02:19:48.798+0000 7f13d0a57cc0 -1 mon.h@-1(???).log v0 log_external_backlog rewinding external_log_to from 104 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:19:48.817 INFO:tasks.ceph.mon.d.smithi047.stderr:2024-04-20T02:19:48.819+0000 7f8427595cc0 -1 mon.d@-1(???).log v0 log_external_backlog rewinding external_log_to from 104 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:19:48.818 INFO:tasks.ceph.mon.c.smithi047.stderr:2024-04-20T02:19:48.819+0000 7f8632a63cc0 -1 mon.c@-1(???).log v0 log_external_backlog rewinding external_log_to from 104 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.860+0000 7f5ca271d700 10 monclient: get_monmap_and_config 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.860+0000 7f5ca271d700 10 monclient: build_initial_monmap 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.860+0000 7f5ca271d700 10 monclient: monmap: 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:19:48.861031+0000 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:19:48.861031+0000 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:19:48.860 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:19:48.861 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.862+0000 7f5ca271d700 1 Processor -- start 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.862+0000 7f5ca271d700 1 -- start start 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.862+0000 7f5ca271d700 10 monclient: init 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient: _reopen_session rank -1 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient: _add_conns ranks=[4,7,0,9,3,2,1,10,5,8,6] 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 0x7f5c9c0a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): picked mon.noname-c con 0x7f5c9c0a4c80 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:19:48.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 0x7f5c9c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): picked mon.noname-i con 0x7f5c9c0a0d20 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:19:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c9c0a55a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:19:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c0751e0 con 0x7f5c9c0a55a0 2024-04-20T02:19:48.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0a0000 con 0x7f5c9c0a4c80 2024-04-20T02:19:48.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0a6a10 con 0x7f5c9c0a0d20 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca271d700 10 monclient(hunting): authenticate will time out at 1868.263714s 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c93fff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 0x7f5c9c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c9bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 0x7f5c9c0a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca0cba700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5c9c0a55a0 0x7f5c9c137a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53292/0 (socket says 172.21.15.47:53292) 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c93fff700 10 monclient(hunting): get_auth_request con 0x7f5c9c0a0d20 auth_method 0 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c93fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c93fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5ca0cba700 1 -- 172.21.15.47:0/2552157851 learned_addr learned my addr 172.21.15.47:0/2552157851 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c93fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c9bfff700 10 monclient(hunting): get_auth_request con 0x7f5c9c0a4c80 auth_method 0 2024-04-20T02:19:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c9bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c9bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.863+0000 7f5c9bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c93fff700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 msgr2=0x7f5c9c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c93fff700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 msgr2=0x7f5c9c0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 0x7f5c9c0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 0x7f5c9c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9bfff700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 msgr2=0x7f5c9c0a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9bfff700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 msgr2=0x7f5c9c0a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9bfff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 0x7f5c9c0a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9bfff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 0x7f5c9c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233589875 0 0) 0x7f5c9c0751e0 con 0x7f5c9c0a55a0 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient(hunting): _negotiate 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient(hunting): my global_id is 34803 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c880035b0 con 0x7f5c9c0a55a0 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150170241 0 0) 0x7f5c880035b0 con 0x7f5c9c0a55a0 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 msgr2=0x7f5c9c0a5060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5c9c0a0d20 0x7f5c9c0a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 msgr2=0x7f5c9c0a07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c9c0a4c80 0x7f5c9c0a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c09e440 con 0x7f5c9c0a55a0 2024-04-20T02:19:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f5c94001c60 con 0x7f5c9c0a55a0 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: dump: 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:48.879 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 >> v1:172.21.15.47:6789/0 conn(0x7f5c9c0a55a0 legacy=0x7f5c9c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 10 monclient: _add_conns ranks=[6,8,3,5,1,0,7,2,4] 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.864+0000 7f5c9b7fe700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5c88004250 0x7f5c88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): picked mon.d con 0x7f5c88004250 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5c88004de0 0x7f5c88005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): picked mon.e con 0x7f5c88004de0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c88009ba0 0x7f5c8800c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): picked mon.g con 0x7f5c88009ba0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0a0000 con 0x7f5c88009ba0 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0a6a10 con 0x7f5c88004250 2024-04-20T02:19:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8800c940 con 0x7f5c88004de0 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9b7fe700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5c88004de0 0x7f5c88005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): get_auth_request con 0x7f5c88004de0 auth_method 0 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c88009ba0 0x7f5c8800c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): get_auth_request con 0x7f5c88009ba0 auth_method 0 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9bfff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5c88004250 0x7f5c88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9bfff700 10 monclient(hunting): get_auth_request con 0x7f5c88004250 auth_method 0 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c9bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient(hunting): handle_auth_done global_id 34803 payload 293 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5c88004250 msgr2=0x7f5c88004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5c88004250 0x7f5c88004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c88009ba0 msgr2=0x7f5c8800c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c88009ba0 0x7f5c8800c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient: _finish_hunting 0 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 monclient: found mon.e 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca0cba700 1 -- 172.21.15.47:0/2552157851 reap_dead start 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 1 -- 172.21.15.47:0/2552157851 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c880035b0 con 0x7f5c88004de0 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient: _finish_auth 0 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 10 monclient: _check_auth_tickets 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5c93fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca271d700 5 monclient: authenticate success, global_id 34803 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.865+0000 7f5ca271d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:19:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5c88004de0 0x7f5c88005240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009840 tx=0x7f5c88005880 comp rx=0 tx=0).ready entity=mon.8 client_cookie=570dc52db5e2e931 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c8400c070 con 0x7f5c88004de0 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: dump: 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c84004030 con 0x7f5c88004de0 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 1 -- 172.21.15.47:0/2552157851 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c840036a0 con 0x7f5c88004de0 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5c9b7fe700 10 monclient: dump: 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5ca271d700 10 monclient: get_monmap_and_config success 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.866+0000 7f5ca271d700 4 set_mon_vals no callback set 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.867+0000 7f5ca271d700 10 monclient: shutdown 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.867+0000 7f5ca271d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.867+0000 7f5ca271d700 1 -- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5c88004de0 msgr2=0x7f5c88005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.867+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5c88004de0 0x7f5c88005240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009840 tx=0x7f5c88005880 comp rx=0 tx=0).stop 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.869+0000 7f5ca271d700 1 -- 172.21.15.47:0/2552157851 shutdown_connections 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.869+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2552157851 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5c88004de0 0x7f5c88005240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.869+0000 7f5ca271d700 1 -- 172.21.15.47:0/2552157851 >> 172.21.15.47:0/2552157851 conn(0x7f5c9c0aa280 msgr2=0x7f5c9c0aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.869+0000 7f5ca271d700 1 -- 172.21.15.47:0/2552157851 shutdown_connections 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.869+0000 7f5ca271d700 1 -- 172.21.15.47:0/2552157851 wait complete. 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.870+0000 7f5ca271d700 10 monclient: build_initial_monmap 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 10 monclient: monmap: 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:19:48.887 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:19:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 1 librados: starting msgr at 2024-04-20T02:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 1 librados: starting objecter 2024-04-20T02:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 1 Processor -- start 2024-04-20T02:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 1 -- start start 2024-04-20T02:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 1 librados: setting wanted keys 2024-04-20T02:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 1 librados: calling monclient init 2024-04-20T02:19:48.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.871+0000 7f5ca271d700 10 monclient: init 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient: _reopen_session rank -1 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient: _add_conns ranks=[2,0,3,6,8,1,5,4,7] 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5c88004de0 0x7f5c9c1c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): picked mon.noname-b con 0x7f5c88004de0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c9c1bbb10 0x7f5c9c1bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): picked mon.noname-a con 0x7f5c9c1bbb10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:19:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 0x7f5c9c1a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): picked mon.noname-g con 0x7f5c9c1a55e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1a8a00 con 0x7f5c9c1bbb10 2024-04-20T02:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1a78d0 con 0x7f5c88004de0 2024-04-20T02:19:48.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1c8c10 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca271d700 10 monclient(hunting): authenticate will time out at 1868.272685s 2024-04-20T02:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.872+0000 7f5ca0cba700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 0x7f5c9c1a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 0x7f5c9c1a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:34210/0 (socket says 172.21.15.47:34210) 2024-04-20T02:19:48.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 -- 172.21.15.47:0/2116946568 learned_addr learned my addr 172.21.15.47:0/2116946568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): get_auth_request con 0x7f5c9c1a55e0 auth_method 0 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c9c1bbb10 0x7f5c9c1bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): get_auth_request con 0x7f5c9c1bbb10 auth_method 0 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:48.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c93fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:19:48.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient(hunting): handle_auth_done global_id 21522 payload 824 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 -- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5c88004de0 msgr2=0x7f5c9c1c7f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5c88004de0 0x7f5c9c1c7f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 -- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c9c1bbb10 msgr2=0x7f5c9c1bbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c9c1bbb10 0x7f5c9c1bbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient: _finish_hunting 0 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 monclient: found mon.noname-g 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:19:48.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 -- 172.21.15.47:0/2116946568 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c1bc490 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca0cba700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 0x7f5c9c1a5a40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c09f620 tx=0x7f5c9400f6e0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 client.?.objecter ms_handle_connect 0x7f5c9c1a55e0 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c9400fe20 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 monclient: dump: 2024-04-20T02:19:48.897 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:48.898 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 monclient: _finish_auth 0 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 10 monclient: _check_auth_tickets 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94013860 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca271d700 5 monclient: authenticate success, global_id 21522 2024-04-20T02:19:48.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca271d700 10 monclient: _renew_subs 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca271d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.873+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c09dd20 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5c997fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c94013e40 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5c997fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5c997fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5c997fa700 10 monclient: dump: 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:48.900 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5ca271d700 10 client.21522.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5ca271d700 10 monclient: _renew_subs 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5ca271d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c1a92a0 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5ca271d700 1 librados: init done 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.874+0000 7f5c9affd700 4 set_mon_vals no callback set 2024-04-20T02:19:48.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5c940139d0 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c7c03a2f0 0x7f5c7c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7291+0+0 (secure 0 0 0) 0x7f5c9404b4f0 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 10 client.21522.objecter ms_dispatch 0x7f5c9c001550 osd_map(184..184 src has 1..184) v4 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 3 client.21522.objecter handle_osd_map got epochs [184,184] > 0 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 3 client.21522.objecter handle_osd_map decoding full epoch 184 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5ca271d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5ca271d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5ca271d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c997fa700 20 client.21522.objecter dump_active .. 0 homeless 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c1a8c40 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c9bfff700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c7c03a2f0 0x7f5c7c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:48.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.875+0000 7f5c9bfff700 10 monclient: get_auth_request con 0x7f5c7c03a2f0 auth_method 0 2024-04-20T02:19:48.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.876+0000 7f5c9bfff700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c7c03a2f0 0x7f5c7c03c7b0 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c068e30 tx=0x7f5c8c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:19:48.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.876+0000 7f5c997fa700 10 client.21522.objecter ms_handle_connect 0x7f5c7c03a2f0 2024-04-20T02:19:48.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.881+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f5c94013c80 con 0x7f5c9c1a55e0 2024-04-20T02:19:48.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.881+0000 7f5c997fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:19:48.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:48.881+0000 7f5c997fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:19:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.217+0000 7f5ca271d700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:19:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.217+0000 7f5ca271d700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:19:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.217+0000 7f5ca271d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:19:49.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.217+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5c9c06c2d0 con 0x7f5c9c1a55e0 2024-04-20T02:19:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.218+0000 7f5c997fa700 1 -- 172.21.15.47:0/2116946568 <== mon.3 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3152 (secure 0 0 0) 0x7f5c9401a730 con 0x7f5c9c1a55e0 2024-04-20T02:19:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.218+0000 7f5c997fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:19:49.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.218+0000 7f5c997fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:19:49.218 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":22,"quorum":[0,1,2,3,8],"quorum_names":["a","f","b","g","e"],"quorum_leader_name":"f","quorum_age":95,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:19:49.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 10 librados: watch_flush enter 2024-04-20T02:19:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 10 librados: watch_flush exit 2024-04-20T02:19:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 20 client.21522.objecter shutdown clearing up homeless session... 2024-04-20T02:19:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 10 client.21522.objecter successfully canceled tick 2024-04-20T02:19:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c7c03a2f0 msgr2=0x7f5c7c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c7c03a2f0 0x7f5c7c03c7b0 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c068e30 tx=0x7f5c8c008040 comp rx=0 tx=0).stop 2024-04-20T02:19:49.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 10 monclient: shutdown 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 msgr2=0x7f5c9c1a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 0x7f5c9c1a5a40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c09f620 tx=0x7f5c9400f6e0 comp rx=0 tx=0).stop 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 shutdown_connections 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c7c03a2f0 0x7f5c7c03c7b0 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c9c1a55e0 0x7f5c9c1a5a40 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5c88004de0 0x7f5c9c1c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:49.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 --2- 172.21.15.47:0/2116946568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c9c1bbb10 0x7f5c9c1bbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 >> 172.21.15.47:0/2116946568 conn(0x7f5c9c0aa280 msgr2=0x7f5c9c0a25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:19:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 shutdown_connections 2024-04-20T02:19:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 -- 172.21.15.47:0/2116946568 wait complete. 2024-04-20T02:19:49.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:49.220+0000 7f5ca271d700 1 librados: shutdown 2024-04-20T02:19:52.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:19:52.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.378+0000 7fa14a961700 10 monclient: get_monmap_and_config 2024-04-20T02:19:52.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.378+0000 7fa14a961700 10 monclient: build_initial_monmap 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.378+0000 7fa14a961700 10 monclient: monmap: 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:19:52.380043+0000 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:19:52.380043+0000 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:19:52.379 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:19:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:19:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:19:52.380 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:19:52.380 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.379+0000 7fa14a961700 1 Processor -- start 2024-04-20T02:19:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.379+0000 7fa14a961700 1 -- start start 2024-04-20T02:19:52.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.379+0000 7fa14a961700 10 monclient: init 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient: _reopen_session rank -1 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient: _add_conns ranks=[7,4,2,3,5,0,6,10,8,1,9] 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): picked mon.noname-i con 0x7fa1440a4c80 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): picked mon.noname-c con 0x7fa1440a0d20 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): picked mon.noname-b con 0x7fa1440a55a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440a0000 con 0x7fa1440a55a0 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440a6a10 con 0x7fa1440a0d20 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa14409e440 con 0x7fa1440a4c80 2024-04-20T02:19:52.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa14a961700 10 monclient(hunting): authenticate will time out at 1871.781358s 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:38664/0 (socket says 172.21.15.47:38664) 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 learned_addr learned my addr 172.21.15.47:0/2139515250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47440/0 (socket says 172.21.15.47:47440) 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 10 monclient(hunting): get_auth_request con 0x7fa1440a0d20 auth_method 0 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 10 monclient(hunting): get_auth_request con 0x7fa1440a55a0 auth_method 0 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 10 monclient(hunting): get_auth_request con 0x7fa1440a4c80 auth_method 0 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.380+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 10 monclient(hunting): get_auth_request con 0x7fa1440a0d20 auth_method 0 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 10 monclient(hunting): get_auth_request con 0x7fa1440a55a0 auth_method 0 2024-04-20T02:19:52.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 10 monclient(hunting): get_auth_request con 0x7fa1440a4c80 auth_method 0 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:19:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:19:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.581+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:19:52.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 10 monclient(hunting): get_auth_request con 0x7fa1440a0d20 auth_method 0 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 10 monclient(hunting): get_auth_request con 0x7fa1440a55a0 auth_method 0 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:52.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 10 monclient(hunting): get_auth_request con 0x7fa1440a4c80 auth_method 0 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:52.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:52.982+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 10 monclient(hunting): get_auth_request con 0x7fa1440a55a0 auth_method 0 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 10 monclient(hunting): get_auth_request con 0x7fa1440a0d20 auth_method 0 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:53.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 10 monclient(hunting): get_auth_request con 0x7fa1440a4c80 auth_method 0 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:53.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:53.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:53.783+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:19:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): tick 2024-04-20T02:19:55.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 monclient(hunting): continuing hunt 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 msgr2=0x7fa1440a07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1440a4c80 0x7fa1440a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 msgr2=0x7fa1440a5060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1440a0d20 0x7fa1440a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 msgr2=0x7fa144137a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa1440a55a0 0x7fa144137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient: _add_conns ranks=[10,8,6,5,3,1,9,0] 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 --2- 172.21.15.47:0/2139515250 >> v2:172.21.15.179:3300/0 conn(0x7fa1300038d0 0x7fa130005ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): picked mon.noname-f con 0x7fa1300038d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa130006220 0x7fa130008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): picked mon.noname-e con 0x7fa130006220 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 0x7fa13000b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): picked mon.noname-d con 0x7fa130008cf0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 -- 172.21.15.47:0/2139515250 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1440a0000 con 0x7fa1300038d0 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 -- 172.21.15.47:0/2139515250 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440a6a10 con 0x7fa130008cf0 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 1 -- 172.21.15.47:0/2139515250 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa14409e440 con 0x7fa130006220 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa141ffb700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa130006220 0x7fa130008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 0x7fa13000b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): get_auth_request con 0x7fa130006220 auth_method 0 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 10 monclient(hunting): get_auth_request con 0x7fa130008cf0 auth_method 0 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa1437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:19:55.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa143fff700 1 --2- 172.21.15.47:0/2139515250 >> v2:172.21.15.179:3300/0 conn(0x7fa1300038d0 0x7fa130005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 msgr2=0x7fa13000b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 msgr2=0x7fa13000b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 0x7fa13000b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa143fff700 10 monclient(hunting): get_auth_request con 0x7fa1300038d0 auth_method 0 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa143fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa143fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa143fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.380+0000 7fa148efe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 0x7fa13000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa143fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa143fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa143fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 10 monclient(hunting): handle_auth_done global_id 34712 payload 274 2024-04-20T02:19:55.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 msgr2=0x7fa13000b1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa130008cf0 0x7fa13000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 >> v2:172.21.15.179:3300/0 conn(0x7fa1300038d0 msgr2=0x7fa130005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> v2:172.21.15.179:3300/0 conn(0x7fa1300038d0 0x7fa130005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 10 monclient: _finish_hunting 0 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 monclient: found mon.noname-e 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa148efe700 1 -- 172.21.15.47:0/2139515250 reap_dead start 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 -- 172.21.15.47:0/2139515250 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1440751e0 con 0x7fa130006220 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa1437fe700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa130006220 0x7fa130008690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1440ab1a0 tx=0x7fa12c010dd0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d8f10c058ed7b91b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:19:55.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 1 -- 172.21.15.47:0/2139515250 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa12c0105d0 con 0x7fa130006220 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: dump: 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:55.395 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: _finish_auth 0 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:19:55.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 1 -- 172.21.15.47:0/2139515250 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c00c040 con 0x7fa130006220 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa14a961700 5 monclient: authenticate success, global_id 34712 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa14a961700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa14a961700 10 monclient: get_monmap_and_config success 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa14a961700 4 set_mon_vals no callback set 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 1 -- 172.21.15.47:0/2139515250 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa12c014c50 con 0x7fa130006220 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.381+0000 7fa142ffd700 10 monclient: dump: 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.397 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:55.398 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 10 monclient: shutdown 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 -- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa130006220 msgr2=0x7fa130008690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa130006220 0x7fa130008690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1440ab1a0 tx=0x7fa12c010dd0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 -- 172.21.15.47:0/2139515250 shutdown_connections 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 --2- 172.21.15.47:0/2139515250 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa130006220 0x7fa130008690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 -- 172.21.15.47:0/2139515250 >> 172.21.15.47:0/2139515250 conn(0x7fa1440aa280 msgr2=0x7fa1440aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 -- 172.21.15.47:0/2139515250 shutdown_connections 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.382+0000 7fa14a961700 1 -- 172.21.15.47:0/2139515250 wait complete. 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.383+0000 7fa14a961700 10 monclient: build_initial_monmap 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.383+0000 7fa14a961700 10 monclient: monmap: 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:19:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:19:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.383+0000 7fa14a961700 1 librados: starting msgr at 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.384+0000 7fa14a961700 1 librados: starting objecter 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.384+0000 7fa14a961700 1 Processor -- start 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.384+0000 7fa14a961700 1 -- start start 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.384+0000 7fa14a961700 1 librados: setting wanted keys 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.384+0000 7fa14a961700 1 librados: calling monclient init 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.384+0000 7fa14a961700 10 monclient: init 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient: _reopen_session rank -1 2024-04-20T02:19:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient: _add_conns ranks=[7,0,3,2,8,6,1,5,4] 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 0x7fa1441cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): picked mon.noname-i con 0x7fa130006220 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 0x7fa1441bce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): picked mon.noname-a con 0x7fa1441bca70 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1441a6640 0x7fa1441a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): picked mon.noname-g con 0x7fa1441a6640 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441a9b90 con 0x7fa1441bca70 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441a8a60 con 0x7fa1441a6640 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa14409ea00 con 0x7fa130006220 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa14a961700 10 monclient(hunting): authenticate will time out at 1874.786155s 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 0x7fa1441bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 0x7fa1441bce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58252/0 (socket says 172.21.15.47:58252) 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 -- 172.21.15.47:0/1038001760 learned_addr learned my addr 172.21.15.47:0/1038001760 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): get_auth_request con 0x7fa1441bca70 auth_method 0 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 0x7fa1441cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1441a6640 0x7fa1441a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 10 monclient(hunting): get_auth_request con 0x7fa130006220 auth_method 0 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): get_auth_request con 0x7fa1441a6640 auth_method 0 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 1 -- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 msgr2=0x7fa1441cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 1 -- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 msgr2=0x7fa1441cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 0x7fa1441cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa143fff700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 0x7fa1441cc580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa148efe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient(hunting): handle_auth_done global_id 34821 payload 824 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 -- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 msgr2=0x7fa1441cc580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 0x7fa1441cc580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 -- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1441a6640 msgr2=0x7fa1441a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1441a6640 0x7fa1441a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient: _finish_hunting 0 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 monclient: found mon.noname-a 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.385+0000 7fa1437fe700 1 -- 172.21.15.47:0/1038001760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c010750 con 0x7fa1441bca70 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa1437fe700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 0x7fa1441bce50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa12c010e10 tx=0x7fa12c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 client.?.objecter ms_handle_connect 0x7fa1441bca70 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa12c006d40 con 0x7fa1441bca70 2024-04-20T02:19:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: dump: 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: _finish_auth 0 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c0105b0 con 0x7fa1441bca70 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 5 monclient: authenticate success, global_id 34821 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa12c014e50 con 0x7fa1441bca70 2024-04-20T02:19:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa140ff9700 10 monclient: dump: 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:19:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa141ffb700 4 set_mon_vals no callback set 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 10 monclient: _renew_subs 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14409dd20 con 0x7fa1441bca70 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 10 client.34821.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 10 monclient: _renew_subs 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1441a8060 con 0x7fa1441bca70 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.386+0000 7fa14a961700 1 librados: init done 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa12c0036a0 con 0x7fa1441bca70 2024-04-20T02:19:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa14a961700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa14a961700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa14a961700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1440a5a20 con 0x7fa1441bca70 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa13003b580 0x7fa13003da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fa12c04c2d0 con 0x7fa1441bca70 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 10 client.34821.objecter ms_dispatch 0x7fa144001550 osd_map(186..186 src has 1..186) v4 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 3 client.34821.objecter handle_osd_map got epochs [186,186] > 0 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 3 client.34821.objecter handle_osd_map decoding full epoch 186 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa140ff9700 20 client.34821.objecter dump_active .. 0 homeless 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa143fff700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa13003b580 0x7fa13003da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.387+0000 7fa143fff700 10 monclient: get_auth_request con 0x7fa13003b580 auth_method 0 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.388+0000 7fa143fff700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa13003b580 0x7fa13003da40 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fa134008010 tx=0x7fa134008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:19:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.388+0000 7fa140ff9700 10 client.34821.objecter ms_handle_connect 0x7fa13003b580 2024-04-20T02:19:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.393+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa12c003950 con 0x7fa1441bca70 2024-04-20T02:19:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.393+0000 7fa140ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:19:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.393+0000 7fa140ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:19:55.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.705+0000 7fa14a961700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:19:55.705 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":24,"quorum":[0,1,2,3,5,8],"quorum_names":["a","f","b","g","h","e"],"quorum_leader_name":"f","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.705+0000 7fa14a961700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.705+0000 7fa14a961700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.705+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa1441a9d90 con 0x7fa1441bca70 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.706+0000 7fa140ff9700 1 -- 172.21.15.47:0/1038001760 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3157 (secure 0 0 0) 0x7fa12c0163d0 con 0x7fa1441bca70 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.706+0000 7fa140ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.706+0000 7fa140ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:19:55.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 10 librados: watch_flush enter 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 10 librados: watch_flush exit 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 20 client.34821.objecter shutdown clearing up homeless session... 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 10 client.34821.objecter successfully canceled tick 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa13003b580 msgr2=0x7fa13003da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa13003b580 0x7fa13003da40 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fa134008010 tx=0x7fa134008040 comp rx=0 tx=0).stop 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 10 monclient: shutdown 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 msgr2=0x7fa1441bce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 0x7fa1441bce50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa12c010e10 tx=0x7fa12c004750 comp rx=0 tx=0).stop 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 shutdown_connections 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa13003b580 0x7fa13003da40 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1441a6640 0x7fa1441a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa130006220 0x7fa1441cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 --2- 172.21.15.47:0/1038001760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1441bca70 0x7fa1441bce50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:19:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 >> 172.21.15.47:0/1038001760 conn(0x7fa1440aa280 msgr2=0x7fa1440a40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:19:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 shutdown_connections 2024-04-20T02:19:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 -- 172.21.15.47:0/1038001760 wait complete. 2024-04-20T02:19:55.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:55.708+0000 7fa14a961700 1 librados: shutdown 2024-04-20T02:19:58.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:19:58.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.867+0000 7f818cf86700 10 monclient: get_monmap_and_config 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.867+0000 7f818cf86700 10 monclient: build_initial_monmap 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.867+0000 7f818cf86700 10 monclient: monmap: 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:19:58.868771+0000 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:19:58.868771+0000 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:19:58.868 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 Processor -- start 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 -- start start 2024-04-20T02:19:58.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient: init 2024-04-20T02:19:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient: _reopen_session rank -1 2024-04-20T02:19:58.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient: _add_conns ranks=[8,7,9,4,5,1,2,6,0,3,10] 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): picked mon.noname-e con 0x7f81880a0950 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): picked mon.noname-i con 0x7f81880a0160 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): picked mon.noname-a con 0x7f81880a2e90 addr v2:172.21.15.47:3300/0 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8188072830 con 0x7f81880a2e90 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f818809e650 con 0x7f81880a0160 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f818809eee0 con 0x7f81880a0950 2024-04-20T02:19:58.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): _renew_subs 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.868+0000 7f818cf86700 10 monclient(hunting): authenticate will time out at 1878.269999s 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:35332/0 (socket says 172.21.15.47:35332) 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 learned_addr learned my addr 172.21.15.47:0/619678359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 10 monclient(hunting): get_auth_request con 0x7f81880a0950 auth_method 0 2024-04-20T02:19:58.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58266/0 (socket says 172.21.15.47:58266) 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 10 monclient(hunting): get_auth_request con 0x7f81880a2e90 auth_method 0 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:58.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 10 monclient(hunting): get_auth_request con 0x7f81880a0160 auth_method 0 2024-04-20T02:19:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:58.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:19:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:58.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:58.869+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:19:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 10 monclient(hunting): get_auth_request con 0x7f81880a2e90 auth_method 0 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 10 monclient(hunting): get_auth_request con 0x7f81880a0950 auth_method 0 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 10 monclient(hunting): get_auth_request con 0x7f81880a0160 auth_method 0 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.070+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 10 monclient(hunting): get_auth_request con 0x7f81880a2e90 auth_method 0 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 10 monclient(hunting): get_auth_request con 0x7f81880a0950 auth_method 0 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:59.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 10 monclient(hunting): get_auth_request con 0x7f81880a0160 auth_method 0 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:19:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:19:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:19:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:19:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:19:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:19:59.471+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 10 monclient(hunting): get_auth_request con 0x7f81880a0950 auth_method 0 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 10 monclient(hunting): get_auth_request con 0x7f81880a2e90 auth_method 0 2024-04-20T02:20:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 10 monclient(hunting): get_auth_request con 0x7f81880a0160 auth_method 0 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:00.272+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:01.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): tick 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 monclient(hunting): continuing hunt 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 msgr2=0x7f81880a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81880a0950 0x7f81880a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 msgr2=0x7f81880a2950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81880a0160 0x7f81880a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 msgr2=0x7f81880a3310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.47:3300/0 conn(0x7f81880a2e90 0x7f81880a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient: _add_conns ranks=[4,2,10,1,5,0,6,3] 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 0x7f8170005d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): picked mon.noname-c con 0x7f81700038b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:01.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81700062e0 0x7f8170008750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): picked mon.noname-b con 0x7f81700062e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.179:3300/0 conn(0x7f8170008e20 0x7f817000b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): picked mon.noname-f con 0x7f8170008e20 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 -- 172.21.15.47:0/619678359 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8188072830 con 0x7f8170008e20 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 -- 172.21.15.47:0/619678359 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818809e650 con 0x7f81700062e0 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 1 -- 172.21.15.47:0/619678359 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f818809eee0 con 0x7f81700038b0 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8177fff700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 0x7f8170005d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81700062e0 0x7f8170008750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 10 monclient(hunting): get_auth_request con 0x7f81700038b0 auth_method 0 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): get_auth_request con 0x7f81700062e0 auth_method 0 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 msgr2=0x7f8170005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 msgr2=0x7f8170005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 0x7f8170005d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f818659c700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 0x7f8170005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8186d9d700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.179:3300/0 conn(0x7f8170008e20 0x7f817000b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8186d9d700 10 monclient(hunting): get_auth_request con 0x7f8170008e20 auth_method 0 2024-04-20T02:20:01.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8186d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8186d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8186d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient(hunting): handle_auth_done global_id 23483 payload 274 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 msgr2=0x7f8170005d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81700038b0 0x7f8170005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 >> v2:172.21.15.179:3300/0 conn(0x7f8170008e20 msgr2=0x7f817000b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> v2:172.21.15.179:3300/0 conn(0x7f8170008e20 0x7f817000b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 monclient: found mon.noname-b 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8186d9d700 1 -- 172.21.15.47:0/619678359 reap_dead start 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.869+0000 7f8185d9b700 1 -- 172.21.15.47:0/619678359 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880751e0 con 0x7f81700062e0 2024-04-20T02:20:01.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f8185d9b700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81700062e0 0x7f8170008750 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f81880a7e20 tx=0x7f817c012dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327424a004512d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 1 -- 172.21.15.47:0/619678359 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f817c014c20 con 0x7f81700062e0 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: dump: 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:01.883 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: _finish_auth 0 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: _check_auth_tickets 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818cf86700 5 monclient: authenticate success, global_id 23483 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 1 -- 172.21.15.47:0/619678359 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00c040 con 0x7f81700062e0 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818cf86700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 1 -- 172.21.15.47:0/619678359 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f817c014de0 con 0x7f81700062e0 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:01.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818cf86700 4 set_mon_vals no callback set 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.870+0000 7f818559a700 10 monclient: dump: 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:01.885 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 10 monclient: shutdown 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 -- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81700062e0 msgr2=0x7f8170008750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81700062e0 0x7f8170008750 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f81880a7e20 tx=0x7f817c012dc0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 -- 172.21.15.47:0/619678359 shutdown_connections 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 --2- 172.21.15.47:0/619678359 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f81700062e0 0x7f8170008750 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 -- 172.21.15.47:0/619678359 >> 172.21.15.47:0/619678359 conn(0x7f81880a7570 msgr2=0x7f81880a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 -- 172.21.15.47:0/619678359 shutdown_connections 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.871+0000 7f818cf86700 1 -- 172.21.15.47:0/619678359 wait complete. 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.872+0000 7f818cf86700 10 monclient: build_initial_monmap 2024-04-20T02:20:01.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.872+0000 7f818cf86700 10 monclient: monmap: 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:01.887 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.872+0000 7f818cf86700 1 librados: starting msgr at 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.872+0000 7f818cf86700 1 librados: starting objecter 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 Processor -- start 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 -- start start 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 librados: setting wanted keys 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 librados: calling monclient init 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient: init 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient: _add_conns ranks=[6,0,7,3,1,8,4,2,5] 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): picked mon.noname-d con 0x7f81700062e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 0x7f81881bd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): picked mon.noname-a con 0x7f81881cc5a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:01.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 0x7f81881a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): picked mon.noname-i con 0x7f81881a66b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81881cc980 con 0x7f81881cc5a0 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f81881a8e40 con 0x7f81700062e0 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f81881a9d30 con 0x7f81881a66b0 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818cf86700 10 monclient(hunting): authenticate will time out at 1881.274711s 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f8185d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 0x7f81881bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:49402/0 (socket says 172.21.15.47:49402) 2024-04-20T02:20:01.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 1 -- 172.21.15.47:0/4187153108 learned_addr learned my addr 172.21.15.47:0/4187153108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f8185d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 0x7f81881bd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58304/0 (socket says 172.21.15.47:58304) 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 10 monclient(hunting): get_auth_request con 0x7f81700062e0 auth_method 0 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f818659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f8185d9b700 10 monclient(hunting): get_auth_request con 0x7f81881cc5a0 auth_method 0 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f8185d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f8185d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.873+0000 7f8185d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f818659c700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 msgr2=0x7f81881bcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f818659c700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 msgr2=0x7f81881bcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:20:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f818659c700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 0x7f81881a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 10 monclient(hunting): get_auth_request con 0x7f81881a66b0 auth_method 0 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f818659c700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 msgr2=0x7f81881a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 msgr2=0x7f81881a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 0x7f81881a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8186d9d700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 0x7f81881a6b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 10 monclient(hunting): handle_auth_done global_id 34830 payload 824 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 msgr2=0x7f81881a6b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 0x7f81881a6b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 msgr2=0x7f81881bcbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 monclient: found mon.noname-a 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 -- 172.21.15.47:0/4187153108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c012740 con 0x7f81881cc5a0 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8185d9b700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 0x7f81881bd130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f817c01e000 tx=0x7f817c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 client.?.objecter ms_handle_connect 0x7f81881cc5a0 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f817c006ab0 con 0x7f81881cc5a0 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: dump: 2024-04-20T02:20:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:01.893 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f818cf86700 5 monclient: authenticate success, global_id 34830 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c02b450 con 0x7f81881cc5a0 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f817c02ba30 con 0x7f81881cc5a0 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8184d99700 4 set_mon_vals no callback set 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.874+0000 7f8176ffd700 10 monclient: dump: 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 10 monclient: _renew_subs 2024-04-20T02:20:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81881bd670 con 0x7f81881cc5a0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 10 client.34830.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 10 monclient: _renew_subs 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81881aa5d0 con 0x7f81881cc5a0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.875+0000 7f818cf86700 1 librados: init done 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f817c02bbb0 con 0x7f81881cc5a0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f818cf86700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f818cf86700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f818cf86700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818809fb60 con 0x7f81881cc5a0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f817003b3f0 0x7f817003d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6488+0+0 (secure 0 0 0) 0x7f817c04c440 con 0x7f81881cc5a0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 10 client.34830.objecter ms_dispatch 0x7f8188001550 osd_map(188..188 src has 1..188) v4 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 3 client.34830.objecter handle_osd_map got epochs [188,188] > 0 2024-04-20T02:20:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 3 client.34830.objecter handle_osd_map decoding full epoch 188 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f818659c700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f817003b3f0 0x7f817003d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f818659c700 10 monclient: get_auth_request con 0x7f817003b3f0 auth_method 0 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.876+0000 7f8176ffd700 20 client.34830.objecter dump_active .. 0 homeless 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.877+0000 7f8176ffd700 10 client.34830.objecter ms_handle_connect 0x7f817003b3f0 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.877+0000 7f818659c700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f817003b3f0 0x7f817003d8b0 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7f8178006b20 tx=0x7f8178008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.881+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f817c0343a0 con 0x7f81881cc5a0 2024-04-20T02:20:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.881+0000 7f8176ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:01.881+0000 7f8176ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.206+0000 7f818cf86700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:20:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.206+0000 7f818cf86700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.206+0000 7f818cf86700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:02.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.206+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f818809e890 con 0x7f81881cc5a0 2024-04-20T02:20:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.207+0000 7f8176ffd700 1 -- 172.21.15.47:0/4187153108 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3169 (secure 0 0 0) 0x7f817c030c80 con 0x7f81881cc5a0 2024-04-20T02:20:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.207+0000 7f8176ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:02.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.207+0000 7f8176ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:02.207 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":26,"quorum":[0,1,2,3,5,6,7,8],"quorum_names":["a","f","b","g","h","d","i","e"],"quorum_leader_name":"f","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 10 librados: watch_flush enter 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 10 librados: watch_flush exit 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 20 client.34830.objecter shutdown clearing up homeless session... 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 10 client.34830.objecter successfully canceled tick 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f817003b3f0 msgr2=0x7f817003d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f817003b3f0 0x7f817003d8b0 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7f8178006b20 tx=0x7f8178008040 comp rx=0 tx=0).stop 2024-04-20T02:20:02.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 10 monclient: shutdown 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 msgr2=0x7f81881bd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 0x7f81881bd130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f817c01e000 tx=0x7f817c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 shutdown_connections 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f817003b3f0 0x7f817003d8b0 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f81881a66b0 0x7f81881a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f81700062e0 0x7f81881bcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 --2- 172.21.15.47:0/4187153108 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81881cc5a0 0x7f81881bd130 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:02.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 >> 172.21.15.47:0/4187153108 conn(0x7f81880a7570 msgr2=0x7f81881bfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 shutdown_connections 2024-04-20T02:20:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 -- 172.21.15.47:0/4187153108 wait complete. 2024-04-20T02:20:02.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:02.209+0000 7f818cf86700 1 librados: shutdown 2024-04-20T02:20:05.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.367+0000 7efced6e2700 10 monclient: get_monmap_and_config 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.367+0000 7efced6e2700 10 monclient: build_initial_monmap 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.367+0000 7efced6e2700 10 monclient: monmap: 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:05.368621+0000 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:05.368621+0000 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.367 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:05.368 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 Processor -- start 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 -- start start 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient: init 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient: _add_conns ranks=[3,0,7,10,9,6,1,5,2,4,8] 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efce80a5380 0x7efce80a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): picked mon.noname-g con 0x7efce80a5380 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efce80a5760 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce8138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): picked mon.noname-i con 0x7efce80a5ba0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce80751e0 con 0x7efce80a5760 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce80a3410 con 0x7efce80a5380 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efce80a2be0 con 0x7efce80a5ba0 2024-04-20T02:20:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efced6e2700 10 monclient(hunting): authenticate will time out at 1884.769767s 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efce67fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efce80a5760 0x7efce8137a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50784/0 (socket says 172.21.15.47:50784) 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.368+0000 7efce67fc700 1 -- 172.21.15.47:0/1586338905 learned_addr learned my addr 172.21.15.47:0/1586338905 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce8138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efce80a5380 0x7efce80a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): get_auth_request con 0x7efce80a5380 auth_method 0 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): get_auth_request con 0x7efce80a5ba0 auth_method 0 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 1 -- 172.21.15.47:0/1586338905 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509860052 0 0) 0x7efce80751e0 con 0x7efce80a5760 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient(hunting): my global_id is 34839 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 1 -- 172.21.15.47:0/1586338905 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd00035b0 con 0x7efce80a5760 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient(hunting): handle_auth_done global_id 44116 payload 274 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 -- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efce80a5380 msgr2=0x7efce80a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efce80a5380 0x7efce80a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 -- 172.21.15.47:0/1586338905 >> v1:172.21.15.47:6789/0 conn(0x7efce80a5760 legacy=0x7efce8137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 monclient: found mon.noname-i 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 -- 172.21.15.47:0/1586338905 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce80a3130 con 0x7efce80a5ba0 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce6ffd700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efce80a5380 0x7efce80a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce77fe700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce8138190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efce0009dc0 tx=0x7efce000b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a6473bcd0433e802 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 1 -- 172.21.15.47:0/1586338905 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efce000e070 con 0x7efce80a5ba0 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: dump: 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efced6e2700 5 monclient: authenticate success, global_id 44116 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 1 -- 172.21.15.47:0/1586338905 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce00098f0 con 0x7efce80a5ba0 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efced6e2700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efced6e2700 4 set_mon_vals no callback set 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 1 -- 172.21.15.47:0/1586338905 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efce00128a0 con 0x7efce80a5ba0 2024-04-20T02:20:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.369+0000 7efce5ffb700 10 monclient: dump: 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 10 monclient: shutdown 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 -- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 msgr2=0x7efce8138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce8138190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efce0009dc0 tx=0x7efce000b330 comp rx=0 tx=0).stop 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 -- 172.21.15.47:0/1586338905 shutdown_connections 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce8138190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 --2- 172.21.15.47:0/1586338905 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efce80a5380 0x7efce80a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 -- 172.21.15.47:0/1586338905 >> 172.21.15.47:0/1586338905 conn(0x7efce80aa0e0 msgr2=0x7efce80aa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 -- 172.21.15.47:0/1586338905 shutdown_connections 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.370+0000 7efced6e2700 1 -- 172.21.15.47:0/1586338905 wait complete. 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.371+0000 7efced6e2700 10 monclient: build_initial_monmap 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.371+0000 7efced6e2700 10 monclient: monmap: 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.371+0000 7efced6e2700 1 librados: starting msgr at 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.371+0000 7efced6e2700 1 librados: starting objecter 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 Processor -- start 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 -- start start 2024-04-20T02:20:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 librados: setting wanted keys 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 librados: calling monclient init 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient: init 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient: _add_conns ranks=[2,5,7,8,0,1,6,4,3] 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 0x7efce81cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): picked mon.noname-b con 0x7efce80a5380 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efce80a5760 0x7efce8136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): picked mon.noname-h con 0x7efce80a5760 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce81371b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): picked mon.noname-i con 0x7efce80a5ba0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce8124630 con 0x7efce80a5380 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efce81376f0 con 0x7efce80a5760 2024-04-20T02:20:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efce809ddb0 con 0x7efce80a5ba0 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efced6e2700 10 monclient(hunting): authenticate will time out at 1884.773734s 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 0x7efce81cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 0x7efce81cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36860/0 (socket says 172.21.15.47:36860) 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 1 -- 172.21.15.47:0/335491156 learned_addr learned my addr 172.21.15.47:0/335491156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 10 monclient(hunting): get_auth_request con 0x7efce80a5380 auth_method 0 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce77fe700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce81371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce67fc700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efce80a5760 0x7efce8136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce77fe700 10 monclient(hunting): get_auth_request con 0x7efce80a5ba0 auth_method 0 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce67fc700 10 monclient(hunting): get_auth_request con 0x7efce80a5760 auth_method 0 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.372+0000 7efce6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 10 monclient(hunting): handle_auth_done global_id 23492 payload 824 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 -- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 msgr2=0x7efce81371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce81371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 -- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efce80a5760 msgr2=0x7efce8136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efce80a5760 0x7efce8136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 monclient: found mon.noname-b 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 -- 172.21.15.47:0/335491156 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80094a0 con 0x7efce80a5380 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce6ffd700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 0x7efce81cc3b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7efcd800ec10 tx=0x7efcd800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 client.?.objecter ms_handle_connect 0x7efce80a5380 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcd800ccc0 con 0x7efce80a5380 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: dump: 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd800ce20 con 0x7efce80a5380 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 5 monclient: authenticate success, global_id 23492 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 10 monclient: _renew_subs 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce0009a70 con 0x7efce80a5380 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcd80138e0 con 0x7efce80a5380 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efccf7fe700 10 monclient: dump: 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efce57fa700 4 set_mon_vals no callback set 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 10 client.23492.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 10 monclient: _renew_subs 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce80a2150 con 0x7efce80a5380 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.373+0000 7efced6e2700 1 librados: init done 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.374+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7efcd8013470 con 0x7efce80a5380 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.374+0000 7efced6e2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.374+0000 7efced6e2700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.374+0000 7efced6e2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.374+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce806a620 con 0x7efce80a5380 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efcd403a2f0 0x7efcd403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6499+0+0 (secure 0 0 0) 0x7efcd804aa00 con 0x7efce80a5380 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 10 client.23492.objecter ms_dispatch 0x7efce8001550 osd_map(192..192 src has 1..192) v4 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 3 client.23492.objecter handle_osd_map got epochs [192,192] > 0 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 3 client.23492.objecter handle_osd_map decoding full epoch 192 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 20 client.23492.objecter dump_active .. 0 homeless 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efce67fc700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efcd403a2f0 0x7efcd403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efce67fc700 10 monclient: get_auth_request con 0x7efcd403a2f0 auth_method 0 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efce67fc700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efcd403a2f0 0x7efcd403c7b0 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7efce809bd20 tx=0x7efcdc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.375+0000 7efccf7fe700 10 client.23492.objecter ms_handle_connect 0x7efcd403a2f0 2024-04-20T02:20:05.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.380+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7efcd801adc0 con 0x7efce80a5380 2024-04-20T02:20:05.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.380+0000 7efccf7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:05.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.380+0000 7efccf7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efced6e2700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:20:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efced6e2700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efced6e2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efce8124770 con 0x7efce80a5380 2024-04-20T02:20:05.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efccf7fe700 1 -- 172.21.15.47:0/335491156 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7efce8124770 con 0x7efce80a5380 2024-04-20T02:20:05.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efccf7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:05.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.691+0000 7efccf7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:05.692 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:20:05.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 10 librados: watch_flush enter 2024-04-20T02:20:05.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 10 librados: watch_flush exit 2024-04-20T02:20:05.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 20 client.23492.objecter shutdown clearing up homeless session... 2024-04-20T02:20:05.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 10 client.23492.objecter successfully canceled tick 2024-04-20T02:20:05.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efcd403a2f0 msgr2=0x7efcd403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efcd403a2f0 0x7efcd403c7b0 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 crypto rx=0x7efce809bd20 tx=0x7efcdc009040 comp rx=0 tx=0).stop 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 10 monclient: shutdown 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 msgr2=0x7efce81cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 0x7efce81cc3b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7efcd800ec10 tx=0x7efcd800ef20 comp rx=0 tx=0).stop 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 shutdown_connections 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efcd403a2f0 0x7efcd403c7b0 unknown :-1 s=CLOSED pgs=3120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efce80a5ba0 0x7efce81371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efce80a5380 0x7efce81cc3b0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 --2- 172.21.15.47:0/335491156 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efce80a5760 0x7efce8136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 >> 172.21.15.47:0/335491156 conn(0x7efce80aa0e0 msgr2=0x7efce80a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 shutdown_connections 2024-04-20T02:20:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 -- 172.21.15.47:0/335491156 wait complete. 2024-04-20T02:20:05.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.694+0000 7efced6e2700 1 librados: shutdown 2024-04-20T02:20:05.707 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:20:05.707 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:20:05.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.845+0000 7fcbee7dc700 10 monclient: get_monmap_and_config 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.845+0000 7fcbee7dc700 10 monclient: build_initial_monmap 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.845+0000 7fcbee7dc700 10 monclient: monmap: 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:05.846790+0000 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:05.846790+0000 2024-04-20T02:20:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:05.846 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 1 Processor -- start 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 1 -- start start 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 10 monclient: init 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 10 monclient: _add_conns ranks=[5,1,6,7,10,9,8,0,2,4,3] 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbe80a3960 0x7fcbe8071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.846+0000 7fcbee7dc700 10 monclient(hunting): picked mon.noname-h con 0x7fcbe80a3960 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcbe8137b20 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient(hunting): picked mon.noname-d con 0x7fcbe8138630 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80751e0 con 0x7fcbe8137b20 2024-04-20T02:20:05.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806e530 con 0x7fcbe80a3960 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe809ef40 con 0x7fcbe8138630 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient(hunting): authenticate will time out at 1885.248060s 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe813fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41062/0 (socket says 172.21.15.47:41062) 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 -- 172.21.15.47:0/3804483820 learned_addr learned my addr 172.21.15.47:0/3804483820 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): get_auth_request con 0x7fcbe8138630 auth_method 0 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe7fff700 1 --2- 172.21.15.47:0/3804483820 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbe80a3960 0x7fcbe8071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe7fff700 10 monclient(hunting): get_auth_request con 0x7fcbe80a3960 auth_method 0 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient(hunting): handle_auth_done global_id 44106 payload 274 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 -- 172.21.15.47:0/3804483820 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbe80a3960 msgr2=0x7fcbe8071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 --2- 172.21.15.47:0/3804483820 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbe80a3960 0x7fcbe8071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 -- 172.21.15.47:0/3804483820 >> v1:172.21.15.179:6789/0 conn(0x7fcbe8137b20 legacy=0x7fcbe8137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient: _finish_hunting 0 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 monclient: found mon.noname-d 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 -- 172.21.15.47:0/3804483820 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80a3d40 con 0x7fcbe8138630 2024-04-20T02:20:05.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbecd79700 1 --2- 172.21.15.47:0/3804483820 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe813fe60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009e80 tx=0x7fcbe000b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=db5b397e16cdcb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 1 -- 172.21.15.47:0/3804483820 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbe000e070 con 0x7fcbe8138630 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: dump: 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.865 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 5 monclient: authenticate success, global_id 44106 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 1 -- 172.21.15.47:0/3804483820 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00099b0 con 0x7fcbe8138630 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbee7dc700 4 set_mon_vals no callback set 2024-04-20T02:20:05.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 1 -- 172.21.15.47:0/3804483820 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbe000be30 con 0x7fcbe8138630 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.847+0000 7fcbe6ffd700 10 monclient: dump: 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.867 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 10 monclient: shutdown 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 -- 172.21.15.47:0/3804483820 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 msgr2=0x7fcbe813fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/3804483820 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe813fe60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009e80 tx=0x7fcbe000b330 comp rx=0 tx=0).stop 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 -- 172.21.15.47:0/3804483820 shutdown_connections 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/3804483820 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe813fe60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/3804483820 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbe80a3960 0x7fcbe8071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 -- 172.21.15.47:0/3804483820 >> 172.21.15.47:0/3804483820 conn(0x7fcbe80a4a50 msgr2=0x7fcbe80a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:05.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 -- 172.21.15.47:0/3804483820 shutdown_connections 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.848+0000 7fcbee7dc700 1 -- 172.21.15.47:0/3804483820 wait complete. 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 10 monclient: build_initial_monmap 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 10 monclient: monmap: 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:05.869 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 1 librados: starting msgr at 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 1 librados: starting objecter 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 1 Processor -- start 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 1 -- start start 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 1 librados: setting wanted keys 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 1 librados: calling monclient init 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.849+0000 7fcbee7dc700 10 monclient: init 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient: _add_conns ranks=[1,2,6,3,8,5,0,4,7] 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe80a3960 0x7fcbe81cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): picked mon.noname-f con 0x7fcbe80a3960 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:05.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbe8137b20 0x7fcbe8120f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): picked mon.noname-b con 0x7fcbe8137b20 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe8121480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): picked mon.noname-d con 0x7fcbe8138630 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe81245b0 con 0x7fcbe80a3960 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe8123650 con 0x7fcbe8137b20 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806d9d0 con 0x7fcbe8138630 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbee7dc700 10 monclient(hunting): authenticate will time out at 1885.251396s 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe8121480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbe8137b20 0x7fcbe8120f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe8121480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41076/0 (socket says 172.21.15.47:41076) 2024-04-20T02:20:05.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 -- 172.21.15.47:0/2856313958 learned_addr learned my addr 172.21.15.47:0/2856313958 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): get_auth_request con 0x7fcbe8138630 auth_method 0 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): get_auth_request con 0x7fcbe8137b20 auth_method 0 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe80a3960 0x7fcbe81cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): get_auth_request con 0x7fcbe80a3960 auth_method 0 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient(hunting): handle_auth_done global_id 44115 payload 824 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbe8137b20 msgr2=0x7fcbe8120f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbe8137b20 0x7fcbe8120f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe80a3960 msgr2=0x7fcbe81cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbe80a3960 0x7fcbe81cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient: _finish_hunting 0 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 monclient: found mon.noname-d 2024-04-20T02:20:05.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbecd79700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe0009b30 con 0x7fcbe8138630 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.850+0000 7fcbe77fe700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbe8137b20 0x7fcbe8120f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbecd79700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe8121480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0012000 tx=0x7fcbe0004ad0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 client.?.objecter ms_handle_connect 0x7fcbe8138630 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbe00092e0 con 0x7fcbe8138630 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: dump: 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.874 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:20:05.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00046c0 con 0x7fcbe8138630 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbe0004830 con 0x7fcbe8138630 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 5 monclient: authenticate success, global_id 44115 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 10 monclient: _renew_subs 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe8072160 con 0x7fcbe8138630 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe67fc700 4 set_mon_vals no callback set 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbe4ff9700 10 monclient: dump: 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.876 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 10 client.44115.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 10 monclient: _renew_subs 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe8122a80 con 0x7fcbe8138630 2024-04-20T02:20:05.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.851+0000 7fcbee7dc700 1 librados: init done 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcbe0004250 con 0x7fcbe8138630 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbee7dc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 0x7fcbe81219c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe8068430 con 0x7fcbe81257c0 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 0x7fcbe81219c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbc803a2f0 0x7fcbc803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient: get_auth_request con 0x7fcbe81257c0 auth_method 0 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fcbe004c220 con 0x7fcbe8138630 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 10 client.44115.objecter ms_dispatch 0x7fcbe8001550 osd_map(192..192 src has 1..192) v4 2024-04-20T02:20:05.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 3 client.44115.objecter handle_osd_map got epochs [192,192] > 0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 3 client.44115.objecter handle_osd_map decoding full epoch 192 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 20 client.44115.objecter dump_active .. 0 homeless 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe77fe700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbc803a2f0 0x7fcbc803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_done global_id 34848 payload 824 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbe806e250 con 0x7fcbe81257c0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe77fe700 10 monclient: get_auth_request con 0x7fcbc803a2f0 auth_method 0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe7fff700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 0x7fcbe81219c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80107f0 tx=0x7fcbd8010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 10 client.44115.objecter ms_handle_connect 0x7fcbe81257c0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.852+0000 7fcbe4ff9700 10 client.44115.objecter resend_mon_ops 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbd80044a0 con 0x7fcbe81257c0 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:05.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 10 monclient: dump: 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 10 client.44115.objecter ms_handle_connect 0x7fcbc803a2f0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe77fe700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbc803a2f0 0x7fcbc803c7b0 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7fcbe806ba20 tx=0x7fcbd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcbd8004030 con 0x7fcbe81257c0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 msgr2=0x7fcbe81219c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 0x7fcbe81219c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80107f0 tx=0x7fcbd8010bb0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 msgr2=0x7fcbe81219c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.853+0000 7fcbe4ff9700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe81257c0 0x7fcbe81219c0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80107f0 tx=0x7fcbd8010bb0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbee7dc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 0x7fcbe8121f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbee7dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806a400 con 0x7fcbe806aac0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 0x7fcbe8121f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient: get_auth_request con 0x7fcbe806aac0 auth_method 0 2024-04-20T02:20:05.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 10 monclient(hunting): handle_auth_done global_id 34857 payload 824 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 1 -- 172.21.15.47:0/2856313958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcbe8067f90 con 0x7fcbe806aac0 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe4ff9700 10 client.44115.objecter ms_handle_connect 0x7fcbe806aac0 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe7fff700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 0x7fcbe8121f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80107f0 tx=0x7fcbd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe4ff9700 10 client.44115.objecter resend_mon_ops 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbd801c3f0 con 0x7fcbe806aac0 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:05.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.861+0000 7fcbe4ff9700 10 monclient: dump: 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:05.883 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fcbd8003aa0 con 0x7fcbe806aac0 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 msgr2=0x7fcbe8121f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 0x7fcbe8121f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80107f0 tx=0x7fcbd8004750 comp rx=0 tx=0).stop 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 msgr2=0x7fcbe8121f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbe4ff9700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbe806aac0 0x7fcbe8121f00 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80107f0 tx=0x7fcbd8004750 comp rx=0 tx=0).stop 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:05.905 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:05.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 10 librados: watch_flush enter 2024-04-20T02:20:05.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 10 librados: watch_flush exit 2024-04-20T02:20:05.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 20 client.44115.objecter shutdown clearing up homeless session... 2024-04-20T02:20:05.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 10 client.44115.objecter successfully canceled tick 2024-04-20T02:20:05.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbc803a2f0 msgr2=0x7fcbc803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbc803a2f0 0x7fcbc803c7b0 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7fcbe806ba20 tx=0x7fcbd0009040 comp rx=0 tx=0).stop 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 10 monclient: shutdown 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 msgr2=0x7fcbe8121480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbecd79700 1 -- 172.21.15.47:0/2856313958 reap_dead start 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe8121480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0012000 tx=0x7fcbe0004ad0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 shutdown_connections 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 --2- 172.21.15.47:0/2856313958 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcbe8138630 0x7fcbe8121480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 >> 172.21.15.47:0/2856313958 conn(0x7fcbe80a4a50 msgr2=0x7fcbe8070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 shutdown_connections 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 -- 172.21.15.47:0/2856313958 wait complete. 2024-04-20T02:20:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:05.862+0000 7fcbee7dc700 1 librados: shutdown 2024-04-20T02:20:05.909 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:05.909 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:20:05.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.048+0000 7fef8967a700 10 monclient: get_monmap_and_config 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.048+0000 7fef8967a700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.048+0000 7fef8967a700 10 monclient: monmap: 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:06.049477+0000 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:06.049477+0000 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.048+0000 7fef8967a700 1 Processor -- start 2024-04-20T02:20:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 -- start start 2024-04-20T02:20:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient: init 2024-04-20T02:20:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient: _add_conns ranks=[9,10,2,7,1,0,6,3,8,4,5] 2024-04-20T02:20:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef84137d90 0x7fef840a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): picked mon.noname-a con 0x7fef84137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:06.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef84138170 0x7fef840a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): picked mon.noname-f con 0x7fef84138170 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:06.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef841385b0 0x7fef840a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): picked mon.noname-b con 0x7fef841385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:06.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8409fa40 con 0x7fef84137d90 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8409dfc0 con 0x7fef84138170 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef840a7bc0 con 0x7fef841385b0 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef8967a700 10 monclient(hunting): authenticate will time out at 1885.450572s 2024-04-20T02:20:06.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef84137d90 0x7fef840a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef84137d90 0x7fef840a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58410/0 (socket says 172.21.15.47:58410) 2024-04-20T02:20:06.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 1 -- 172.21.15.47:0/3996770061 learned_addr learned my addr 172.21.15.47:0/3996770061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): get_auth_request con 0x7fef84137d90 auth_method 0 2024-04-20T02:20:06.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 1 --2- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef841385b0 0x7fef840a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): get_auth_request con 0x7fef841385b0 auth_method 0 2024-04-20T02:20:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.049+0000 7fef827fc700 1 --2- 172.21.15.47:0/3996770061 >> v2:172.21.15.179:3300/0 conn(0x7fef84138170 0x7fef840a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 10 monclient(hunting): handle_auth_done global_id 34866 payload 274 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 -- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef841385b0 msgr2=0x7fef840a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 --2- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef841385b0 0x7fef840a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 -- 172.21.15.47:0/3996770061 >> v2:172.21.15.179:3300/0 conn(0x7fef84138170 msgr2=0x7fef840a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 --2- 172.21.15.47:0/3996770061 >> v2:172.21.15.179:3300/0 conn(0x7fef84138170 0x7fef840a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 monclient: found mon.noname-a 2024-04-20T02:20:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 -- 172.21.15.47:0/3996770061 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840751e0 con 0x7fef84137d90 2024-04-20T02:20:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef827fc700 10 monclient: get_auth_request con 0x7fef84138170 auth_method 0 2024-04-20T02:20:06.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef827fc700 1 --2- 172.21.15.47:0/3996770061 >> v2:172.21.15.179:3300/0 conn(0x7fef84138170 0x7fef840a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef82ffd700 1 --2- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef84137d90 0x7fef840a09e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef8409e9b0 tx=0x7fef74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e25bdd92710a4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 1 -- 172.21.15.47:0/3996770061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef74011aa0 con 0x7fef84137d90 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: dump: 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.081 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.082 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.083 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 1 -- 172.21.15.47:0/3996770061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef74011c00 con 0x7fef84137d90 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef8967a700 5 monclient: authenticate success, global_id 34866 2024-04-20T02:20:06.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef8967a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 1 -- 172.21.15.47:0/3996770061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef74011d70 con 0x7fef84137d90 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef81ffb700 10 monclient: dump: 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.085 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.086 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef8967a700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.050+0000 7fef8967a700 4 set_mon_vals no callback set 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 10 monclient: shutdown 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 -- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef84137d90 msgr2=0x7fef840a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 --2- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef84137d90 0x7fef840a09e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef8409e9b0 tx=0x7fef74009bd0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 -- 172.21.15.47:0/3996770061 shutdown_connections 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 --2- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef841385b0 0x7fef840a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 --2- 172.21.15.47:0/3996770061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef84137d90 0x7fef840a09e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 --2- 172.21.15.47:0/3996770061 >> v2:172.21.15.179:3300/0 conn(0x7fef84138170 0x7fef840a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 -- 172.21.15.47:0/3996770061 >> 172.21.15.47:0/3996770061 conn(0x7fef840aa8b0 msgr2=0x7fef840a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 -- 172.21.15.47:0/3996770061 shutdown_connections 2024-04-20T02:20:06.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 1 -- 172.21.15.47:0/3996770061 wait complete. 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.051+0000 7fef8967a700 10 monclient: monmap: 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.089 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 librados: starting msgr at 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 librados: starting objecter 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 Processor -- start 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 -- start start 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 librados: setting wanted keys 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 librados: calling monclient init 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient: init 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient: _add_conns ranks=[3,1,7,5,4,2,0,8,6] 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef84137d90 0x7fef841210e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): picked mon.noname-g con 0x7fef84137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 0x7fef84121620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): picked mon.noname-f con 0x7fef84138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fef841385b0 0x7fef84121b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): picked mon.noname-i con 0x7fef841385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef84125160 con 0x7fef84138170 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef84123370 con 0x7fef84137d90 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fef8409dc20 con 0x7fef841385b0 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.052+0000 7fef8967a700 10 monclient(hunting): authenticate will time out at 1885.453957s 2024-04-20T02:20:06.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 0x7fef84121620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fef841385b0 0x7fef84121b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef84137d90 0x7fef841210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 0x7fef84121620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:53378/0 (socket says 172.21.15.47:53378) 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 -- 172.21.15.47:0/2893278465 learned_addr learned my addr 172.21.15.47:0/2893278465 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): get_auth_request con 0x7fef84138170 auth_method 0 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef84137d90 0x7fef841210e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:35522/0 (socket says 172.21.15.47:35522) 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): get_auth_request con 0x7fef84137d90 auth_method 0 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): get_auth_request con 0x7fef841385b0 auth_method 0 2024-04-20T02:20:06.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient(hunting): handle_auth_done global_id 26299 payload 824 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fef841385b0 msgr2=0x7fef84121b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fef841385b0 0x7fef84121b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef84137d90 msgr2=0x7fef841210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef84137d90 0x7fef841210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 monclient: found mon.noname-f 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef827fc700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009550 con 0x7fef84138170 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef82ffd700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef84137d90 0x7fef841210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.053+0000 7fef837fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fef841385b0 0x7fef84121b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef827fc700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 0x7fef84121620 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fef7800c770 tx=0x7fef7800ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 client.?.objecter ms_handle_connect 0x7fef84138170 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef78011a70 con 0x7fef84138170 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: dump: 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.095 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef78011c10 con 0x7fef84138170 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef78011d80 con 0x7fef84138170 2024-04-20T02:20:06.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef6b7fe700 10 monclient: dump: 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.097 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 5 monclient: authenticate success, global_id 26299 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 10 monclient: _renew_subs 2024-04-20T02:20:06.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8409e9e0 con 0x7fef84138170 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef817fa700 4 set_mon_vals no callback set 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 10 client.26299.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 10 monclient: _renew_subs 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef841244a0 con 0x7fef84138170 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.054+0000 7fef8967a700 1 librados: init done 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef8967a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef8967a700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 0x7fef8409ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef84075760 con 0x7fef8406aac0 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 0x7fef8409ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient: get_auth_request con 0x7fef8406aac0 auth_method 0 2024-04-20T02:20:06.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fef78011600 con 0x7fef84138170 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 10 monclient(hunting): handle_auth_done global_id 23519 payload 824 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef8409f760 con 0x7fef8406aac0 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fef7003a2f0 0x7fef7003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fef7804a5c0 con 0x7fef84138170 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 10 client.26299.objecter ms_dispatch 0x7fef84001550 osd_map(192..192 src has 1..192) v4 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 3 client.26299.objecter handle_osd_map got epochs [192,192] > 0 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 3 client.26299.objecter handle_osd_map decoding full epoch 192 2024-04-20T02:20:06.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 20 client.26299.objecter dump_active .. 0 homeless 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 10 client.26299.objecter ms_handle_connect 0x7fef8406aac0 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 10 client.26299.objecter resend_mon_ops 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef82ffd700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 0x7fef8409ab70 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fef8409e9b0 tx=0x7fef74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef837fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fef7003a2f0 0x7fef7003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef837fe700 10 monclient: get_auth_request con 0x7fef7003a2f0 auth_method 0 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef74020070 con 0x7fef8406aac0 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.055+0000 7fef6b7fe700 10 monclient: dump: 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.101 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 10 client.26299.objecter ms_handle_connect 0x7fef7003a2f0 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef837fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fef7003a2f0 0x7fef7003c7b0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009fd0 tx=0x7fef7c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fef740038b0 con 0x7fef8406aac0 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 msgr2=0x7fef8409ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 0x7fef8409ab70 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fef8409e9b0 tx=0x7fef74004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 msgr2=0x7fef8409ab70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.056+0000 7fef6b7fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8406aac0 0x7fef8409ab70 secure :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fef8409e9b0 tx=0x7fef74004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef8967a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef8967a700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 0x7fef8409b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef8967a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef84068180 con 0x7fef8409b0b0 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 0x7fef8409b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient: get_auth_request con 0x7fef8409b0b0 auth_method 0 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 10 monclient(hunting): handle_auth_done global_id 23528 payload 824 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 1 -- 172.21.15.47:0/2893278465 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef841249c0 con 0x7fef8409b0b0 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef82ffd700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 0x7fef8409b4c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fef74009840 tx=0x7fef74011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef6b7fe700 10 client.26299.objecter ms_handle_connect 0x7fef8409b0b0 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef6b7fe700 10 client.26299.objecter resend_mon_ops 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef74025e20 con 0x7fef8409b0b0 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef6b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.064+0000 7fef6b7fe700 10 monclient: dump: 2024-04-20T02:20:06.104 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.105 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fef7402b680 con 0x7fef8409b0b0 2024-04-20T02:20:06.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:06.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:06.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 msgr2=0x7fef8409b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 0x7fef8409b4c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fef74009840 tx=0x7fef74011000 comp rx=0 tx=0).stop 2024-04-20T02:20:06.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 msgr2=0x7fef8409b4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef6b7fe700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fef8409b0b0 0x7fef8409b4c0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.108 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:06.108 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:06.109 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:06.110 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.111 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:06.112 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.113 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:06.114 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:06.115 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:06.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.117 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.119 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:06.121 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.125 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.127 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:06.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:06.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:06.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:06.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:06.133 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 10 librados: watch_flush enter 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 10 librados: watch_flush exit 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 20 client.26299.objecter shutdown clearing up homeless session... 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 10 client.26299.objecter successfully canceled tick 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fef7003a2f0 msgr2=0x7fef7003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fef7003a2f0 0x7fef7003c7b0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009fd0 tx=0x7fef7c00b040 comp rx=0 tx=0).stop 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 10 monclient: shutdown 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 msgr2=0x7fef84121620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 0x7fef84121620 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fef7800c770 tx=0x7fef7800ca80 comp rx=0 tx=0).stop 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef837fe700 1 -- 172.21.15.47:0/2893278465 reap_dead start 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 shutdown_connections 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 --2- 172.21.15.47:0/2893278465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef84138170 0x7fef84121620 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 >> 172.21.15.47:0/2893278465 conn(0x7fef840aa8b0 msgr2=0x7fef840a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 shutdown_connections 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 -- 172.21.15.47:0/2893278465 wait complete. 2024-04-20T02:20:06.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.065+0000 7fef8967a700 1 librados: shutdown 2024-04-20T02:20:06.136 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:06.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:20:06.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.276+0000 7f122d3c5700 10 monclient: get_monmap_and_config 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.276+0000 7f122d3c5700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.276+0000 7f122d3c5700 10 monclient: monmap: 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:06.277658+0000 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:06.277658+0000 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:06.279 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 Processor -- start 2024-04-20T02:20:06.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 -- start start 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient: init 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient: _add_conns ranks=[8,0,5,9,4,7,1,6,3,10,2] 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12280a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): picked mon.noname-e con 0x7f12280a1510 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f122809f9a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f122809fe00 0x7f12280a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): picked mon.noname-h con 0x7f122809fe00 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12280751e0 con 0x7f122809f9a0 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12280720a0 con 0x7f122809fe00 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1228070640 con 0x7f12280a1510 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f122d3c5700 10 monclient(hunting): authenticate will time out at 1885.678734s 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f1226ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12280a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f12267fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f122809f9a0 0x7f12280a38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50798/0 (socket says 172.21.15.47:50798) 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f12267fc700 1 -- 172.21.15.47:0/3164578035 learned_addr learned my addr 172.21.15.47:0/3164578035 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f1226ffd700 10 monclient(hunting): get_auth_request con 0x7f12280a1510 auth_method 0 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f1226ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f1226ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.277+0000 7f1226ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 1 --2- 172.21.15.47:0/3164578035 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f122809fe00 0x7f12280a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): get_auth_request con 0x7f122809fe00 auth_method 0 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 1 -- 172.21.15.47:0/3164578035 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094742256 0 0) 0x7f12280751e0 con 0x7f122809f9a0 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f12277fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient(hunting): my global_id is 34884 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 1 -- 172.21.15.47:0/3164578035 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12100035b0 con 0x7f122809f9a0 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 10 monclient(hunting): handle_auth_done global_id 34739 payload 274 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 1 -- 172.21.15.47:0/3164578035 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f122809fe00 msgr2=0x7f12280a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 1 --2- 172.21.15.47:0/3164578035 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f122809fe00 0x7f12280a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 1 -- 172.21.15.47:0/3164578035 >> v1:172.21.15.47:6789/0 conn(0x7f122809f9a0 legacy=0x7f12280a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 1 monclient: found mon.noname-e 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 1 -- 172.21.15.47:0/3164578035 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228071d30 con 0x7f12280a1510 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1226ffd700 1 --2- 172.21.15.47:0/3164578035 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12280a18f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1218009870 tx=0x7f1218009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=745ab31c0be19eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 1 -- 172.21.15.47:0/3164578035 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1218011aa0 con 0x7f12280a1510 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: dump: 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.294 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f122d3c5700 5 monclient: authenticate success, global_id 34739 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 1 -- 172.21.15.47:0/3164578035 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218011c00 con 0x7f12280a1510 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f122d3c5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f122d3c5700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 1 -- 172.21.15.47:0/3164578035 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1218011d70 con 0x7f12280a1510 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f122d3c5700 4 set_mon_vals no callback set 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.278+0000 7f1225ffb700 10 monclient: dump: 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.296 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 10 monclient: shutdown 2024-04-20T02:20:06.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 -- 172.21.15.47:0/3164578035 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 msgr2=0x7f12280a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 --2- 172.21.15.47:0/3164578035 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12280a18f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1218009870 tx=0x7f1218009b80 comp rx=0 tx=0).stop 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 -- 172.21.15.47:0/3164578035 shutdown_connections 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 --2- 172.21.15.47:0/3164578035 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12280a18f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 --2- 172.21.15.47:0/3164578035 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f122809fe00 0x7f12280a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 -- 172.21.15.47:0/3164578035 >> 172.21.15.47:0/3164578035 conn(0x7f12280a8510 msgr2=0x7f12280a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 -- 172.21.15.47:0/3164578035 shutdown_connections 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 1 -- 172.21.15.47:0/3164578035 wait complete. 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.279+0000 7f122d3c5700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 10 monclient: monmap: 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.298 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 1 librados: starting msgr at 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 1 librados: starting objecter 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 1 Processor -- start 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 1 -- start start 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 1 librados: setting wanted keys 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 1 librados: calling monclient init 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.280+0000 7f122d3c5700 10 monclient: init 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient: _add_conns ranks=[2,7,8,4,0,6,5,1,3] 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f122809f9a0 0x7f1228135c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): picked mon.noname-b con 0x7f122809f9a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f122809fe00 0x7f1228136140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): picked mon.noname-i con 0x7f122809fe00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12281cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): picked mon.noname-e con 0x7f12280a1510 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1228125420 con 0x7f122809f9a0 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f12280a2130 con 0x7f122809fe00 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f122806e3d0 con 0x7f12280a1510 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient(hunting): authenticate will time out at 1885.682152s 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f122809f9a0 0x7f1228135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12281cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12281cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41810/0 (socket says 172.21.15.47:41810) 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f122809f9a0 0x7f1228135c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36896/0 (socket says 172.21.15.47:36896) 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 -- 172.21.15.47:0/1557094975 learned_addr learned my addr 172.21.15.47:0/1557094975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): get_auth_request con 0x7f12280a1510 auth_method 0 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): get_auth_request con 0x7f122809f9a0 auth_method 0 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12267fc700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f122809fe00 0x7f1228136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12267fc700 10 monclient(hunting): get_auth_request con 0x7f122809fe00 auth_method 0 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12267fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12267fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12267fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient(hunting): handle_auth_done global_id 34748 payload 824 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f122809fe00 msgr2=0x7f1228136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f122809fe00 0x7f1228136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f122809f9a0 msgr2=0x7f1228135c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f122809f9a0 0x7f1228135c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 monclient: found mon.noname-e 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180094a0 con 0x7f12280a1510 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12277fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12281cbd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f122000efc0 tx=0x7f122000c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 client.?.objecter ms_handle_connect 0x7f12280a1510 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1220009ea0 con 0x7f12280a1510 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 monclient: dump: 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.304 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 5 monclient: authenticate success, global_id 34748 2024-04-20T02:20:06.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122000f040 con 0x7f12280a1510 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient: _renew_subs 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12200148f0 con 0x7f12280a1510 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1228124d60 con 0x7f12280a1510 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f120f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.281+0000 7f12257fa700 4 set_mon_vals no callback set 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f120f7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f120f7fe700 10 monclient: dump: 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.307 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 10 client.34748.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 10 monclient: _renew_subs 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12281cc2c0 con 0x7f12280a1510 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 1 librados: init done 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1220014480 con 0x7f12280a1510 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 0x7f12281c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.282+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12280682f0 con 0x7f122806aac0 2024-04-20T02:20:06.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f121403a2f0 0x7f121403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 0x7f12281c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f122004b390 con 0x7f12280a1510 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 client.34748.objecter ms_dispatch 0x7f1228001550 osd_map(193..193 src has 1..193) v4 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient: get_auth_request con 0x7f122806aac0 auth_method 0 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 3 client.34748.objecter handle_osd_map got epochs [193,193] > 0 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 3 client.34748.objecter handle_osd_map decoding full epoch 193 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 20 client.34748.objecter dump_active .. 0 homeless 2024-04-20T02:20:06.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f12267fc700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f121403a2f0 0x7f121403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f12267fc700 10 monclient: get_auth_request con 0x7f121403a2f0 auth_method 0 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 10 monclient(hunting): handle_auth_done global_id 44104 payload 824 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1228125280 con 0x7f122806aac0 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 client.34748.objecter ms_handle_connect 0x7f122806aac0 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 client.34748.objecter resend_mon_ops 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f1226ffd700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 0x7f12281c8880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1218009870 tx=0x7f1218004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1218020070 con 0x7f122806aac0 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 monclient: dump: 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.310 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f120f7fe700 10 client.34748.objecter ms_handle_connect 0x7f121403a2f0 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.283+0000 7f12267fc700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f121403a2f0 0x7f121403c7b0 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7f122806de80 tx=0x7f121c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1218004380 con 0x7f122806aac0 2024-04-20T02:20:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 msgr2=0x7f12281c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 0x7f12281c8880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1218009870 tx=0x7f1218004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 msgr2=0x7f12281c8880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.284+0000 7f120f7fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f122806aac0 0x7f12281c8880 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1218009870 tx=0x7f1218004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f122d3c5700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f122d3c5700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 0x7f12281c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f122d3c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f122806a400 con 0x7f12281cc4e0 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 0x7f12281c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient: get_auth_request con 0x7f12281cc4e0 auth_method 0 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 10 monclient(hunting): handle_auth_done global_id 44113 payload 824 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 1 -- 172.21.15.47:0/1557094975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f12280a1e30 con 0x7f12281cc4e0 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f1226ffd700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 0x7f12281c8dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009210 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f120f7fe700 10 client.34748.objecter ms_handle_connect 0x7f12281cc4e0 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f120f7fe700 10 client.34748.objecter resend_mon_ops 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f121802cd50 con 0x7f12281cc4e0 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f120f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f120f7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.292+0000 7f120f7fe700 10 monclient: dump: 2024-04-20T02:20:06.315 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.316 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f121802c8e0 con 0x7f12281cc4e0 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 msgr2=0x7f12281c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 0x7f12281c8dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009210 comp rx=0 tx=0).stop 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 msgr2=0x7f12281c8dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f120f7fe700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f12281cc4e0 0x7f12281c8dc0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1218009840 tx=0x7f1218009210 comp rx=0 tx=0).stop 2024-04-20T02:20:06.317 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:06.318 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.320 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:06.321 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:06.322 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:06.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.327 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:06.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.330 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:06.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:06.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.336 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.340 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.340 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:06.340 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 10 librados: watch_flush enter 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 10 librados: watch_flush exit 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 20 client.34748.objecter shutdown clearing up homeless session... 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 10 client.34748.objecter successfully canceled tick 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f121403a2f0 msgr2=0x7f121403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f121403a2f0 0x7f121403c7b0 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7f122806de80 tx=0x7f121c009040 comp rx=0 tx=0).stop 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 10 monclient: shutdown 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 msgr2=0x7f12281cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 --2- 172.21.15.47:0/1557094975 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12280a1510 0x7f12281cbd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f122000efc0 tx=0x7f122000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f12277fe700 1 -- 172.21.15.47:0/1557094975 reap_dead start 2024-04-20T02:20:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f12277fe700 1 -- 172.21.15.47:0/1557094975 reap_dead start 2024-04-20T02:20:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 shutdown_connections 2024-04-20T02:20:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 >> 172.21.15.47:0/1557094975 conn(0x7f12280a8510 msgr2=0x7f12280a6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 shutdown_connections 2024-04-20T02:20:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 -- 172.21.15.47:0/1557094975 wait complete. 2024-04-20T02:20:06.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.293+0000 7f122d3c5700 1 librados: shutdown 2024-04-20T02:20:06.343 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:06.343 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:20:06.344 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:20:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.482+0000 7fa1486bc700 10 monclient: get_monmap_and_config 2024-04-20T02:20:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.482+0000 7fa1486bc700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.482+0000 7fa1486bc700 10 monclient: monmap: 2024-04-20T02:20:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.483 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:06.483982+0000 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:06.483982+0000 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:06.484 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:06.485 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.483+0000 7fa1486bc700 1 Processor -- start 2024-04-20T02:20:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.483+0000 7fa1486bc700 1 -- start start 2024-04-20T02:20:06.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.483+0000 7fa1486bc700 10 monclient: init 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient: _add_conns ranks=[9,0,7,3,2,8,4,1,5,10,6] 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa140135b60 0x7fa14009b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): picked mon.noname-a con 0x7fa140135b60 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa140135f40 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa140136320 0x7fa1400a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): picked mon.noname-i con 0x7fa140136320 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1400751e0 con 0x7fa140135f40 2024-04-20T02:20:06.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa140067d40 con 0x7fa140135b60 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa140072610 con 0x7fa140136320 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa1486bc700 10 monclient(hunting): authenticate will time out at 1885.885364s 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa140135b60 0x7fa14009b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa140135b60 0x7fa14009b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58412/0 (socket says 172.21.15.47:58412) 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa145c57700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa140135f40 0x7fa1400a2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50814/0 (socket says 172.21.15.47:50814) 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 -- 172.21.15.47:0/1707385034 learned_addr learned my addr 172.21.15.47:0/1707385034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): get_auth_request con 0x7fa140135b60 auth_method 0 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 1 --2- 172.21.15.47:0/1707385034 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa140136320 0x7fa1400a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): get_auth_request con 0x7fa140136320 auth_method 0 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146c59700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient(hunting): handle_auth_done global_id 34893 payload 274 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 -- 172.21.15.47:0/1707385034 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa140136320 msgr2=0x7fa1400a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 --2- 172.21.15.47:0/1707385034 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa140136320 0x7fa1400a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 -- 172.21.15.47:0/1707385034 >> v1:172.21.15.47:6789/0 conn(0x7fa140135f40 legacy=0x7fa1400a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 monclient: found mon.noname-a 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa145456700 1 -- 172.21.15.47:0/1707385034 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960608661 0 0) 0x7fa1400751e0 con 0x7fa140135f40 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa146458700 1 -- 172.21.15.47:0/1707385034 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140067a60 con 0x7fa140135b60 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.484+0000 7fa145456700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa146458700 1 --2- 172.21.15.47:0/1707385034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa140135b60 0x7fa14009b690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098f0 tx=0x7fa13c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857c3200a37f0a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 1 -- 172.21.15.47:0/1707385034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa13c011aa0 con 0x7fa140135b60 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: dump: 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 1 -- 172.21.15.47:0/1707385034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c011c00 con 0x7fa140135b60 2024-04-20T02:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 5 monclient: authenticate success, global_id 34893 2024-04-20T02:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 1 -- 172.21.15.47:0/1707385034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa13c011d70 con 0x7fa140135b60 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 4 set_mon_vals no callback set 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa145456700 10 monclient: dump: 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 10 monclient: shutdown 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 1 -- 172.21.15.47:0/1707385034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa140135b60 msgr2=0x7fa14009b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 1 --2- 172.21.15.47:0/1707385034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa140135b60 0x7fa14009b690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098f0 tx=0x7fa13c009c00 comp rx=0 tx=0).stop 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 1 -- 172.21.15.47:0/1707385034 shutdown_connections 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 1 --2- 172.21.15.47:0/1707385034 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa140136320 0x7fa1400a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 1 --2- 172.21.15.47:0/1707385034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa140135b60 0x7fa14009b690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.485+0000 7fa1486bc700 1 -- 172.21.15.47:0/1707385034 >> 172.21.15.47:0/1707385034 conn(0x7fa1400a7570 msgr2=0x7fa1400a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.486+0000 7fa1486bc700 1 -- 172.21.15.47:0/1707385034 shutdown_connections 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.486+0000 7fa1486bc700 1 -- 172.21.15.47:0/1707385034 wait complete. 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.486+0000 7fa1486bc700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.486+0000 7fa1486bc700 10 monclient: monmap: 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.505 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 1 librados: starting msgr at 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 1 librados: starting objecter 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 1 Processor -- start 2024-04-20T02:20:06.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 1 -- start start 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 1 librados: setting wanted keys 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 1 librados: calling monclient init 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.487+0000 7fa1486bc700 10 monclient: init 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient: _add_conns ranks=[6,1,5,2,0,7,8,3,4] 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa140135b60 0x7fa140122ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): picked mon.noname-d con 0x7fa140135b60 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa140135f40 0x7fa1401cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): picked mon.noname-f con 0x7fa140135f40 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa140136320 0x7fa140134da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): picked mon.noname-h con 0x7fa140136320 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401ccc80 con 0x7fa140135f40 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400a0100 con 0x7fa140136320 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa14009df20 con 0x7fa140135b60 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa1486bc700 10 monclient(hunting): authenticate will time out at 1885.889268s 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa140135b60 0x7fa140122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa140135b60 0x7fa140122ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41084/0 (socket says 172.21.15.47:41084) 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 -- 172.21.15.47:0/2465468541 learned_addr learned my addr 172.21.15.47:0/2465468541 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): get_auth_request con 0x7fa140135b60 auth_method 0 2024-04-20T02:20:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa145c57700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa140135f40 0x7fa1401cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146c59700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa140136320 0x7fa140134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient(hunting): handle_auth_done global_id 44133 payload 824 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa140136320 msgr2=0x7fa140134da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa140136320 0x7fa140134da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa140135f40 msgr2=0x7fa1401cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa140135f40 0x7fa1401cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 monclient: found mon.noname-d 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c009550 con 0x7fa140135b60 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa145c57700 10 monclient: get_auth_request con 0x7fa140135f40 auth_method 0 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146c59700 10 monclient: get_auth_request con 0x7fa140136320 auth_method 0 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa145c57700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa140135f40 0x7fa1401cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146c59700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa140136320 0x7fa140134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa146458700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa140135b60 0x7fa140122ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009130 tx=0x7fa13c004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa136ffd700 10 client.?.objecter ms_handle_connect 0x7fa140135b60 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.488+0000 7fa136ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa13c004320 con 0x7fa140135b60 2024-04-20T02:20:06.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: dump: 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.511 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 5 monclient: authenticate success, global_id 44133 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c01f070 con 0x7fa140135b60 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 10 monclient: _renew_subs 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa13c0044a0 con 0x7fa140135b60 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa140123ee0 con 0x7fa140135b60 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa144c55700 4 set_mon_vals no callback set 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa136ffd700 10 monclient: dump: 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.513 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.514 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 10 client.44133.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 10 monclient: _renew_subs 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401244a0 con 0x7fa140135b60 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.489+0000 7fa1486bc700 1 librados: init done 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa1486bc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa1486bc700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 0x7fa1401c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa140124630 con 0x7fa14006aac0 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa13c021030 con 0x7fa140135b60 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 0x7fa1401c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient: get_auth_request con 0x7fa14006aac0 auth_method 0 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa12403a2f0 0x7fa12403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 10 monclient(hunting): handle_auth_done global_id 44142 payload 824 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fa13c04bdb0 con 0x7fa140135b60 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 10 client.44133.objecter ms_dispatch 0x7fa140001550 osd_map(193..193 src has 1..193) v4 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 3 client.44133.objecter handle_osd_map got epochs [193,193] > 0 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 3 client.44133.objecter handle_osd_map decoding full epoch 193 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1400a1160 con 0x7fa14006aac0 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 20 client.44133.objecter dump_active .. 0 homeless 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa145c57700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 0x7fa1401c8d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1400a39d0 tx=0x7fa13000da80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 10 client.44133.objecter ms_handle_connect 0x7fa14006aac0 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 10 client.44133.objecter resend_mon_ops 2024-04-20T02:20:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa146c59700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa12403a2f0 0x7fa12403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1300070d0 con 0x7fa14006aac0 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa136ffd700 10 monclient: dump: 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.490+0000 7fa146c59700 10 monclient: get_auth_request con 0x7fa12403a2f0 auth_method 0 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa130004a80 con 0x7fa14006aac0 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 msgr2=0x7fa1401c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 0x7fa1401c8d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1400a39d0 tx=0x7fa13000da80 comp rx=0 tx=0).stop 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 msgr2=0x7fa1401c8d60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa14006aac0 0x7fa1401c8d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa136ffd700 10 client.44133.objecter ms_handle_connect 0x7fa12403a2f0 2024-04-20T02:20:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.491+0000 7fa146c59700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa12403a2f0 0x7fa12403c7b0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7fa14009fb50 tx=0x7fa13800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa1486bc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa1486bc700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 0x7fa1401c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa1486bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa140068310 con 0x7fa1401352e0 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 0x7fa1401c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient: get_auth_request con 0x7fa1401352e0 auth_method 0 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 10 monclient(hunting): handle_auth_done global_id 44151 payload 824 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 1 -- 172.21.15.47:0/2465468541 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa1400a0dc0 con 0x7fa1401352e0 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa136ffd700 10 client.44133.objecter ms_handle_connect 0x7fa1401352e0 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa145c57700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 0x7fa1401c92a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1400a39d0 tx=0x7fa130000ed0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa136ffd700 10 client.44133.objecter resend_mon_ops 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa130020070 con 0x7fa1401352e0 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa136ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa136ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.499+0000 7fa136ffd700 10 monclient: dump: 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fa13001a300 con 0x7fa1401352e0 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 msgr2=0x7fa1401c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 0x7fa1401c92a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1400a39d0 tx=0x7fa130000ed0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 msgr2=0x7fa1401c92a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa136ffd700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1401352e0 0x7fa1401c92a0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1400a39d0 tx=0x7fa130000ed0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:06.524 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.525 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.526 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:06.527 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.528 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.530 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.531 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.533 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.539 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:06.541 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.542 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.544 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.546 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:06.546 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 10 librados: watch_flush enter 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 10 librados: watch_flush exit 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 20 client.44133.objecter shutdown clearing up homeless session... 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 10 client.44133.objecter successfully canceled tick 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa12403a2f0 msgr2=0x7fa12403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa12403a2f0 0x7fa12403c7b0 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7fa14009fb50 tx=0x7fa13800b040 comp rx=0 tx=0).stop 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 10 monclient: shutdown 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa140135b60 msgr2=0x7fa140122ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 --2- 172.21.15.47:0/2465468541 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa140135b60 0x7fa140122ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009130 tx=0x7fa13c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa146c59700 1 -- 172.21.15.47:0/2465468541 reap_dead start 2024-04-20T02:20:06.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa146c59700 1 -- 172.21.15.47:0/2465468541 reap_dead start 2024-04-20T02:20:06.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 shutdown_connections 2024-04-20T02:20:06.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 >> 172.21.15.47:0/2465468541 conn(0x7fa1400a7570 msgr2=0x7fa1401bd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 shutdown_connections 2024-04-20T02:20:06.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 -- 172.21.15.47:0/2465468541 wait complete. 2024-04-20T02:20:06.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.500+0000 7fa1486bc700 1 librados: shutdown 2024-04-20T02:20:06.549 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:06.549 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:20:06.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:20:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.688+0000 7f5ff0640700 10 monclient: get_monmap_and_config 2024-04-20T02:20:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.688+0000 7f5ff0640700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.688+0000 7f5ff0640700 10 monclient: monmap: 2024-04-20T02:20:06.689 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:06.689911+0000 2024-04-20T02:20:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:06.689911+0000 2024-04-20T02:20:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.690 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.691 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:06.692 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.689+0000 7f5ff0640700 1 Processor -- start 2024-04-20T02:20:06.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.689+0000 7f5ff0640700 1 -- start start 2024-04-20T02:20:06.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.689+0000 7f5ff0640700 10 monclient: init 2024-04-20T02:20:06.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient: _add_conns ranks=[4,1,0,2,8,5,3,6,9,7,10] 2024-04-20T02:20:06.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe80a3960 0x7f5fe8071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient(hunting): picked mon.noname-c con 0x7f5fe80a3960 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:06.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5fe8137b20 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:06.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5fe8138630 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:06.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe80751e0 con 0x7f5fe8138630 2024-04-20T02:20:06.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe809ef40 con 0x7f5fe8137b20 2024-04-20T02:20:06.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe806e530 con 0x7f5fe80a3960 2024-04-20T02:20:06.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5ff0640700 10 monclient(hunting): authenticate will time out at 1886.091224s 2024-04-20T02:20:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe80a3960 0x7f5fe8071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5feebdd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5fe8138630 0x7f5fe813fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50822/0 (socket says 172.21.15.47:50822) 2024-04-20T02:20:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5feebdd700 1 -- 172.21.15.47:0/686139964 learned_addr learned my addr 172.21.15.47:0/686139964 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:06.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): get_auth_request con 0x7f5fe80a3960 auth_method 0 2024-04-20T02:20:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fed3da700 1 -- 172.21.15.47:0/686139964 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629004698 0 0) 0x7f5fe80751e0 con 0x7f5fe8138630 2024-04-20T02:20:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fed3da700 10 monclient(hunting): _negotiate 2024-04-20T02:20:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fed3da700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fed3da700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fed3da700 10 monclient(hunting): my global_id is 34911 2024-04-20T02:20:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fed3da700 1 -- 172.21.15.47:0/686139964 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd40035b0 con 0x7f5fe8138630 2024-04-20T02:20:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_done global_id 44122 payload 274 2024-04-20T02:20:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 1 -- 172.21.15.47:0/686139964 >> v1:172.21.15.179:6789/0 conn(0x7f5fe8137b20 legacy=0x7f5fe8137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 1 -- 172.21.15.47:0/686139964 >> v1:172.21.15.47:6789/0 conn(0x7f5fe8138630 legacy=0x7f5fe813fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 1 monclient: found mon.noname-c 2024-04-20T02:20:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:20:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 1 -- 172.21.15.47:0/686139964 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80a3d40 con 0x7f5fe80a3960 2024-04-20T02:20:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.690+0000 7f5fee3dc700 1 --2- 172.21.15.47:0/686139964 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe80a3960 0x7f5fe8071880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009870 tx=0x7f5fe4009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=40363c3f5082efac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 1 -- 172.21.15.47:0/686139964 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5fe4011aa0 con 0x7f5fe80a3960 2024-04-20T02:20:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:20:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: dump: 2024-04-20T02:20:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 1 -- 172.21.15.47:0/686139964 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4011c00 con 0x7f5fe80a3960 2024-04-20T02:20:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 5 monclient: authenticate success, global_id 44122 2024-04-20T02:20:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 4 set_mon_vals no callback set 2024-04-20T02:20:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 1 -- 172.21.15.47:0/686139964 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5fe4011d70 con 0x7f5fe80a3960 2024-04-20T02:20:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5fed3da700 10 monclient: dump: 2024-04-20T02:20:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.734 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.734 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.734 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.734 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.734 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.735 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 10 monclient: shutdown 2024-04-20T02:20:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 -- 172.21.15.47:0/686139964 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe80a3960 msgr2=0x7f5fe8071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 --2- 172.21.15.47:0/686139964 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe80a3960 0x7f5fe8071880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009870 tx=0x7f5fe4009b80 comp rx=0 tx=0).stop 2024-04-20T02:20:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 -- 172.21.15.47:0/686139964 shutdown_connections 2024-04-20T02:20:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 --2- 172.21.15.47:0/686139964 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe80a3960 0x7f5fe8071880 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 -- 172.21.15.47:0/686139964 >> 172.21.15.47:0/686139964 conn(0x7f5fe80a4a50 msgr2=0x7f5fe80a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 -- 172.21.15.47:0/686139964 shutdown_connections 2024-04-20T02:20:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.691+0000 7f5ff0640700 1 -- 172.21.15.47:0/686139964 wait complete. 2024-04-20T02:20:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 10 monclient: monmap: 2024-04-20T02:20:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.739 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 1 librados: starting msgr at 2024-04-20T02:20:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 1 librados: starting objecter 2024-04-20T02:20:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 1 Processor -- start 2024-04-20T02:20:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 1 -- start start 2024-04-20T02:20:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 1 librados: setting wanted keys 2024-04-20T02:20:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 1 librados: calling monclient init 2024-04-20T02:20:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.692+0000 7f5ff0640700 10 monclient: init 2024-04-20T02:20:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient: _add_conns ranks=[8,4,2,5,7,3,0,6,1] 2024-04-20T02:20:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe80a3960 0x7f5fe81ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): picked mon.noname-e con 0x7f5fe80a3960 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe8137b20 0x7f5fe8133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): picked mon.noname-c con 0x7f5fe8137b20 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5fe8138630 0x7f5fe8133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): picked mon.noname-b con 0x7f5fe8138630 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81cd8e0 con 0x7f5fe8138630 2024-04-20T02:20:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8125080 con 0x7f5fe8137b20 2024-04-20T02:20:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8072160 con 0x7f5fe80a3960 2024-04-20T02:20:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5ff0640700 10 monclient(hunting): authenticate will time out at 1886.094494s 2024-04-20T02:20:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe80a3960 0x7f5fe81ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe8137b20 0x7f5fe8133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe80a3960 0x7f5fe81ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41818/0 (socket says 172.21.15.47:41818) 2024-04-20T02:20:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 1 -- 172.21.15.47:0/2217987198 learned_addr learned my addr 172.21.15.47:0/2217987198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): get_auth_request con 0x7f5fe80a3960 auth_method 0 2024-04-20T02:20:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): get_auth_request con 0x7f5fe8137b20 auth_method 0 2024-04-20T02:20:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5fe8138630 0x7f5fe8133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): get_auth_request con 0x7f5fe8138630 auth_method 0 2024-04-20T02:20:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5feebdd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 10 monclient(hunting): handle_auth_done global_id 34757 payload 824 2024-04-20T02:20:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe8137b20 msgr2=0x7f5fe8133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.693+0000 7f5fee3dc700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5fe8137b20 0x7f5fe8133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5fe8138630 msgr2=0x7f5fe8133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5fe8138630 0x7f5fe8133d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 1 monclient: found mon.noname-e 2024-04-20T02:20:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe40094a0 con 0x7f5fe80a3960 2024-04-20T02:20:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fee3dc700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe80a3960 0x7f5fe81ccba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4014040 tx=0x7f5fe4004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 client.?.objecter ms_handle_connect 0x7f5fe80a3960 2024-04-20T02:20:06.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5fe4003b10 con 0x7f5fe80a3960 2024-04-20T02:20:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: dump: 2024-04-20T02:20:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.751 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.753 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe401e070 con 0x7f5fe80a3960 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 5 monclient: authenticate success, global_id 34757 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 10 monclient: _renew_subs 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe806d9d0 con 0x7f5fe80a3960 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5fe4003c90 con 0x7f5fe80a3960 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fecbd9700 4 set_mon_vals no callback set 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 10 client.34757.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5fdeffd700 10 monclient: dump: 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.755 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 10 monclient: _renew_subs 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe8124770 con 0x7f5fe80a3960 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.694+0000 7f5ff0640700 1 librados: init done 2024-04-20T02:20:06.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5fe40036a0 con 0x7f5fe80a3960 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5ff0640700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5ff0640700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 0x7f5fe8121100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5ff0640700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8068430 con 0x7f5fe806aac0 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 0x7f5fe8121100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient: get_auth_request con 0x7f5fe806aac0 auth_method 0 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5fcc03a320 0x7f5fcc03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f5fe4020070 con 0x7f5fe80a3960 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 10 client.34757.objecter ms_dispatch 0x7f5fe8001550 osd_map(193..193 src has 1..193) v4 2024-04-20T02:20:06.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 3 client.34757.objecter handle_osd_map got epochs [193,193] > 0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 3 client.34757.objecter handle_osd_map decoding full epoch 193 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 20 client.34757.objecter dump_active .. 0 homeless 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_done global_id 34766 payload 824 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5fe806e250 con 0x7f5fe806aac0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5feebdd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5fcc03a320 0x7f5fcc03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5feebdd700 10 monclient: get_auth_request con 0x7f5fcc03a320 auth_method 0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 10 client.34757.objecter ms_handle_connect 0x7f5fe806aac0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fdeffd700 10 client.34757.objecter resend_mon_ops 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.695+0000 7f5fedbdb700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 0x7f5fe8121100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fe806a6e0 tx=0x7f5fd800fb00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5fd8006cd0 con 0x7f5fe806aac0 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 10 monclient: dump: 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.759 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5fd8006e30 con 0x7f5fe806aac0 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 msgr2=0x7f5fe8121100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 0x7f5fe8121100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fe806a6e0 tx=0x7f5fd800fb00 comp rx=0 tx=0).stop 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 msgr2=0x7f5fe8121100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe806aac0 0x7f5fe8121100 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fe806a6e0 tx=0x7f5fd800fb00 comp rx=0 tx=0).stop 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5fdeffd700 10 client.34757.objecter ms_handle_connect 0x7f5fcc03a320 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.696+0000 7f5feebdd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5fcc03a320 0x7f5fcc03c7e0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001560 tx=0x7f5fe0001a10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5ff0640700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5ff0640700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 0x7f5fe8121a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5ff0640700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe806a400 con 0x7f5fe8121640 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 0x7f5fe8121a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient: get_auth_request con 0x7f5fe8121640 auth_method 0 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 10 monclient(hunting): handle_auth_done global_id 34775 payload 824 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 1 -- 172.21.15.47:0/2217987198 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5fe8121f90 con 0x7f5fe8121640 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fedbdb700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 0x7f5fe8121a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8016000 tx=0x7f5fd8000ed0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fdeffd700 10 client.34757.objecter ms_handle_connect 0x7f5fe8121640 2024-04-20T02:20:06.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fdeffd700 10 client.34757.objecter resend_mon_ops 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5fd8022070 con 0x7f5fe8121640 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.704+0000 7f5fdeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 10 monclient: dump: 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.762 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f5fd801c320 con 0x7f5fe8121640 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 msgr2=0x7f5fe8121a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 0x7f5fe8121a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8016000 tx=0x7f5fd8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 msgr2=0x7f5fe8121a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5fdeffd700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe8121640 0x7f5fe8121a50 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8016000 tx=0x7f5fd8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:06.764 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.765 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.766 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.768 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:06.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.782 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.783 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:06.784 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:06.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 10 librados: watch_flush enter 2024-04-20T02:20:06.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 10 librados: watch_flush exit 2024-04-20T02:20:06.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 20 client.34757.objecter shutdown clearing up homeless session... 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 10 client.34757.objecter successfully canceled tick 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5fcc03a320 msgr2=0x7f5fcc03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5fcc03a320 0x7f5fcc03c7e0 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001560 tx=0x7f5fe0001a10 comp rx=0 tx=0).stop 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 10 monclient: shutdown 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe80a3960 msgr2=0x7f5fe81ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 --2- 172.21.15.47:0/2217987198 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5fe80a3960 0x7f5fe81ccba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4014040 tx=0x7f5fe4004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5feebdd700 1 -- 172.21.15.47:0/2217987198 reap_dead start 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5feebdd700 1 -- 172.21.15.47:0/2217987198 reap_dead start 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 shutdown_connections 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 >> 172.21.15.47:0/2217987198 conn(0x7f5fe80a4a50 msgr2=0x7f5fe8070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 shutdown_connections 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 -- 172.21.15.47:0/2217987198 wait complete. 2024-04-20T02:20:06.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.705+0000 7f5ff0640700 1 librados: shutdown 2024-04-20T02:20:06.788 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:06.788 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:20:06.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.930+0000 7f46e74f5700 10 monclient: get_monmap_and_config 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.930+0000 7f46e74f5700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.930+0000 7f46e74f5700 10 monclient: monmap: 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:06.931396+0000 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:06.931396+0000 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:06.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.930+0000 7f46e74f5700 1 Processor -- start 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 1 -- start start 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient: init 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient: _add_conns ranks=[8,2,1,9,10,6,0,3,4,5,7] 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46e00a0a70 0x7f46e00a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): picked mon.noname-e con 0x7f46e00a0a70 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 0x7f46e00a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): picked mon.noname-b con 0x7f46e00a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f46e00a3290 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e00751e0 con 0x7f46e00a3290 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00a0260 con 0x7f46e00a2950 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00a3740 con 0x7f46e00a0a70 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e74f5700 10 monclient(hunting): authenticate will time out at 1886.332671s 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 0x7f46e00a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 0x7f46e00a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36914/0 (socket says 172.21.15.47:36914) 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 1 -- 172.21.15.47:0/359602579 learned_addr learned my addr 172.21.15.47:0/359602579 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): get_auth_request con 0x7f46e00a2950 auth_method 0 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 1 --2- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46e00a0a70 0x7f46e00a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 10 monclient(hunting): get_auth_request con 0x7f46e00a0a70 auth_method 0 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.931+0000 7f46e5291700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e5291700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 10 monclient(hunting): handle_auth_done global_id 23555 payload 274 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 1 -- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46e00a0a70 msgr2=0x7f46e00a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 1 --2- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46e00a0a70 0x7f46e00a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 1 -- 172.21.15.47:0/359602579 >> v1:172.21.15.179:6789/0 conn(0x7f46e00a3290 legacy=0x7f46e013eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 1 monclient: found mon.noname-b 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:06.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 1 -- 172.21.15.47:0/359602579 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e009ef90 con 0x7f46e00a2950 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e4a90700 1 --2- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 0x7f46e00a2d50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009920 tx=0x7f46d0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8407db7f2f99c2af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 1 -- 172.21.15.47:0/359602579 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46d0011aa0 con 0x7f46e00a2950 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: dump: 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.945 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 1 -- 172.21.15.47:0/359602579 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0011c00 con 0x7f46e00a2950 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e74f5700 5 monclient: authenticate success, global_id 23555 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e74f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:06.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 1 -- 172.21.15.47:0/359602579 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46d0011d70 con 0x7f46e00a2950 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e74f5700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46e74f5700 4 set_mon_vals no callback set 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.932+0000 7f46d7fff700 10 monclient: dump: 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.947 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 10 monclient: shutdown 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 -- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 msgr2=0x7f46e00a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 --2- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 0x7f46e00a2d50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009920 tx=0x7f46d0009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 -- 172.21.15.47:0/359602579 shutdown_connections 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 --2- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46e00a0a70 0x7f46e00a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 --2- 172.21.15.47:0/359602579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a2950 0x7f46e00a2d50 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 -- 172.21.15.47:0/359602579 >> 172.21.15.47:0/359602579 conn(0x7f46e00a7570 msgr2=0x7f46e00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 -- 172.21.15.47:0/359602579 shutdown_connections 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.933+0000 7f46e74f5700 1 -- 172.21.15.47:0/359602579 wait complete. 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient: build_initial_monmap 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient: monmap: 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:06.949 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 librados: starting msgr at 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 librados: starting objecter 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 Processor -- start 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 -- start start 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 librados: setting wanted keys 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 librados: calling monclient init 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient: init 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient: _add_conns ranks=[0,3,2,7,8,1,6,4,5] 2024-04-20T02:20:06.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e00a0a70 0x7f46e01cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): picked mon.noname-a con 0x7f46e00a0a70 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46e00a2950 0x7f46e01bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): picked mon.noname-g con 0x7f46e00a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a3290 0x7f46e01bc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): picked mon.noname-b con 0x7f46e00a3290 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01ab3e0 con 0x7f46e00a0a70 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01ab630 con 0x7f46e00a3290 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01cd160 con 0x7f46e00a2950 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.934+0000 7f46e74f5700 10 monclient(hunting): authenticate will time out at 1886.335928s 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e00a0a70 0x7f46e01cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a3290 0x7f46e01bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e00a0a70 0x7f46e01cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58426/0 (socket says 172.21.15.47:58426) 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a3290 0x7f46e01bc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36924/0 (socket says 172.21.15.47:36924) 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 1 -- 172.21.15.47:0/4271313578 learned_addr learned my addr 172.21.15.47:0/4271313578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): get_auth_request con 0x7f46e00a0a70 auth_method 0 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): get_auth_request con 0x7f46e00a3290 auth_method 0 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46e00a2950 0x7f46e01bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): get_auth_request con 0x7f46e00a2950 auth_method 0 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5291700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient(hunting): handle_auth_done global_id 23564 payload 824 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46e00a2950 msgr2=0x7f46e01bbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46e00a2950 0x7f46e01bbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e00a0a70 msgr2=0x7f46e01cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46e00a0a70 0x7f46e01cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient: _finish_hunting 0 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 monclient: found mon.noname-b 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d0009550 con 0x7f46e00a3290 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e5a92700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a3290 0x7f46e01bc470 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f46d800cb10 tx=0x7f46d800ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 client.?.objecter ms_handle_connect 0x7f46e00a3290 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46d8003e70 con 0x7f46e00a3290 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: dump: 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.954 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8001b80 con 0x7f46e00a3290 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e74f5700 5 monclient: authenticate success, global_id 23564 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e74f5700 10 monclient: _renew_subs 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e74f5700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e009eab0 con 0x7f46e00a3290 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46d800ac10 con 0x7f46e00a3290 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d5ffb700 10 monclient: dump: 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.956 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.935+0000 7f46d77fe700 4 set_mon_vals no callback set 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 10 client.23564.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:06.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 10 monclient: _renew_subs 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e01aa110 con 0x7f46e00a3290 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 1 librados: init done 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f46d800a7a0 con 0x7f46e00a3290 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46d5ffb700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46c403a220 0x7f46c403c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f46d804a5f0 con 0x7f46e00a3290 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46d5ffb700 10 client.23564.objecter ms_dispatch 0x7f46e0001550 osd_map(193..193 src has 1..193) v4 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46d5ffb700 3 client.23564.objecter handle_osd_map got epochs [193,193] > 0 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46d5ffb700 3 client.23564.objecter handle_osd_map decoding full epoch 193 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 0x7f46e01a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.936+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01a9660 con 0x7f46e01ab770 2024-04-20T02:20:06.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46d5ffb700 20 client.23564.objecter dump_active .. 0 homeless 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e5291700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46c403a220 0x7f46c403c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e5291700 10 monclient: get_auth_request con 0x7f46c403a220 auth_method 0 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 0x7f46e01a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient: get_auth_request con 0x7f46e01ab770 auth_method 0 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46d5ffb700 10 client.23564.objecter ms_handle_connect 0x7f46c403a220 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e5291700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46c403a220 0x7f46c403c6e0 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ca0 tx=0x7f46dc009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 10 monclient(hunting): handle_auth_done global_id 26326 payload 824 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46e00a1390 con 0x7f46e01ab770 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46d5ffb700 10 client.23564.objecter ms_handle_connect 0x7f46e01ab770 2024-04-20T02:20:06.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46d5ffb700 10 client.23564.objecter resend_mon_ops 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.937+0000 7f46e4a90700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 0x7f46e01a7b10 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f46e006daf0 tx=0x7f46d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46d0020070 con 0x7f46e01ab770 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 10 monclient: dump: 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.960 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f46d00036a0 con 0x7f46e01ab770 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 msgr2=0x7f46e01a7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 0x7f46e01a7b10 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f46e006daf0 tx=0x7f46d0004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 msgr2=0x7f46e01a7b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.938+0000 7f46d5ffb700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e01ab770 0x7f46e01a7b10 secure :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f46e006daf0 tx=0x7f46d0004750 comp rx=0 tx=0).stop 2024-04-20T02:20:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e74f5700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e74f5700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 0x7f46e01a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e74f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0068310 con 0x7f46e006aac0 2024-04-20T02:20:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 0x7f46e01a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient: get_auth_request con 0x7f46e006aac0 auth_method 0 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.946+0000 7f46e4a90700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46e4a90700 10 monclient(hunting): handle_auth_done global_id 26335 payload 824 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46e4a90700 1 -- 172.21.15.47:0/4271313578 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f46e0067cb0 con 0x7f46e006aac0 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46e4a90700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 0x7f46e01a8050 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009c70 tx=0x7f46d0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 client.23564.objecter ms_handle_connect 0x7f46e006aac0 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 client.23564.objecter resend_mon_ops 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46d002bd00 con 0x7f46e006aac0 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:20:06.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 monclient: dump: 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:06.965 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f46d002be60 con 0x7f46e006aac0 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 msgr2=0x7f46e01a8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.947+0000 7f46d5ffb700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 0x7f46e01a8050 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009c70 tx=0x7f46d0011000 comp rx=0 tx=0).stop 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46d5ffb700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 msgr2=0x7f46e01a8050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46d5ffb700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e006aac0 0x7f46e01a8050 secure :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009c70 tx=0x7f46d0011000 comp rx=0 tx=0).stop 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:06.966 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.967 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.968 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:06.969 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:06.970 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:06.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.977 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:06.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:06.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:06.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:06.986 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 10 librados: watch_flush enter 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 10 librados: watch_flush exit 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 20 client.23564.objecter shutdown clearing up homeless session... 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 10 client.23564.objecter successfully canceled tick 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46c403a220 msgr2=0x7f46c403c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46c403a220 0x7f46c403c6e0 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ca0 tx=0x7f46dc009210 comp rx=0 tx=0).stop 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 10 monclient: shutdown 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a3290 msgr2=0x7f46e01bc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 --2- 172.21.15.47:0/4271313578 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46e00a3290 0x7f46e01bc470 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f46d800cb10 tx=0x7f46d800ce20 comp rx=0 tx=0).stop 2024-04-20T02:20:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e5a92700 1 -- 172.21.15.47:0/4271313578 reap_dead start 2024-04-20T02:20:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e5a92700 1 -- 172.21.15.47:0/4271313578 reap_dead start 2024-04-20T02:20:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 shutdown_connections 2024-04-20T02:20:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 >> 172.21.15.47:0/4271313578 conn(0x7f46e00a7570 msgr2=0x7f46e00a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 shutdown_connections 2024-04-20T02:20:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 -- 172.21.15.47:0/4271313578 wait complete. 2024-04-20T02:20:06.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:06.948+0000 7f46e74f5700 1 librados: shutdown 2024-04-20T02:20:06.989 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:06.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:20:06.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.135+0000 7fb15442b700 10 monclient: get_monmap_and_config 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.135+0000 7fb15442b700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.135+0000 7fb15442b700 10 monclient: monmap: 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:07.136549+0000 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:07.136549+0000 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:07.136 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 Processor -- start 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 -- start start 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient: init 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient: _add_conns ranks=[1,7,4,0,3,6,10,9,2,5,8] 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb14c0a2950 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:07.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb14c067b90 0x7fb14c13cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): picked mon.noname-i con 0x7fb14c067b90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 0x7fb14c13f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): picked mon.noname-c con 0x7fb14c0a3440 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0751e0 con 0x7fb14c0a2950 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c072810 con 0x7fb14c0a3440 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c09f710 con 0x7fb14c067b90 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.136+0000 7fb15442b700 10 monclient(hunting): authenticate will time out at 1886.537903s 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 0x7fb14c13f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb14c067b90 0x7fb14c13cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1521c7700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb14c0a2950 0x7fb14c0a2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:50166/0 (socket says 172.21.15.47:50166) 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1521c7700 1 -- 172.21.15.47:0/1664949722 learned_addr learned my addr 172.21.15.47:0/1664949722 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:07.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb14c067b90 0x7fb14c13cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:33504/0 (socket says 172.21.15.47:33504) 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): get_auth_request con 0x7fb14c067b90 auth_method 0 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 0x7fb14c13f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43044/0 (socket says 172.21.15.47:43044) 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): get_auth_request con 0x7fb14c0a3440 auth_method 0 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient(hunting): handle_auth_done global_id 44140 payload 274 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 -- 172.21.15.47:0/1664949722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb14c067b90 msgr2=0x7fb14c13cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 --2- 172.21.15.47:0/1664949722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb14c067b90 0x7fb14c13cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 -- 172.21.15.47:0/1664949722 >> v1:172.21.15.179:6789/0 conn(0x7fb14c0a2950 legacy=0x7fb14c0a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 monclient: found mon.noname-c 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 -- 172.21.15.47:0/1664949722 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c067500 con 0x7fb14c0a3440 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 1 -- 172.21.15.47:0/1664949722 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196586734 0 0) 0x7fb14c0751e0 con 0x7fb14c0a2950 2024-04-20T02:20:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1529c8700 1 --2- 172.21.15.47:0/1664949722 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 0x7fb14c13f290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb144009dc0 tx=0x7fb14400b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6e7a317576a32608 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 1 -- 172.21.15.47:0/1664949722 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb14400e070 con 0x7fb14c0a3440 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: dump: 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.152 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 1 -- 172.21.15.47:0/1664949722 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb144012430 con 0x7fb14c0a3440 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb15442b700 5 monclient: authenticate success, global_id 44140 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb15442b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 1 -- 172.21.15.47:0/1664949722 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb144012a10 con 0x7fb14c0a3440 2024-04-20T02:20:07.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb15442b700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb15442b700 4 set_mon_vals no callback set 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.137+0000 7fb1511c5700 10 monclient: dump: 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.154 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 10 monclient: shutdown 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 -- 172.21.15.47:0/1664949722 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 msgr2=0x7fb14c13f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 --2- 172.21.15.47:0/1664949722 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 0x7fb14c13f290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb144009dc0 tx=0x7fb14400b330 comp rx=0 tx=0).stop 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 -- 172.21.15.47:0/1664949722 shutdown_connections 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 --2- 172.21.15.47:0/1664949722 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb14c0a3440 0x7fb14c13f290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 --2- 172.21.15.47:0/1664949722 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb14c067b90 0x7fb14c13cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 -- 172.21.15.47:0/1664949722 >> 172.21.15.47:0/1664949722 conn(0x7fb14c0a7570 msgr2=0x7fb14c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 -- 172.21.15.47:0/1664949722 shutdown_connections 2024-04-20T02:20:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.138+0000 7fb15442b700 1 -- 172.21.15.47:0/1664949722 wait complete. 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 10 monclient: monmap: 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.156 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 1 librados: starting msgr at 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 1 librados: starting objecter 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 1 Processor -- start 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 1 -- start start 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 1 librados: setting wanted keys 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 1 librados: calling monclient init 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.139+0000 7fb15442b700 10 monclient: init 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient: _add_conns ranks=[6,2,0,1,3,4,7,5,8] 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb14c067b90 0x7fb14c1cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): picked mon.noname-d con 0x7fb14c067b90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb14c0a2950 0x7fb14c1bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): picked mon.noname-b con 0x7fb14c0a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14c0a3440 0x7fb14c1bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): picked mon.noname-a con 0x7fb14c0a3440 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1aaf90 con 0x7fb14c0a3440 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1a9b80 con 0x7fb14c0a2950 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1cc6f0 con 0x7fb14c067b90 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb15442b700 10 monclient(hunting): authenticate will time out at 1886.541175s 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb14c067b90 0x7fb14c1cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb14c067b90 0x7fb14c1cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41118/0 (socket says 172.21.15.47:41118) 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb14c0a2950 0x7fb14c1bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 -- 172.21.15.47:0/2513446193 learned_addr learned my addr 172.21.15.47:0/2513446193 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): get_auth_request con 0x7fb14c067b90 auth_method 0 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14c0a3440 0x7fb14c1bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): get_auth_request con 0x7fb14c0a2950 auth_method 0 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): get_auth_request con 0x7fb14c0a3440 auth_method 0 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient(hunting): handle_auth_done global_id 44160 payload 824 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb14c0a2950 msgr2=0x7fb14c1bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb14c0a2950 0x7fb14c1bbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14c0a3440 msgr2=0x7fb14c1bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14c0a3440 0x7fb14c1bc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 monclient: found mon.noname-d 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144009a70 con 0x7fb14c067b90 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1519c6700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb14c0a2950 0x7fb14c1bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1529c8700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb14c0a3440 0x7fb14c1bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb1521c7700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb14c067b90 0x7fb14c1cc1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb14800c580 tx=0x7fb14800c890 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb142ffd700 10 client.?.objecter ms_handle_connect 0x7fb14c067b90 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.140+0000 7fb142ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1480070d0 con 0x7fb14c067b90 2024-04-20T02:20:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: dump: 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb148007270 con 0x7fb14c067b90 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 5 monclient: authenticate success, global_id 44160 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 10 monclient: _renew_subs 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c1aacb0 con 0x7fb14c067b90 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1480133c0 con 0x7fb14c067b90 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb142ffd700 10 monclient: dump: 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb1509c4700 4 set_mon_vals no callback set 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 10 client.44160.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 10 monclient: _renew_subs 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c1ab550 con 0x7fb14c067b90 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.141+0000 7fb15442b700 1 librados: init done 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb15442b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb15442b700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 0x7fb14c1a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c068310 con 0x7fb14c06aac0 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb1480073e0 con 0x7fb14c067b90 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 0x7fb14c1a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient: get_auth_request con 0x7fb14c06aac0 auth_method 0 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb13003a2f0 0x7fb13003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7172+0+0 (secure 0 0 0) 0x7fb14804a690 con 0x7fb14c067b90 2024-04-20T02:20:07.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 10 client.44160.objecter ms_dispatch 0x7fb14c001550 osd_map(194..194 src has 1..194) v4 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 3 client.44160.objecter handle_osd_map got epochs [194,194] > 0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 3 client.44160.objecter handle_osd_map decoding full epoch 194 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 10 monclient(hunting): handle_auth_done global_id 21594 payload 824 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb14c09ee70 con 0x7fb14c06aac0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 20 client.44160.objecter dump_active .. 0 homeless 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1529c8700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb13003a2f0 0x7fb13003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1529c8700 10 monclient: get_auth_request con 0x7fb13003a2f0 auth_method 0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 10 client.44160.objecter ms_handle_connect 0x7fb14c06aac0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb142ffd700 10 client.44160.objecter resend_mon_ops 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.142+0000 7fb1519c6700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 0x7fb14c1a7640 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0a4740 tx=0x7fb13c00f520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb13c00fc90 con 0x7fb14c06aac0 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 10 monclient: dump: 2024-04-20T02:20:07.167 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 10 client.44160.objecter ms_handle_connect 0x7fb13003a2f0 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb1529c8700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb13003a2f0 0x7fb13003c7b0 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7fb144014040 tx=0x7fb144011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb13c00fdf0 con 0x7fb14c06aac0 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 msgr2=0x7fb14c1a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 0x7fb14c1a7640 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0a4740 tx=0x7fb13c00f520 comp rx=0 tx=0).stop 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 msgr2=0x7fb14c1a7640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:07.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.143+0000 7fb142ffd700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c06aac0 0x7fb14c1a7640 secure :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0a4740 tx=0x7fb13c00f520 comp rx=0 tx=0).stop 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb15442b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb15442b700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 0x7fb14c1a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb15442b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c06a400 con 0x7fb14c1a7b80 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 0x7fb14c1a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient: get_auth_request con 0x7fb14c1a7b80 auth_method 0 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.151+0000 7fb1519c6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb1519c6700 10 monclient(hunting): handle_auth_done global_id 21603 payload 824 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb1519c6700 1 -- 172.21.15.47:0/2513446193 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb14c09fb30 con 0x7fb14c1a7b80 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 client.44160.objecter ms_handle_connect 0x7fb14c1a7b80 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb1519c6700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 0x7fb14c1a7f90 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb13c01b000 tx=0x7fb13c004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 client.44160.objecter resend_mon_ops 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb13c01dcb0 con 0x7fb14c1a7b80 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 monclient: dump: 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7fb13c023040 con 0x7fb14c1a7b80 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:07.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 msgr2=0x7fb14c1a7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 0x7fb14c1a7f90 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb13c01b000 tx=0x7fb13c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 msgr2=0x7fb14c1a7f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.152+0000 7fb142ffd700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb14c1a7b80 0x7fb14c1a7f90 secure :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb13c01b000 tx=0x7fb13c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:07.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:07.175 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:07.176 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:07.177 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:07.178 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.179 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.182 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:07.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:07.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:07.192 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:07.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:07.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.196 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:07.197 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 10 librados: watch_flush enter 2024-04-20T02:20:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 10 librados: watch_flush exit 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 20 client.44160.objecter shutdown clearing up homeless session... 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 10 client.44160.objecter successfully canceled tick 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb13003a2f0 msgr2=0x7fb13003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb13003a2f0 0x7fb13003c7b0 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7fb144014040 tx=0x7fb144011000 comp rx=0 tx=0).stop 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 10 monclient: shutdown 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb14c067b90 msgr2=0x7fb14c1cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 --2- 172.21.15.47:0/2513446193 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb14c067b90 0x7fb14c1cc1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb14800c580 tx=0x7fb14800c890 comp rx=0 tx=0).stop 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb1529c8700 1 -- 172.21.15.47:0/2513446193 reap_dead start 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb1529c8700 1 -- 172.21.15.47:0/2513446193 reap_dead start 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 shutdown_connections 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 >> 172.21.15.47:0/2513446193 conn(0x7fb14c0a7570 msgr2=0x7fb14c1bd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 shutdown_connections 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 -- 172.21.15.47:0/2513446193 wait complete. 2024-04-20T02:20:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.153+0000 7fb15442b700 1 librados: shutdown 2024-04-20T02:20:07.200 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:07.200 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:20:07.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:20:07.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.339+0000 7f40bb8af700 10 monclient: get_monmap_and_config 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.339+0000 7f40bb8af700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.339+0000 7f40bb8af700 10 monclient: monmap: 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:07.340346+0000 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:07.340346+0000 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:07.339 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:07.340 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.339+0000 7f40bb8af700 1 Processor -- start 2024-04-20T02:20:07.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 -- start start 2024-04-20T02:20:07.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient: init 2024-04-20T02:20:07.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient: _add_conns ranks=[3,6,7,9,8,1,4,10,0,5,2] 2024-04-20T02:20:07.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b40a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): picked mon.noname-g con 0x7f40b40a0b90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 0x7f40b40a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): picked mon.noname-d con 0x7f40b40a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f40b40a3290 0x7f40b413eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): picked mon.noname-i con 0x7f40b40a3290 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40a14b0 con 0x7f40b40a0b90 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40a0210 con 0x7f40b40a2950 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40673e0 con 0x7f40b40a3290 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40bb8af700 10 monclient(hunting): authenticate will time out at 1886.741624s 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 0x7f40b40a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 0x7f40b40a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41124/0 (socket says 172.21.15.47:41124) 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 1 -- 172.21.15.47:0/4100721730 learned_addr learned my addr 172.21.15.47:0/4100721730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): get_auth_request con 0x7f40b40a2950 auth_method 0 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b40a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b9e4c700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f40b40a3290 0x7f40b413eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 10 monclient(hunting): get_auth_request con 0x7f40b40a0b90 auth_method 0 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b9e4c700 10 monclient(hunting): get_auth_request con 0x7f40b40a3290 auth_method 0 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b9e4c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b9e4c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b9e4c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.340+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_done global_id 44169 payload 274 2024-04-20T02:20:07.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 -- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f40b40a3290 msgr2=0x7f40b413eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f40b40a3290 0x7f40b413eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 -- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 msgr2=0x7f40b40a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b40a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 monclient: found mon.noname-d 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 -- 172.21.15.47:0/4100721730 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40751e0 con 0x7f40b40a2950 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40b8e4a700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 0x7f40b40a2d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009920 tx=0x7f40a4009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=16ca234e067f7793 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 1 -- 172.21.15.47:0/4100721730 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a4011aa0 con 0x7f40b40a2950 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: dump: 2024-04-20T02:20:07.355 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.356 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 1 -- 172.21.15.47:0/4100721730 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a4011c00 con 0x7f40b40a2950 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40bb8af700 5 monclient: authenticate success, global_id 44169 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40bb8af700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40bb8af700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 1 -- 172.21.15.47:0/4100721730 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40a4011d70 con 0x7f40b40a2950 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40bb8af700 4 set_mon_vals no callback set 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.341+0000 7f40abfff700 10 monclient: dump: 2024-04-20T02:20:07.357 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.358 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 10 monclient: shutdown 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 -- 172.21.15.47:0/4100721730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 msgr2=0x7f40b40a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 0x7f40b40a2d50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009920 tx=0x7f40a4009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 -- 172.21.15.47:0/4100721730 shutdown_connections 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f40b40a3290 0x7f40b413eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b40a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 --2- 172.21.15.47:0/4100721730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f40b40a2950 0x7f40b40a2d50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 -- 172.21.15.47:0/4100721730 >> 172.21.15.47:0/4100721730 conn(0x7f40b40a7570 msgr2=0x7f40b40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 -- 172.21.15.47:0/4100721730 shutdown_connections 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 1 -- 172.21.15.47:0/4100721730 wait complete. 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.342+0000 7f40bb8af700 10 monclient: monmap: 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.359 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 librados: starting msgr at 2024-04-20T02:20:07.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 librados: starting objecter 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 Processor -- start 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 -- start start 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 librados: setting wanted keys 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 librados: calling monclient init 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient: init 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient: _add_conns ranks=[3,2,0,1,8,6,7,5,4] 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b41c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): picked mon.noname-g con 0x7f40b40a0b90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40b40a2950 0x7f40b41bac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): picked mon.noname-b con 0x7f40b40a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40a3290 0x7f40b41cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): picked mon.noname-a con 0x7f40b40a3290 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b41a7c30 con 0x7f40b40a3290 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b41bb640 con 0x7f40b40a2950 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b409e560 con 0x7f40b40a0b90 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40bb8af700 10 monclient(hunting): authenticate will time out at 1886.744800s 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40a3290 0x7f40b41cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b8e4a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40b40a2950 0x7f40b41bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40a3290 0x7f40b41cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58458/0 (socket says 172.21.15.47:58458) 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b8e4a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40b40a2950 0x7f40b41bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36946/0 (socket says 172.21.15.47:36946) 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 1 -- 172.21.15.47:0/3687584861 learned_addr learned my addr 172.21.15.47:0/3687584861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b964b700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b41c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 10 monclient(hunting): get_auth_request con 0x7f40b40a3290 auth_method 0 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b9e4c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b8e4a700 10 monclient(hunting): get_auth_request con 0x7f40b40a2950 auth_method 0 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b8e4a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b8e4a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.343+0000 7f40b8e4a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): get_auth_request con 0x7f40b40a0b90 auth_method 0 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b8e4a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 10 monclient(hunting): handle_auth_done global_id 34938 payload 824 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 msgr2=0x7f40b41c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f40b40a0b90 0x7f40b41c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40b40a2950 msgr2=0x7f40b41bac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40b40a2950 0x7f40b41bac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 monclient: found mon.noname-a 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4009550 con 0x7f40b40a3290 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b8e4a700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40b40a2950 0x7f40b41bac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40b9e4c700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40a3290 0x7f40b41cc420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00efc0 tx=0x7f40ac009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 client.?.objecter ms_handle_connect 0x7f40b40a3290 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40ac009b20 con 0x7f40b40a3290 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: dump: 2024-04-20T02:20:07.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.366 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40bb8af700 5 monclient: authenticate success, global_id 34938 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac00f040 con 0x7f40b40a3290 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40bb8af700 10 monclient: _renew_subs 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40bb8af700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b41a9d80 con 0x7f40b40a3290 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40ac00cce0 con 0x7f40b40a3290 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40a9ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.344+0000 7f40ab7fe700 4 set_mon_vals no callback set 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 10 client.34938.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:07.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40a9ffb700 10 monclient: dump: 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.368 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 10 monclient: _renew_subs 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b41a8250 con 0x7f40b40a3290 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 1 librados: init done 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 0x7f40b41cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.345+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4068310 con 0x7f40b406aac0 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 0x7f40b41cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f40ac00c830 con 0x7f40b40a3290 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient: get_auth_request con 0x7f40b406aac0 auth_method 0 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f409803a2f0 0x7f409803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7172+0+0 (secure 0 0 0) 0x7f40ac04b3b0 con 0x7f40b40a3290 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 10 client.34938.objecter ms_dispatch 0x7f40b4001550 osd_map(194..194 src has 1..194) v4 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 3 client.34938.objecter handle_osd_map got epochs [194,194] > 0 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 3 client.34938.objecter handle_osd_map decoding full epoch 194 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 10 monclient(hunting): handle_auth_done global_id 44141 payload 824 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 20 client.34938.objecter dump_active .. 0 homeless 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40b409ef90 con 0x7f40b406aac0 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b8e4a700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f409803a2f0 0x7f409803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b8e4a700 10 monclient: get_auth_request con 0x7f409803a2f0 auth_method 0 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 10 client.34938.objecter ms_handle_connect 0x7f40b406aac0 2024-04-20T02:20:07.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 10 client.34938.objecter resend_mon_ops 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40b964b700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 0x7f40b41cc960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b00119d0 tx=0x7f40b0011d90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40b00036a0 con 0x7f40b406aac0 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.346+0000 7f40a9ffb700 10 monclient: dump: 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.371 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 10 client.34938.objecter ms_handle_connect 0x7f409803a2f0 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40b8e4a700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f409803a2f0 0x7f409803c7b0 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009920 tx=0x7f40a4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f40b0004240 con 0x7f40b406aac0 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 msgr2=0x7f40b41cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 0x7f40b41cc960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b00119d0 tx=0x7f40b0011d90 comp rx=0 tx=0).stop 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 msgr2=0x7f40b41cc960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:07.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.347+0000 7f40a9ffb700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b406aac0 0x7f40b41cc960 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b00119d0 tx=0x7f40b0011d90 comp rx=0 tx=0).stop 2024-04-20T02:20:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.354+0000 7f40bb8af700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.354+0000 7f40bb8af700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 0x7f40b41ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.354+0000 7f40bb8af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.354+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40b41a9f10 con 0x7f40b41bb180 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 0x7f40b41ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient: get_auth_request con 0x7f40b41bb180 auth_method 0 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 10 monclient(hunting): handle_auth_done global_id 44150 payload 824 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.355+0000 7f40b964b700 1 -- 172.21.15.47:0/3687584861 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f40b409f270 con 0x7f40b41bb180 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40b964b700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 0x7f40b41ccea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40b001b000 tx=0x7f40b0004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 client.34938.objecter ms_handle_connect 0x7f40b41bb180 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 client.34938.objecter resend_mon_ops 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40b000fdd0 con 0x7f40b41bb180 2024-04-20T02:20:07.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 monclient: dump: 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.376 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f40b00283a0 con 0x7f40b41bb180 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 msgr2=0x7f40b41ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 0x7f40b41ccea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40b001b000 tx=0x7f40b0004750 comp rx=0 tx=0).stop 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 msgr2=0x7f40b41ccea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.356+0000 7f40a9ffb700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f40b41bb180 0x7f40b41ccea0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.377 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:07.378 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.379 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.380 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:07.381 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.382 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:07.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.385 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.388 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:07.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:07.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:07.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:07.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.396 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.397 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:07.398 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:07.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 10 librados: watch_flush enter 2024-04-20T02:20:07.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 10 librados: watch_flush exit 2024-04-20T02:20:07.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 20 client.34938.objecter shutdown clearing up homeless session... 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 10 client.34938.objecter successfully canceled tick 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f409803a2f0 msgr2=0x7f409803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f409803a2f0 0x7f409803c7b0 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009920 tx=0x7f40a4008030 comp rx=0 tx=0).stop 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 10 monclient: shutdown 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40a3290 msgr2=0x7f40b41cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 --2- 172.21.15.47:0/3687584861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40b40a3290 0x7f40b41cc420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00efc0 tx=0x7f40ac009f60 comp rx=0 tx=0).stop 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40b9e4c700 1 -- 172.21.15.47:0/3687584861 reap_dead start 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40b9e4c700 1 -- 172.21.15.47:0/3687584861 reap_dead start 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 shutdown_connections 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 >> 172.21.15.47:0/3687584861 conn(0x7f40b40a7570 msgr2=0x7f40b40a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 shutdown_connections 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 -- 172.21.15.47:0/3687584861 wait complete. 2024-04-20T02:20:07.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.357+0000 7f40bb8af700 1 librados: shutdown 2024-04-20T02:20:07.401 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:07.401 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:20:07.401 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:20:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.539+0000 7f0d596c2700 10 monclient: get_monmap_and_config 2024-04-20T02:20:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.539+0000 7f0d596c2700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.539+0000 7f0d596c2700 10 monclient: monmap: 2024-04-20T02:20:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.539 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:07.540847+0000 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:07.540847+0000 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:07.540 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:07.541 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 Processor -- start 2024-04-20T02:20:07.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 -- start start 2024-04-20T02:20:07.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient: init 2024-04-20T02:20:07.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient: _add_conns ranks=[3,9,10,1,6,7,2,0,4,5,8] 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d54137d90 0x7f0d540a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): picked mon.noname-g con 0x7f0d54137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d54138170 0x7f0d540a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): picked mon.noname-a con 0x7f0d54138170 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0d541385b0 0x7f0d540a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): picked mon.noname-f con 0x7f0d541385b0 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5409fa40 con 0x7f0d54138170 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5409dfc0 con 0x7f0d541385b0 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540a7bc0 con 0x7f0d54137d90 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.540+0000 7f0d596c2700 10 monclient(hunting): authenticate will time out at 1886.941984s 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d54138170 0x7f0d540a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0d54138170 0x7f0d540a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58468/0 (socket says 172.21.15.47:58468) 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 -- 172.21.15.47:0/3766872793 learned_addr learned my addr 172.21.15.47:0/3766872793 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): get_auth_request con 0x7f0d54138170 auth_method 0 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d52ffd700 1 --2- 172.21.15.47:0/3766872793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d54137d90 0x7f0d540a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d537fe700 1 --2- 172.21.15.47:0/3766872793 >> v2:172.21.15.179:3300/0 conn(0x7f0d541385b0 0x7f0d540a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d52ffd700 10 monclient(hunting): get_auth_request con 0x7f0d54137d90 auth_method 0 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d537fe700 10 monclient(hunting): get_auth_request con 0x7f0d541385b0 auth_method 0 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d537fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d537fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d537fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient(hunting): handle_auth_done global_id 34947 payload 274 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 -- 172.21.15.47:0/3766872793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d54137d90 msgr2=0x7f0d540a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 --2- 172.21.15.47:0/3766872793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d54137d90 0x7f0d540a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 -- 172.21.15.47:0/3766872793 >> v2:172.21.15.179:3300/0 conn(0x7f0d541385b0 msgr2=0x7f0d540a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 --2- 172.21.15.47:0/3766872793 >> v2:172.21.15.179:3300/0 conn(0x7f0d541385b0 0x7f0d540a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 monclient: found mon.noname-a 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 -- 172.21.15.47:0/3766872793 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540751e0 con 0x7f0d54138170 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d527fc700 1 --2- 172.21.15.47:0/3766872793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d54138170 0x7f0d540a5c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d5409f3a0 tx=0x7f0d48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea53ce609ef0639 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 1 -- 172.21.15.47:0/3766872793 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d4800bad0 con 0x7f0d54138170 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: dump: 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.555 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 1 -- 172.21.15.47:0/3766872793 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4800bc30 con 0x7f0d54138170 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d596c2700 5 monclient: authenticate success, global_id 34947 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d596c2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d596c2700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d596c2700 4 set_mon_vals no callback set 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 1 -- 172.21.15.47:0/3766872793 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d4800bda0 con 0x7f0d54138170 2024-04-20T02:20:07.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.541+0000 7f0d51ffb700 10 monclient: dump: 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.557 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 10 monclient: shutdown 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 -- 172.21.15.47:0/3766872793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d54138170 msgr2=0x7f0d540a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 --2- 172.21.15.47:0/3766872793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d54138170 0x7f0d540a5c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0d5409f3a0 tx=0x7f0d48009c00 comp rx=0 tx=0).stop 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 -- 172.21.15.47:0/3766872793 shutdown_connections 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 --2- 172.21.15.47:0/3766872793 >> v2:172.21.15.179:3300/0 conn(0x7f0d541385b0 0x7f0d540a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 --2- 172.21.15.47:0/3766872793 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0d54137d90 0x7f0d540a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 --2- 172.21.15.47:0/3766872793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d54138170 0x7f0d540a5c90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 -- 172.21.15.47:0/3766872793 >> 172.21.15.47:0/3766872793 conn(0x7f0d540aa8b0 msgr2=0x7f0d540a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 -- 172.21.15.47:0/3766872793 shutdown_connections 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.542+0000 7f0d596c2700 1 -- 172.21.15.47:0/3766872793 wait complete. 2024-04-20T02:20:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 10 monclient: monmap: 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 1 librados: starting msgr at 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 1 librados: starting objecter 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 1 Processor -- start 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 1 -- start start 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 1 librados: setting wanted keys 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 1 librados: calling monclient init 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.543+0000 7f0d596c2700 10 monclient: init 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient: _add_conns ranks=[6,2,5,4,0,7,3,8,1] 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0d54137d90 0x7f0d54136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): picked mon.noname-d con 0x7f0d54137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d54138170 0x7f0d54137410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): picked mon.noname-b con 0x7f0d54138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0d541385b0 0x7f0d541c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): picked mon.noname-h con 0x7f0d541385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5409ea70 con 0x7f0d54138170 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54137950 con 0x7f0d541385b0 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54124640 con 0x7f0d54137d90 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d596c2700 10 monclient(hunting): authenticate will time out at 1886.945438s 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d54138170 0x7f0d54137410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d54138170 0x7f0d54137410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36956/0 (socket says 172.21.15.47:36956) 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 -- 172.21.15.47:0/859097434 learned_addr learned my addr 172.21.15.47:0/859097434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): get_auth_request con 0x7f0d54138170 auth_method 0 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0d54137d90 0x7f0d54136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0d541385b0 0x7f0d541c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): get_auth_request con 0x7f0d541385b0 auth_method 0 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): get_auth_request con 0x7f0d54137d90 auth_method 0 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient(hunting): handle_auth_done global_id 23591 payload 824 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0d54137d90 msgr2=0x7f0d54136ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0d54137d90 0x7f0d54136ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0d541385b0 msgr2=0x7f0d541c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0d541385b0 0x7f0d541c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 monclient: found mon.noname-b 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d527fc700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48009550 con 0x7f0d54138170 2024-04-20T02:20:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.544+0000 7f0d537fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0d541385b0 0x7f0d541c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d527fc700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d54138170 0x7f0d54137410 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0d5409fef0 tx=0x7f0d48004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 client.?.objecter ms_handle_connect 0x7f0d54138170 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d480036a0 con 0x7f0d54138170 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: dump: 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.564 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 5 monclient: authenticate success, global_id 23591 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4801f070 con 0x7f0d54138170 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d48003cd0 con 0x7f0d54138170 2024-04-20T02:20:07.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d517fa700 4 set_mon_vals no callback set 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d3b7fe700 10 monclient: dump: 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.566 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 10 monclient: _renew_subs 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d54123ff0 con 0x7f0d54138170 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 10 client.23591.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 10 monclient: _renew_subs 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d541224c0 con 0x7f0d54138170 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.545+0000 7f0d596c2700 1 librados: init done 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d596c2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d596c2700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 0x7f0d541c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540aa020 con 0x7f0d5406aac0 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0d48003860 con 0x7f0d54138170 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0d4003a2f0 0x7f0d4003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 0x7f0d541c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient: get_auth_request con 0x7f0d5406aac0 auth_method 0 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7172+0+0 (secure 0 0 0) 0x7f0d4804b9e0 con 0x7f0d54138170 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 10 client.23591.objecter ms_dispatch 0x7f0d54001550 osd_map(194..194 src has 1..194) v4 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 3 client.23591.objecter handle_osd_map got epochs [194,194] > 0 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 3 client.23591.objecter handle_osd_map decoding full epoch 194 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d537fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0d4003a2f0 0x7f0d4003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d3b7fe700 20 client.23591.objecter dump_active .. 0 homeless 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d537fe700 10 monclient: get_auth_request con 0x7f0d4003a2f0 auth_method 0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_done global_id 44188 payload 824 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.546+0000 7f0d52ffd700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d5409dcb0 con 0x7f0d5406aac0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 client.23591.objecter ms_handle_connect 0x7f0d5406aac0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 client.23591.objecter resend_mon_ops 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d52ffd700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 0x7f0d541c9330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d44011600 tx=0x7f0d440119c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 client.23591.objecter ms_handle_connect 0x7f0d4003a2f0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d537fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0d4003a2f0 0x7f0d4003c7b0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7f0d54138a20 tx=0x7f0d4c009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d4400c070 con 0x7f0d5406aac0 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 monclient: dump: 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.569 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0d44004030 con 0x7f0d5406aac0 2024-04-20T02:20:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 msgr2=0x7f0d541c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 0x7f0d541c9330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d44011600 tx=0x7f0d440119c0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 msgr2=0x7f0d541c9330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.547+0000 7f0d3b7fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d5406aac0 0x7f0d541c9330 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d44011600 tx=0x7f0d440119c0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d596c2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d596c2700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 0x7f0d541c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d596c2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54068180 con 0x7f0d54124780 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 0x7f0d541c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient: get_auth_request con 0x7f0d54124780 auth_method 0 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.555+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d52ffd700 10 monclient(hunting): handle_auth_done global_id 44197 payload 824 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d52ffd700 1 -- 172.21.15.47:0/859097434 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0d5409f760 con 0x7f0d54124780 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 client.23591.objecter ms_handle_connect 0x7f0d54124780 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d52ffd700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 0x7f0d541c9870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006fd0 tx=0x7f0d44004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 client.23591.objecter resend_mon_ops 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d44015cf0 con 0x7f0d54124780 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 monclient: dump: 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f0d44015e50 con 0x7f0d54124780 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 msgr2=0x7f0d541c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 0x7f0d541c9870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006fd0 tx=0x7f0d44004750 comp rx=0 tx=0).stop 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 msgr2=0x7f0d541c9870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.556+0000 7f0d3b7fe700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0d54124780 0x7f0d541c9870 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d44006fd0 tx=0x7f0d44004750 comp rx=0 tx=0).stop 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:07.576 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.577 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.578 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:07.579 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:07.580 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.582 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:07.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:07.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.588 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:07.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:07.591 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:07.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.595 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:07.596 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 10 librados: watch_flush enter 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 10 librados: watch_flush exit 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 20 client.23591.objecter shutdown clearing up homeless session... 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 10 client.23591.objecter successfully canceled tick 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0d4003a2f0 msgr2=0x7f0d4003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0d4003a2f0 0x7f0d4003c7b0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7f0d54138a20 tx=0x7f0d4c009780 comp rx=0 tx=0).stop 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 10 monclient: shutdown 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d54138170 msgr2=0x7f0d54137410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 --2- 172.21.15.47:0/859097434 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0d54138170 0x7f0d54137410 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0d5409fef0 tx=0x7f0d48004750 comp rx=0 tx=0).stop 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d537fe700 1 -- 172.21.15.47:0/859097434 reap_dead start 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d537fe700 1 -- 172.21.15.47:0/859097434 reap_dead start 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 shutdown_connections 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 >> 172.21.15.47:0/859097434 conn(0x7f0d540aa8b0 msgr2=0x7f0d540a5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 shutdown_connections 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 -- 172.21.15.47:0/859097434 wait complete. 2024-04-20T02:20:07.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.557+0000 7f0d596c2700 1 librados: shutdown 2024-04-20T02:20:07.599 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:07.599 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:20:07.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:20:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.737+0000 7fe53f8d6700 10 monclient: get_monmap_and_config 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.737+0000 7fe53f8d6700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.737+0000 7fe53f8d6700 10 monclient: monmap: 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:07.738220+0000 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:07.738220+0000 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:07.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.737+0000 7fe53f8d6700 1 Processor -- start 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.737+0000 7fe53f8d6700 1 -- start start 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.737+0000 7fe53f8d6700 10 monclient: init 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient: _add_conns ranks=[1,3,2,0,8,9,7,6,10,4,5] 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe5380a4c80 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5380a5060 0x7fe5380a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): picked mon.noname-g con 0x7fe5380a5060 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 0x7fe53813f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): picked mon.noname-b con 0x7fe53813cf00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5380751e0 con 0x7fe5380a4c80 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380a0000 con 0x7fe53813cf00 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380a6a10 con 0x7fe5380a5060 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53f8d6700 10 monclient(hunting): authenticate will time out at 1887.139554s 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 0x7fe53813f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 0x7fe53813f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36968/0 (socket says 172.21.15.47:36968) 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 1 -- 172.21.15.47:0/3566917567 learned_addr learned my addr 172.21.15.47:0/3566917567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): get_auth_request con 0x7fe53813cf00 auth_method 0 2024-04-20T02:20:07.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 1 --2- 172.21.15.47:0/3566917567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5380a5060 0x7fe5380a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): get_auth_request con 0x7fe5380a5060 auth_method 0 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53de73700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.738+0000 7fe53ce71700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 10 monclient(hunting): handle_auth_done global_id 23600 payload 274 2024-04-20T02:20:07.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 1 -- 172.21.15.47:0/3566917567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5380a5060 msgr2=0x7fe5380a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 1 --2- 172.21.15.47:0/3566917567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5380a5060 0x7fe5380a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 1 -- 172.21.15.47:0/3566917567 >> v1:172.21.15.179:6789/0 conn(0x7fe5380a4c80 legacy=0x7fe5380a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 1 monclient: found mon.noname-b 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 1 -- 172.21.15.47:0/3566917567 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53809e440 con 0x7fe53813cf00 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53de73700 1 --2- 172.21.15.47:0/3566917567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 0x7fe53813f2f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe530009e80 tx=0x7fe53000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04851f936dd831c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 1 -- 172.21.15.47:0/3566917567 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe53000e070 con 0x7fe53813cf00 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: dump: 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 5 monclient: authenticate success, global_id 23600 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 1 -- 172.21.15.47:0/3566917567 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5300099b0 con 0x7fe53813cf00 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 4 set_mon_vals no callback set 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 1 -- 172.21.15.47:0/3566917567 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe53000bd80 con 0x7fe53813cf00 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe52ffff700 10 monclient: dump: 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.752 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 10 monclient: shutdown 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 1 -- 172.21.15.47:0/3566917567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 msgr2=0x7fe53813f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.739+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/3566917567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 0x7fe53813f2f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe530009e80 tx=0x7fe53000b330 comp rx=0 tx=0).stop 2024-04-20T02:20:07.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 1 -- 172.21.15.47:0/3566917567 shutdown_connections 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/3566917567 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe53813cf00 0x7fe53813f2f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/3566917567 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe5380a5060 0x7fe5380a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 1 -- 172.21.15.47:0/3566917567 >> 172.21.15.47:0/3566917567 conn(0x7fe5380aa280 msgr2=0x7fe5380aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 1 -- 172.21.15.47:0/3566917567 shutdown_connections 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 1 -- 172.21.15.47:0/3566917567 wait complete. 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 10 monclient: build_initial_monmap 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.740+0000 7fe53f8d6700 10 monclient: monmap: 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 librados: starting msgr at 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 librados: starting objecter 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 Processor -- start 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 -- start start 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 librados: setting wanted keys 2024-04-20T02:20:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 librados: calling monclient init 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient: init 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient: _add_conns ranks=[6,1,7,0,3,4,5,8,2] 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 0x7fe538135e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): picked mon.noname-d con 0x7fe5380a4c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5380a5060 0x7fe538136360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): picked mon.noname-f con 0x7fe5380a5060 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe53813cf00 0x7fe53809ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): picked mon.noname-i con 0x7fe53813cf00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381cd350 con 0x7fe5380a5060 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe53809ea00 con 0x7fe5380a4c80 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe53809dd20 con 0x7fe53813cf00 2024-04-20T02:20:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53f8d6700 10 monclient(hunting): authenticate will time out at 1887.142737s 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 0x7fe538135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 0x7fe538135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41136/0 (socket says 172.21.15.47:41136) 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53ce71700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5380a5060 0x7fe538136360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 1 -- 172.21.15.47:0/2934480290 learned_addr learned my addr 172.21.15.47:0/2934480290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): get_auth_request con 0x7fe5380a4c80 auth_method 0 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53ce71700 10 monclient(hunting): get_auth_request con 0x7fe5380a5060 auth_method 0 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53ce71700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53ce71700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53ce71700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53de73700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe53813cf00 0x7fe53809ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53de73700 10 monclient(hunting): get_auth_request con 0x7fe53813cf00 auth_method 0 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53de73700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53de73700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53de73700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.741+0000 7fe53d672700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53ce71700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53ce71700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53ce71700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53de73700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53de73700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53de73700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 10 monclient(hunting): handle_auth_done global_id 44187 payload 824 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 -- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe53813cf00 msgr2=0x7fe53809ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe53813cf00 0x7fe53809ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 -- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5380a5060 msgr2=0x7fe538136360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5380a5060 0x7fe538136360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 10 monclient: _finish_hunting 0 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 monclient: found mon.noname-d 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 -- 172.21.15.47:0/2934480290 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530009b30 con 0x7fe5380a4c80 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53ce71700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5380a5060 0x7fe538136360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53d672700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 0x7fe538135e20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe53400c4e0 tx=0x7fe53400c7f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 client.?.objecter ms_handle_connect 0x7fe5380a4c80 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe534005540 con 0x7fe5380a4c80 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:07.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: dump: 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.760 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe534003680 con 0x7fe5380a4c80 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 5 monclient: authenticate success, global_id 44187 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 10 monclient: _renew_subs 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53809d750 con 0x7fe5380a4c80 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe534003c60 con 0x7fe5380a4c80 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:07.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52dffb700 10 monclient: dump: 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:07.762 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe52f7fe700 4 set_mon_vals no callback set 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 10 client.44187.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 10 monclient: _renew_subs 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5381227d0 con 0x7fe5380a4c80 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.742+0000 7fe53f8d6700 1 librados: init done 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe5340037f0 con 0x7fe5380a4c80 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe51c03a2f0 0x7fe51c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 7172+0+0 (secure 0 0 0) 0x7fe534049a40 con 0x7fe5380a4c80 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 10 client.44187.objecter ms_dispatch 0x7fe538001550 osd_map(194..194 src has 1..194) v4 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 3 client.44187.objecter handle_osd_map got epochs [194,194] > 0 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 3 client.44187.objecter handle_osd_map decoding full epoch 194 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe53f8d6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe53f8d6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe53f8d6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5380008d0 con 0x7fe5380a4c80 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.743+0000 7fe52dffb700 20 client.44187.objecter dump_active .. 0 homeless 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.744+0000 7fe53ce71700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe51c03a2f0 0x7fe51c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.744+0000 7fe53ce71700 10 monclient: get_auth_request con 0x7fe51c03a2f0 auth_method 0 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.744+0000 7fe53ce71700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe51c03a2f0 0x7fe51c03c7b0 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7fe5380ab1a0 tx=0x7fe528009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.744+0000 7fe52dffb700 10 client.44187.objecter ms_handle_connect 0x7fe51c03a2f0 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.749+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fe534017080 con 0x7fe5380a4c80 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.749+0000 7fe52dffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:07.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:07.749+0000 7fe52dffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:08.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.054+0000 7fe53f8d6700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:20:08.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.054+0000 7fe53f8d6700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:20:08.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.054+0000 7fe53f8d6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:08.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.054+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fe5381211f0 con 0x7fe5380a4c80 2024-04-20T02:20:08.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.057+0000 7fe52dffb700 1 -- 172.21.15.47:0/2934480290 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fe534011590 con 0x7fe5380a4c80 2024-04-20T02:20:08.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.057+0000 7fe52dffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:20:08.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.057+0000 7fe52dffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 10 librados: watch_flush enter 2024-04-20T02:20:08.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 10 librados: watch_flush exit 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 20 client.44187.objecter shutdown clearing up homeless session... 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 10 client.44187.objecter successfully canceled tick 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe51c03a2f0 msgr2=0x7fe51c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe51c03a2f0 0x7fe51c03c7b0 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7fe5380ab1a0 tx=0x7fe528009210 comp rx=0 tx=0).stop 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 10 monclient: shutdown 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 msgr2=0x7fe538135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 0x7fe538135e20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe53400c4e0 tx=0x7fe53400c7f0 comp rx=0 tx=0).stop 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 shutdown_connections 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe53813cf00 0x7fe53809ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe5380a4c80 0x7fe538135e20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe51c03a2f0 0x7fe51c03c7b0 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 --2- 172.21.15.47:0/2934480290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5380a5060 0x7fe538136360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 >> 172.21.15.47:0/2934480290 conn(0x7fe5380aa280 msgr2=0x7fe5380a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:08.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.060+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 shutdown_connections 2024-04-20T02:20:08.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.061+0000 7fe53f8d6700 1 -- 172.21.15.47:0/2934480290 wait complete. 2024-04-20T02:20:08.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:08.061+0000 7fe53f8d6700 1 librados: shutdown 2024-04-20T02:20:08.085 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.057+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=1,health=10,kv=10,logm=68} crc {auth=2742468629,config=3678701417,health=3969699934,kv=2083578540,logm=2765045464}) 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=11,kv=9,logm=68} crc {auth=4200849854,config=692619927,health=27260805,kv=636753049,logm=1888987118}) 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=1585687735}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=3167941013,config=692619927,health=3514003259,kv=1530195301,logm=224087084}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=10,config=2,health=10,kv=11,logm=67} crc {auth=1994546259,config=692619927,health=441995927,kv=2525026046,logm=2373341636}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=11,config=2,health=12,kv=10,logm=65} crc {auth=2742468629,config=692619927,health=3514003259,kv=105211281,logm=1651523033}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=3643777674,config=692619927,health=3514003259,kv=1530195301,logm=1690037626}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2742468629,config=692619927,health=3514003259,kv=1530195301,logm=3932540316}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.058+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=11,config=2,health=10,kv=10,logm=67} crc {auth=2742468629,config=692619927,health=363122309,kv=3093236402,logm=4137395059}) 2024-04-20T02:20:08.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1273539582}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1976133909}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4058015222}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2755295471}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2867886429}) 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608829529}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3601935316}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=268903451}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.060+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2200611456}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=273133851}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3332861190}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3788011415}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4028334532}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3663241703}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1993124364}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2290532559}) 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=646236885}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.062+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2538392730}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2596260210}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4062302437}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3753406651}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1629851517}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3876238726}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3832756580}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963845253}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=652198482}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.064+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791833274}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899502304}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2070361105}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3682528762}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.065+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=29116399}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2114321389}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=30995584}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3713889650}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=472800838}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.066+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623140799}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=640114950}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2818704380}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3203311321}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2756612842}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2556152866}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3964946894}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=364120863}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.067+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=93910739}) 2024-04-20T02:20:08.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.068+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2778210071}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1857041462}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1283693885}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3284998854}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1999650868}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4028347924}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=872886192}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4050656342}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2633700350}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.069+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4038667919}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2675450049}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3107892839}) 2024-04-20T02:20:08.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2815496870}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3337318937}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3132273684}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2181063787}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.071+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.072+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.072+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2448348744}) 2024-04-20T02:20:08.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.072+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.072+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3169431542}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.072+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3823291606}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1102741902}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2791394433}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=152099921}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2004528812}) 2024-04-20T02:20:08.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1889243734}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1384732952}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3147977122}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=729213226}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.073+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=176033678}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603438624}) 2024-04-20T02:20:08.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855003883}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2645640613}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=212118030}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2763164147}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1492888292}) 2024-04-20T02:20:08.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1254640569}) 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=165621666}) 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.076+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423115136}) 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3881395320}) 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2331662083}) 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3029218282}) 2024-04-20T02:20:08.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=454478141}) 2024-04-20T02:20:08.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1826393348}) 2024-04-20T02:20:08.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=30496547}) 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3669313230}) 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3513436176}) 2024-04-20T02:20:08.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.077+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3985387716}) 2024-04-20T02:20:08.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1531174110}) 2024-04-20T02:20:08.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=63294057}) 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2607031484}) 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3677054409}) 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=876147925}) 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4034970347}) 2024-04-20T02:20:08.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3975326629}) 2024-04-20T02:20:08.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3270915637}) 2024-04-20T02:20:08.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.079+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3053394989}) 2024-04-20T02:20:08.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2143208069}) 2024-04-20T02:20:08.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3319028915}) 2024-04-20T02:20:08.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3045290}) 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2521130354}) 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4171360566}) 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3802528680}) 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1693039745}) 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1489167463}) 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.081+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1631303645}) 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2887251034}) 2024-04-20T02:20:08.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3716287725}) 2024-04-20T02:20:08.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3680631617}) 2024-04-20T02:20:08.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=230998633}) 2024-04-20T02:20:08.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3264148755}) 2024-04-20T02:20:08.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4024296276}) 2024-04-20T02:20:08.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1423286362}) 2024-04-20T02:20:08.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=187338398}) 2024-04-20T02:20:08.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.083+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2945940076}) 2024-04-20T02:20:08.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1060353613}) 2024-04-20T02:20:08.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279431748}) 2024-04-20T02:20:08.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=214427669}) 2024-04-20T02:20:08.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2019005549}) 2024-04-20T02:20:08.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3131359325}) 2024-04-20T02:20:08.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3534922630}) 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3145759916}) 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3474593482}) 2024-04-20T02:20:08.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.084+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2675649792}) 2024-04-20T02:20:08.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=103979491}) 2024-04-20T02:20:08.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=838404812}) 2024-04-20T02:20:08.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2985085779}) 2024-04-20T02:20:08.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1436376744}) 2024-04-20T02:20:08.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=771673060}) 2024-04-20T02:20:08.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3569094670}) 2024-04-20T02:20:08.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4095098525}) 2024-04-20T02:20:08.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662838966}) 2024-04-20T02:20:08.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.086+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3673873578}) 2024-04-20T02:20:08.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=520491217}) 2024-04-20T02:20:08.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297586752}) 2024-04-20T02:20:08.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3946425719}) 2024-04-20T02:20:08.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.087+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=329150491}) 2024-04-20T02:20:08.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3688805302}) 2024-04-20T02:20:08.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3204509910}) 2024-04-20T02:20:08.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2144145142}) 2024-04-20T02:20:08.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1266722790}) 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.088+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2996055699}) 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1272793389}) 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1605293424}) 2024-04-20T02:20:08.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=777393748}) 2024-04-20T02:20:08.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.091+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1157540933}) 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161994822}) 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3465425722}) 2024-04-20T02:20:08.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4180848403}) 2024-04-20T02:20:08.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131951856}) 2024-04-20T02:20:08.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.092+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2256305238}) 2024-04-20T02:20:08.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=166497502}) 2024-04-20T02:20:08.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=841298051}) 2024-04-20T02:20:08.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2353094856}) 2024-04-20T02:20:08.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=397942936}) 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2563305047}) 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=226046836}) 2024-04-20T02:20:08.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1475682211}) 2024-04-20T02:20:08.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2364566629}) 2024-04-20T02:20:08.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.093+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3074156912}) 2024-04-20T02:20:08.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=468310232}) 2024-04-20T02:20:08.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3617313139}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=278559623}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2297942972}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=152219457}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2014907965}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4085416645}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1716338203}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.133+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4211115980}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4211714066}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2385554328}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.134+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2324068444}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4279720824}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1928072860}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1006997479}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4222182029}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4268465293}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.135+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3465769638}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1614691992}) 2024-04-20T02:20:08.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2261950881}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3400191569}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2183177383}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=263872081}) 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3593651264}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4237710896}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=575460526}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.136+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3732889986}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4267869573}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=75774094}) 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2645727614}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=679969577}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2152817944}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=405627958}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1127415908}) 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1623444379}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.138+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3137860545}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230510550}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1823847603}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3878350539}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2726193138}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367365840}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.139+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2193559003}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3018526353}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321639009}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.140+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=854136190}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3216160622}) 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3566413481}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1873579567}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3583280466}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482257589}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=187687397}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3767009549}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922165143}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.141+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2711740855}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1643805566}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1019916809}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.142+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1496541915}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=947132599}) 2024-04-20T02:20:08.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=658621802}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=403932999}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3983392987}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=544899237}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.143+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3403841706}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3528520095}) 2024-04-20T02:20:08.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2247176086}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1813482794}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3174209710}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529126879}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1948367775}) 2024-04-20T02:20:08.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2653761502}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841380525}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.144+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2959498404}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1351612908}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3017728093}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=79900292}) 2024-04-20T02:20:08.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2375525409}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3430653816}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3440115377}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3383631231}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198267729}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.145+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=62454093}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=639105124}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=96,mdsmap=3,mgr=1} crc {logm=2499222269,mdsmap=1531151339,mgr=3836259515}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2849150437,mdsmap=2826658659}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2847381593,mdsmap=2826658659}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=1561196640,mdsmap=2826658659}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=525587031,mdsmap=1531151339}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2945662403,mdsmap=3585761405}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=192781003,mdsmap=1228700967}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.146+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1884213877}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=33} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=2979802314,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=3395944773,osdmap=1194407603}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=34} crc {mdsmap=1228700967,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=2979802314,monmap=310839406,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2575849088,osdmap=2313947823}) 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=33} crc {mdsmap=1228700967,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=1851457862,osdmap=391281655}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=23,osdmap=39} crc {mdsmap=1228700967,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3715678804,mon_config_key=2868358642,monmap=2370797809,osd_metadata=1493347452,osd_pg_creating=1488215441,osd_snap=236368893,osdmap=3132262217}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=26,osdmap=35} crc {mdsmap=1228700967,mgr=3404280412,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1727142646,mon_config_key=2868358642,monmap=3385036551,osd_metadata=999375775,osd_pg_creating=1488215441,osd_snap=3147399850,osdmap=3858398877}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgrstat=10,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=26,osdmap=33} crc {mdsmap=1228700967,mgr=3533788210,mgr_command_descs=470045235,mgrstat=1554323304,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=3395944773,osdmap=1211660853}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=25,osdmap=34} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=2868358642,monmap=3385036551,osd_metadata=1493347452,osd_pg_creating=1488215441,osd_snap=689056714,osdmap=302788905}) 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=27,osdmap=32} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1554323304,mon_config_key=4071561259,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=1488215441,osd_snap=2600143429,osdmap=4204667520}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.148+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,osd_metadata=5,osd_snap=25,osdmap=39} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1786780637,mon_config_key=2278087863,monmap=3385036551,osd_metadata=999375775,osd_snap=3609558896,osdmap=2952723769}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.150+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.150+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.150+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=484126469}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.150+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.150+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.150+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=132574894}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=896609649}) 2024-04-20T02:20:08.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=113039382}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2258285453}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=873784166}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=903144698}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1424605886}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.151+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1381124132}) 2024-04-20T02:20:08.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2428291779}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1664886340}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1706162720}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3701583636}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4255330965}) 2024-04-20T02:20:08.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=826081023}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=978724824}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3951580184}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.153+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3715518989}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=64} crc {osdmap=2345275656}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=66} crc {osdmap=4151307132}) 2024-04-20T02:20:08.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=67} crc {osdmap=827507865}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=65} crc {osdmap=1660431570}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=65} crc {osdmap=227816840}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=62} crc {osdmap=3077131681}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=63} crc {osdmap=2983370707}) 2024-04-20T02:20:08.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:20:08.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2909805708}) 2024-04-20T02:20:08.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:20:08.164+0000 7f2b02ecc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=68} crc {osdmap=1664000382}) 2024-04-20T02:20:09.085 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:20:09.086 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:20:09.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.246+0000 7f2c12969700 10 monclient: get_monmap_and_config 2024-04-20T02:20:09.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.246+0000 7f2c12969700 10 monclient: build_initial_monmap 2024-04-20T02:20:09.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.246+0000 7f2c12969700 10 monclient: monmap: 2024-04-20T02:20:09.246 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:09.246 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:09.247785+0000 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:09.247785+0000 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:09.247 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:09.248 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:09.248 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:09.248 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:09.248 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:09.248 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 1 Processor -- start 2024-04-20T02:20:09.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 1 -- start start 2024-04-20T02:20:09.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 10 monclient: init 2024-04-20T02:20:09.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:09.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 10 monclient: _add_conns ranks=[10,2,7,5,1,9,6,3,4,8,0] 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c0c0a27c0 0x7f2c0c0a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 10 monclient(hunting): picked mon.noname-f con 0x7f2c0c0a27c0 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.247+0000 7f2c12969700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 0x7f2c0c134620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): picked mon.noname-b con 0x7f2c0c0a30e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2c0c0720e0 0x7f2c0c134b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): picked mon.noname-i con 0x7f2c0c0720e0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c0a0180 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c09ff10 con 0x7f2c0c0a30e0 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c09f080 con 0x7f2c0c0720e0 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:09.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c12969700 10 monclient(hunting): authenticate will time out at 1888.649041s 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 0x7f2c0c134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 0x7f2c0c134620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36974/0 (socket says 172.21.15.47:36974) 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/442638684 learned_addr learned my addr 172.21.15.47:0/442638684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): get_auth_request con 0x7f2c0c0a30e0 auth_method 0 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 1 --2- 172.21.15.47:0/442638684 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2c0c0720e0 0x7f2c0c134b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 1 --2- 172.21.15.47:0/442638684 >> v2:172.21.15.179:3300/0 conn(0x7f2c0c0a27c0 0x7f2c0c0a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): get_auth_request con 0x7f2c0c0720e0 auth_method 0 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): get_auth_request con 0x7f2c0c0a27c0 auth_method 0 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c10f06700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_done global_id 23609 payload 274 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/442638684 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2c0c0720e0 msgr2=0x7f2c0c134b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 --2- 172.21.15.47:0/442638684 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2c0c0720e0 0x7f2c0c134b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/442638684 >> v2:172.21.15.179:3300/0 conn(0x7f2c0c0a27c0 msgr2=0x7f2c0c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 --2- 172.21.15.47:0/442638684 >> v2:172.21.15.179:3300/0 conn(0x7f2c0c0a27c0 0x7f2c0c0a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 monclient: found mon.noname-b 2024-04-20T02:20:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/442638684 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0751e0 con 0x7f2c0c0a30e0 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0b7fe700 1 --2- 172.21.15.47:0/442638684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 0x7f2c0c134620 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2c000098f0 tx=0x7f2c00009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44cac487252f312f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0affd700 1 -- 172.21.15.47:0/442638684 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c00011aa0 con 0x7f2c0c0a30e0 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0affd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.248+0000 7f2c0affd700 10 monclient: dump: 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.264 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 10 monclient: _finish_auth 0 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 5 monclient: authenticate success, global_id 23609 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:09.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 1 -- 172.21.15.47:0/442638684 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00011c00 con 0x7f2c0c0a30e0 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 4 set_mon_vals no callback set 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 1 -- 172.21.15.47:0/442638684 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c00011d70 con 0x7f2c0c0a30e0 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c0affd700 10 monclient: dump: 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.266 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 10 monclient: shutdown 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 -- 172.21.15.47:0/442638684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 msgr2=0x7f2c0c134620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 --2- 172.21.15.47:0/442638684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 0x7f2c0c134620 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2c000098f0 tx=0x7f2c00009c00 comp rx=0 tx=0).stop 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 -- 172.21.15.47:0/442638684 shutdown_connections 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 --2- 172.21.15.47:0/442638684 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2c0c0720e0 0x7f2c0c134b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 --2- 172.21.15.47:0/442638684 >> v2:172.21.15.179:3300/0 conn(0x7f2c0c0a27c0 0x7f2c0c0a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 --2- 172.21.15.47:0/442638684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c0c0a30e0 0x7f2c0c134620 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 -- 172.21.15.47:0/442638684 >> 172.21.15.47:0/442638684 conn(0x7f2c0c06cbf0 msgr2=0x7f2c0c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 -- 172.21.15.47:0/442638684 shutdown_connections 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.249+0000 7f2c12969700 1 -- 172.21.15.47:0/442638684 wait complete. 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.250+0000 7f2c12969700 10 monclient: build_initial_monmap 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.250+0000 7f2c12969700 10 monclient: monmap: 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:09.268 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.250+0000 7f2c12969700 1 librados: starting msgr at 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.250+0000 7f2c12969700 1 librados: starting objecter 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.251+0000 7f2c12969700 1 Processor -- start 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.251+0000 7f2c12969700 1 -- start start 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.251+0000 7f2c12969700 1 librados: setting wanted keys 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.251+0000 7f2c12969700 1 librados: calling monclient init 2024-04-20T02:20:09.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.251+0000 7f2c12969700 10 monclient: init 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient: _add_conns ranks=[3,4,5,7,0,8,2,1,6] 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c0c0720e0 0x7f2c0c066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): picked mon.noname-g con 0x7f2c0c0720e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 0x7f2c0c1b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): picked mon.noname-c con 0x7f2c0c0a27c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2c0c0a30e0 0x7f2c0c1b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): picked mon.noname-h con 0x7f2c0c0a30e0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c1ba190 con 0x7f2c0c0720e0 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0725a0 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c1a6b90 con 0x7f2c0c0a30e0 2024-04-20T02:20:09.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c12969700 10 monclient(hunting): authenticate will time out at 1888.653166s 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 0x7f2c0c1b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 0x7f2c0c1b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43046/0 (socket says 172.21.15.47:43046) 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/1897843565 learned_addr learned my addr 172.21.15.47:0/1897843565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c0c0720e0 0x7f2c0c066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): get_auth_request con 0x7f2c0c0a27c0 auth_method 0 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): get_auth_request con 0x7f2c0c0720e0 auth_method 0 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c10f06700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2c0c0a30e0 0x7f2c0c1b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient(hunting): handle_auth_done global_id 44149 payload 824 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2c0c0a30e0 msgr2=0x7f2c0c1b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2c0c0a30e0 0x7f2c0c1b9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c0c0720e0 msgr2=0x7f2c0c066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c0c0720e0 0x7f2c0c066ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 monclient: found mon.noname-c 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 -- 172.21.15.47:0/1897843565 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c00009550 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c10f06700 10 monclient: get_auth_request con 0x7f2c0c0a30e0 auth_method 0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c10f06700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2c0c0a30e0 0x7f2c0c1b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c0b7fe700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 0x7f2c0c1b9710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c00013000 tx=0x7f2c00004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c08ff9700 10 client.?.objecter ms_handle_connect 0x7f2c0c0a27c0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c08ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c000036a0 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.252+0000 7f2c08ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: dump: 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.274 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: _finish_auth 0 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:09.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00003840 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 5 monclient: authenticate success, global_id 44149 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 10 monclient: _renew_subs 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c00003e20 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c1a7860 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c0a7fc700 4 set_mon_vals no callback set 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c08ff9700 10 monclient: dump: 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.276 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 10 client.44149.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 10 monclient: _renew_subs 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c067940 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.253+0000 7f2c12969700 1 librados: init done 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2c000039b0 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c12969700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2bf803a2f0 0x7f2bf803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c12969700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c12969700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7277+0+0 (secure 0 0 0) 0x7f2c0004bee0 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c1a7d80 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 10 client.44149.objecter ms_dispatch 0x7f2c0c001550 osd_map(196..196 src has 1..196) v4 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 3 client.44149.objecter handle_osd_map got epochs [196,196] > 0 2024-04-20T02:20:09.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 3 client.44149.objecter handle_osd_map decoding full epoch 196 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 20 client.44149.objecter dump_active .. 0 homeless 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c0bfff700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2bf803a2f0 0x7f2bf803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c0bfff700 10 monclient: get_auth_request con 0x7f2bf803a2f0 auth_method 0 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c08ff9700 10 client.44149.objecter ms_handle_connect 0x7f2bf803a2f0 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.254+0000 7f2c0bfff700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2bf803a2f0 0x7f2bf803c7b0 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009b60 tx=0x7f2bfc009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.259+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2c00015070 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.259+0000 7f2c08ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.259+0000 7f2c08ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:09.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.563+0000 7f2c12969700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.563+0000 7f2c12969700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.563+0000 7f2c12969700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.563+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2c0c1a8060 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.564+0000 7f2c08ff9700 1 -- 172.21.15.47:0/1897843565 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f2c00003c60 con 0x7f2c0c0a27c0 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.564+0000 7f2c08ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.564+0000 7f2c08ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:09.564 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 10 librados: watch_flush enter 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 10 librados: watch_flush exit 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 20 client.44149.objecter shutdown clearing up homeless session... 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 10 client.44149.objecter successfully canceled tick 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2bf803a2f0 msgr2=0x7f2bf803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2bf803a2f0 0x7f2bf803c7b0 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009b60 tx=0x7f2bfc009210 comp rx=0 tx=0).stop 2024-04-20T02:20:09.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 10 monclient: shutdown 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 msgr2=0x7f2c0c1b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 0x7f2c0c1b9710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c00013000 tx=0x7f2c00004750 comp rx=0 tx=0).stop 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 shutdown_connections 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2bf803a2f0 0x7f2bf803c7b0 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2c0c0a30e0 0x7f2c0c1b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c0c0720e0 0x7f2c0c066ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 --2- 172.21.15.47:0/1897843565 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c0c0a27c0 0x7f2c0c1b9710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 >> 172.21.15.47:0/1897843565 conn(0x7f2c0c06cbf0 msgr2=0x7f2c0c1bc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.566+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 shutdown_connections 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.567+0000 7f2c12969700 1 -- 172.21.15.47:0/1897843565 wait complete. 2024-04-20T02:20:09.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.567+0000 7f2c12969700 1 librados: shutdown 2024-04-20T02:20:09.579 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:20:09.580 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:20:09.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:20:09.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.715+0000 7f3deceeb700 10 monclient: get_monmap_and_config 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.715+0000 7f3deceeb700 10 monclient: build_initial_monmap 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.715+0000 7f3deceeb700 10 monclient: monmap: 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:09.716538+0000 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:09.716538+0000 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:09.715 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:09.716 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 Processor -- start 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 -- start start 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient: init 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient: _add_conns ranks=[8,4,3,10,9,1,0,2,7,5,6] 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 0x7f3de8135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): picked mon.noname-e con 0x7f3de809fa80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3de809fe60 0x7f3de80a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): picked mon.noname-c con 0x7f3de809fe60 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3de80a1f10 0x7f3de80a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): picked mon.noname-g con 0x7f3de80a1f10 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80a0780 con 0x7f3de80a1f10 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3de809df90 con 0x7f3de809fe60 2024-04-20T02:20:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3de809dcb0 con 0x7f3de809fa80 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3deceeb700 10 monclient(hunting): authenticate will time out at 1889.117713s 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 0x7f3de8135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 0x7f3de8135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41842/0 (socket says 172.21.15.47:41842) 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3de809fe60 0x7f3de80a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 1 -- 172.21.15.47:0/1407025466 learned_addr learned my addr 172.21.15.47:0/1407025466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): get_auth_request con 0x7f3de809fa80 auth_method 0 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): get_auth_request con 0x7f3de809fe60 auth_method 0 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.716+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de6d9d700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3de80a1f10 0x7f3de80a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de6d9d700 10 monclient(hunting): get_auth_request con 0x7f3de80a1f10 auth_method 0 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 10 monclient(hunting): handle_auth_done global_id 34793 payload 274 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 -- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3de809fe60 msgr2=0x7f3de80a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3de809fe60 0x7f3de80a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 -- 172.21.15.47:0/1407025466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3de80a1f10 msgr2=0x7f3de80a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3de80a1f10 0x7f3de80a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 10 monclient: _finish_hunting 0 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 monclient: found mon.noname-e 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 -- 172.21.15.47:0/1407025466 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de80751e0 con 0x7f3de809fa80 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3dddd9b700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3de809fe60 0x7f3de80a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de659c700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 0x7f3de8135df0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009920 tx=0x7f3dd8009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=78066e597c5575bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 1 -- 172.21.15.47:0/1407025466 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dd8011aa0 con 0x7f3de809fa80 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: dump: 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: _finish_auth 0 2024-04-20T02:20:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 1 -- 172.21.15.47:0/1407025466 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd8011c00 con 0x7f3de809fa80 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3deceeb700 5 monclient: authenticate success, global_id 34793 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3deceeb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3deceeb700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 1 -- 172.21.15.47:0/1407025466 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dd8011d70 con 0x7f3de809fa80 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3deceeb700 4 set_mon_vals no callback set 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.717+0000 7f3de5d9b700 10 monclient: dump: 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 10 monclient: shutdown 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:09.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 -- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 msgr2=0x7f3de8135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 0x7f3de8135df0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009920 tx=0x7f3dd8009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 -- 172.21.15.47:0/1407025466 shutdown_connections 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3de80a1f10 0x7f3de80a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3de809fa80 0x7f3de8135df0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 --2- 172.21.15.47:0/1407025466 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3de809fe60 0x7f3de80a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 -- 172.21.15.47:0/1407025466 >> 172.21.15.47:0/1407025466 conn(0x7f3de80a3000 msgr2=0x7f3de80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 -- 172.21.15.47:0/1407025466 shutdown_connections 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.718+0000 7f3deceeb700 1 -- 172.21.15.47:0/1407025466 wait complete. 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 10 monclient: build_initial_monmap 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 10 monclient: monmap: 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 1 librados: starting msgr at 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 1 librados: starting objecter 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 1 Processor -- start 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 1 -- start start 2024-04-20T02:20:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 1 librados: setting wanted keys 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 1 librados: calling monclient init 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.719+0000 7f3deceeb700 10 monclient: init 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient: _add_conns ranks=[2,6,1,8,5,7,4,0,3] 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3de809fa80 0x7f3de81cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): picked mon.noname-b con 0x7f3de809fa80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3de809fe60 0x7f3de8134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): picked mon.noname-d con 0x7f3de809fe60 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de80a1f10 0x7f3de8135220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): picked mon.noname-f con 0x7f3de80a1f10 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8132af0 con 0x7f3de80a1f10 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8132250 con 0x7f3de809fa80 2024-04-20T02:20:09.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8131200 con 0x7f3de809fe60 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3deceeb700 10 monclient(hunting): authenticate will time out at 1889.121221s 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3de809fa80 0x7f3de81cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3de809fe60 0x7f3de8134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3de809fe60 0x7f3de8134ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41150/0 (socket says 172.21.15.47:41150) 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 -- 172.21.15.47:0/3855850186 learned_addr learned my addr 172.21.15.47:0/3855850186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3de809fa80 0x7f3de81cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36988/0 (socket says 172.21.15.47:36988) 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): get_auth_request con 0x7f3de809fe60 auth_method 0 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): get_auth_request con 0x7f3de809fa80 auth_method 0 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de80a1f10 0x7f3de8135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): get_auth_request con 0x7f3de80a1f10 auth_method 0 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient(hunting): handle_auth_done global_id 44196 payload 824 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3de809fa80 msgr2=0x7f3de81cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3de809fa80 0x7f3de81cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de80a1f10 msgr2=0x7f3de8135220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3de80a1f10 0x7f3de8135220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 monclient: found mon.noname-d 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00094a0 con 0x7f3de809fe60 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3de659c700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3de809fa80 0x7f3de81cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3dddd9b700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3de809fe60 0x7f3de8134ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000ebe0 tx=0x7f3dd000eef0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3ddf7fe700 10 client.?.objecter ms_handle_connect 0x7f3de809fe60 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.720+0000 7f3ddf7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dd000ccc0 con 0x7f3de809fe60 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: dump: 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.742 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd000ce40 con 0x7f3de809fe60 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 5 monclient: authenticate success, global_id 44196 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 10 monclient: _renew_subs 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd8009550 con 0x7f3de809fe60 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dd00138e0 con 0x7f3de809fe60 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3de559a700 4 set_mon_vals no callback set 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 10 client.44196.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3ddf7fe700 10 monclient: dump: 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.744 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 10 monclient: _renew_subs 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de81327a0 con 0x7f3de809fe60 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.721+0000 7f3deceeb700 1 librados: init done 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3deceeb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3dd0013470 con 0x7f3de809fe60 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3deceeb700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 0x7f3de809a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8068180 con 0x7f3de806aac0 2024-04-20T02:20:09.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 0x7f3de809a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient: get_auth_request con 0x7f3de806aac0 auth_method 0 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3dc803a2f0 0x7f3dc803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7277+0+0 (secure 0 0 0) 0x7f3dd004ad40 con 0x7f3de809fe60 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 10 client.44196.objecter ms_dispatch 0x7f3de8001550 osd_map(196..196 src has 1..196) v4 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 3 client.44196.objecter handle_osd_map got epochs [196,196] > 0 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 3 client.44196.objecter handle_osd_map decoding full epoch 196 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 10 monclient(hunting): handle_auth_done global_id 34956 payload 824 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 20 client.44196.objecter dump_active .. 0 homeless 2024-04-20T02:20:09.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3de809e550 con 0x7f3de806aac0 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 10 client.44196.objecter ms_handle_connect 0x7f3de806aac0 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 10 client.44196.objecter resend_mon_ops 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3de659c700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 0x7f3de809a500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009920 tx=0x7f3dd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dd8020070 con 0x7f3de806aac0 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.722+0000 7f3ddf7fe700 10 monclient: dump: 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.747 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3dd8006c60 con 0x7f3de806aac0 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 msgr2=0x7f3de809a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 0x7f3de809a500 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009920 tx=0x7f3dd8004750 comp rx=0 tx=0).stop 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 msgr2=0x7f3de809a500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3ddf7fe700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de806aac0 0x7f3de809a500 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009920 tx=0x7f3dd8004750 comp rx=0 tx=0).stop 2024-04-20T02:20:09.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3de6d9d700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3dc803a2f0 0x7f3dc803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.723+0000 7f3de6d9d700 10 monclient: get_auth_request con 0x7f3dc803a2f0 auth_method 0 2024-04-20T02:20:09.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.724+0000 7f3ddf7fe700 10 client.44196.objecter ms_handle_connect 0x7f3dc803a2f0 2024-04-20T02:20:09.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.724+0000 7f3de6d9d700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3dc803a2f0 0x7f3dc803c7b0 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f3de81cc820 tx=0x7f3de0009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.730+0000 7f3deceeb700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.730+0000 7f3deceeb700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 0x7f3de809ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3deceeb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de806a400 con 0x7f3de809aa40 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 0x7f3de809ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient: get_auth_request con 0x7f3de809aa40 auth_method 0 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 10 monclient(hunting): handle_auth_done global_id 34965 payload 824 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 1 -- 172.21.15.47:0/3855850186 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3de80a0930 con 0x7f3de809aa40 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3de659c700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 0x7f3de809ae50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800a010 tx=0x7f3dd8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3ddf7fe700 10 client.44196.objecter ms_handle_connect 0x7f3de809aa40 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3ddf7fe700 10 client.44196.objecter resend_mon_ops 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3dd802b300 con 0x7f3de809aa40 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3ddf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3ddf7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.731+0000 7f3ddf7fe700 10 monclient: dump: 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.752 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f3dd8024400 con 0x7f3de809aa40 2024-04-20T02:20:09.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 msgr2=0x7f3de809ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 0x7f3de809ae50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800a010 tx=0x7f3dd8011000 comp rx=0 tx=0).stop 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 msgr2=0x7f3de809ae50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3ddf7fe700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3de809aa40 0x7f3de809ae50 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800a010 tx=0x7f3dd8011000 comp rx=0 tx=0).stop 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:09.754 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:09.755 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:09.756 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:09.757 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:09.758 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:09.759 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:09.760 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:09.762 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:09.763 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:09.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:09.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:09.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:09.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:09.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:09.774 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 10 librados: watch_flush enter 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 10 librados: watch_flush exit 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 20 client.44196.objecter shutdown clearing up homeless session... 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 10 client.44196.objecter successfully canceled tick 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3dc803a2f0 msgr2=0x7f3dc803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3dc803a2f0 0x7f3dc803c7b0 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f3de81cc820 tx=0x7f3de0009780 comp rx=0 tx=0).stop 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 10 monclient: shutdown 2024-04-20T02:20:09.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3de809fe60 msgr2=0x7f3de8134ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 --2- 172.21.15.47:0/3855850186 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3de809fe60 0x7f3de8134ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000ebe0 tx=0x7f3dd000eef0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3de6d9d700 1 -- 172.21.15.47:0/3855850186 reap_dead start 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3de6d9d700 1 -- 172.21.15.47:0/3855850186 reap_dead start 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 shutdown_connections 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 >> 172.21.15.47:0/3855850186 conn(0x7f3de80a3000 msgr2=0x7f3de81bd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 shutdown_connections 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 -- 172.21.15.47:0/3855850186 wait complete. 2024-04-20T02:20:09.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.732+0000 7f3deceeb700 1 librados: shutdown 2024-04-20T02:20:09.777 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:09.777 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:20:09.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:20:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.916+0000 7ff30bb26700 10 monclient: get_monmap_and_config 2024-04-20T02:20:09.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.916+0000 7ff30bb26700 10 monclient: build_initial_monmap 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.917+0000 7ff30bb26700 10 monclient: monmap: 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:09.918163+0000 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:09.918163+0000 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:09.917 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:09.918 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:09.918 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:09.918 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.917+0000 7ff30bb26700 1 Processor -- start 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.917+0000 7ff30bb26700 1 -- start start 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.917+0000 7ff30bb26700 10 monclient: init 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient: _add_conns ranks=[8,2,1,5,3,7,9,4,10,0,6] 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 0x7ff3040a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): picked mon.noname-e con 0x7ff3040a0830 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 0x7ff3040a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): picked mon.noname-b con 0x7ff3040a01b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff3040a1150 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040751e0 con 0x7ff3040a1150 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff304067820 con 0x7ff3040a01b0 2024-04-20T02:20:09.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff30409deb0 con 0x7ff3040a0830 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff30bb26700 10 monclient(hunting): authenticate will time out at 1889.319539s 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 0x7ff3040a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 0x7ff3040a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 0x7ff3040a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36992/0 (socket says 172.21.15.47:36992) 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 0x7ff3040a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41844/0 (socket says 172.21.15.47:41844) 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 1 -- 172.21.15.47:0/578022783 learned_addr learned my addr 172.21.15.47:0/578022783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): get_auth_request con 0x7ff3040a01b0 auth_method 0 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): get_auth_request con 0x7ff3040a0830 auth_method 0 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3098c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 10 monclient(hunting): handle_auth_done global_id 23627 payload 274 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 1 -- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 msgr2=0x7ff3040a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.918+0000 7ff3090c1700 1 --2- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 0x7ff3040a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 1 -- 172.21.15.47:0/578022783 >> v1:172.21.15.179:6789/0 conn(0x7ff3040a1150 legacy=0x7ff3040a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 10 monclient: _finish_hunting 0 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 1 monclient: found mon.noname-b 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 1 -- 172.21.15.47:0/578022783 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304067330 con 0x7ff3040a01b0 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3098c2700 1 --2- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 0x7ff3040a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:09.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3090c1700 1 --2- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 0x7ff3040a2950 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009920 tx=0x7ff2f4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3bea48b03527d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 1 -- 172.21.15.47:0/578022783 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2f40044a0 con 0x7ff3040a01b0 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: dump: 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.932 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: _finish_auth 0 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: _check_auth_tickets 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 1 -- 172.21.15.47:0/578022783 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f40036a0 con 0x7ff3040a01b0 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 1 -- 172.21.15.47:0/578022783 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2f4003c80 con 0x7ff3040a01b0 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 5 monclient: authenticate success, global_id 23627 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 4 set_mon_vals no callback set 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff3088c0700 10 monclient: dump: 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.933 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 10 monclient: shutdown 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:09.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 1 -- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 msgr2=0x7ff3040a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.919+0000 7ff30bb26700 1 --2- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 0x7ff3040a2950 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009920 tx=0x7ff2f4009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 1 -- 172.21.15.47:0/578022783 shutdown_connections 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 1 --2- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3040a0830 0x7ff3040a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 1 --2- 172.21.15.47:0/578022783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3040a01b0 0x7ff3040a2950 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 1 -- 172.21.15.47:0/578022783 >> 172.21.15.47:0/578022783 conn(0x7ff3040a7570 msgr2=0x7ff3040a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 1 -- 172.21.15.47:0/578022783 shutdown_connections 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 1 -- 172.21.15.47:0/578022783 wait complete. 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 10 monclient: build_initial_monmap 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.920+0000 7ff30bb26700 10 monclient: monmap: 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:09.935 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 librados: starting msgr at 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 librados: starting objecter 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 Processor -- start 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 -- start start 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 librados: setting wanted keys 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 librados: calling monclient init 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient: init 2024-04-20T02:20:09.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient: _add_conns ranks=[6,3,7,4,0,1,8,5,2] 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3040a01b0 0x7ff3041cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): picked mon.noname-d con 0x7ff3040a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff3040a0830 0x7ff3041bcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): picked mon.noname-g con 0x7ff3040a0830 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff3040a1150 0x7ff3041c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): picked mon.noname-i con 0x7ff3040a1150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041aa690 con 0x7ff3040a0830 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041a7b90 con 0x7ff3040a01b0 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041aa340 con 0x7ff3040a1150 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff30bb26700 10 monclient(hunting): authenticate will time out at 1889.322816s 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3040a01b0 0x7ff3041cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3040a01b0 0x7ff3041cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41160/0 (socket says 172.21.15.47:41160) 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 1 -- 172.21.15.47:0/2284064362 learned_addr learned my addr 172.21.15.47:0/2284064362 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 10 monclient(hunting): get_auth_request con 0x7ff3040a01b0 auth_method 0 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.921+0000 7ff3098c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff3040a0830 0x7ff3041bcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff3040a1150 0x7ff3041c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): get_auth_request con 0x7ff3040a0830 auth_method 0 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): get_auth_request con 0x7ff3040a1150 auth_method 0 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30a0c3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 10 monclient(hunting): handle_auth_done global_id 44205 payload 824 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff3040a1150 msgr2=0x7ff3041c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff3040a1150 0x7ff3041c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff3040a0830 msgr2=0x7ff3041bcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff3040a0830 0x7ff3041bcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:09.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 10 monclient: _finish_hunting 0 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 monclient: found mon.noname-d 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000094a0 con 0x7ff3040a01b0 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff3098c2700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3040a01b0 0x7ff3041cc310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff30000ec10 tx=0x7ff30000ef20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 client.?.objecter ms_handle_connect 0x7ff3040a01b0 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff30000ccc0 con 0x7ff3040a01b0 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: dump: 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.940 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: _finish_auth 0 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:09.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30000ce20 con 0x7ff3040a01b0 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 5 monclient: authenticate success, global_id 44205 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 10 monclient: _renew_subs 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f4009550 con 0x7ff3040a01b0 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3000138e0 con 0x7ff3040a01b0 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fbfff700 4 set_mon_vals no callback set 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 10 client.44205.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff2fa7fc700 10 monclient: dump: 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.942 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 10 monclient: _renew_subs 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3041aa060 con 0x7ff3040a01b0 2024-04-20T02:20:09.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.922+0000 7ff30bb26700 1 librados: init done 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff30bb26700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff30bb26700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 0x7ff3041c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff304068310 con 0x7ff30406aac0 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff300013470 con 0x7ff3040a01b0 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 0x7ff3041c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient: get_auth_request con 0x7ff30406aac0 auth_method 0 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff2e803a2f0 0x7ff2e803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7277+0+0 (secure 0 0 0) 0x7ff30004ad30 con 0x7ff3040a01b0 2024-04-20T02:20:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 10 client.44205.objecter ms_dispatch 0x7ff304001550 osd_map(196..196 src has 1..196) v4 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 3 client.44205.objecter handle_osd_map got epochs [196,196] > 0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 3 client.44205.objecter handle_osd_map decoding full epoch 196 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 10 monclient(hunting): handle_auth_done global_id 23636 payload 824 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff2fa7fc700 20 client.44205.objecter dump_active .. 0 homeless 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.923+0000 7ff3090c1700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff30409f1c0 con 0x7ff30406aac0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff30a0c3700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff2e803a2f0 0x7ff2e803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff30a0c3700 10 monclient: get_auth_request con 0x7ff2e803a2f0 auth_method 0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 client.44205.objecter ms_handle_connect 0x7ff30406aac0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 client.44205.objecter resend_mon_ops 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff3090c1700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 0x7ff3041c9350 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a05d0 tx=0x7ff2f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2f4020070 con 0x7ff30406aac0 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 monclient: dump: 2024-04-20T02:20:09.945 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.946 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 client.44205.objecter ms_handle_connect 0x7ff2e803a2f0 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff30a0c3700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff2e803a2f0 0x7ff2e803c7b0 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7ff3041cce00 tx=0x7ff2fc00d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff2f4003c10 con 0x7ff30406aac0 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 msgr2=0x7ff3041c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 0x7ff3041c9350 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a05d0 tx=0x7ff2f4004750 comp rx=0 tx=0).stop 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 msgr2=0x7ff3041c9350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:09.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.924+0000 7ff2fa7fc700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff30406aac0 0x7ff3041c9350 secure :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a05d0 tx=0x7ff2f4004750 comp rx=0 tx=0).stop 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff30bb26700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff30bb26700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 0x7ff3041c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff30bb26700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30406a400 con 0x7ff3041bd180 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 0x7ff3041c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient: get_auth_request con 0x7ff3041bd180 auth_method 0 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:09.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 10 monclient(hunting): handle_auth_done global_id 23645 payload 824 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.932+0000 7ff3090c1700 1 -- 172.21.15.47:0/2284064362 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3040a4450 con 0x7ff3041bd180 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff3090c1700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 0x7ff3041c9890 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 client.44205.objecter ms_handle_connect 0x7ff3041bd180 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 client.44205.objecter resend_mon_ops 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2f40291e0 con 0x7ff3041bd180 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 monclient: dump: 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:09.950 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7ff2f402d070 con 0x7ff3041bd180 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 msgr2=0x7ff3041c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 0x7ff3041c9890 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4011000 comp rx=0 tx=0).stop 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 msgr2=0x7ff3041c9890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff2fa7fc700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3041bd180 0x7ff3041c9890 secure :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4011000 comp rx=0 tx=0).stop 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:09.952 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:09.953 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:09.954 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:09.955 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:09.956 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.958 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.961 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:09.967 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:09.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:09.971 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:09.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:09.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:09.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:09.973 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:09.973 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 10 librados: watch_flush enter 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 10 librados: watch_flush exit 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 20 client.44205.objecter shutdown clearing up homeless session... 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 10 client.44205.objecter successfully canceled tick 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff2e803a2f0 msgr2=0x7ff2e803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff2e803a2f0 0x7ff2e803c7b0 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 crypto rx=0x7ff3041cce00 tx=0x7ff2fc00d040 comp rx=0 tx=0).stop 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 10 monclient: shutdown 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3040a01b0 msgr2=0x7ff3041cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 1 --2- 172.21.15.47:0/2284064362 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3040a01b0 0x7ff3041cc310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff30000ec10 tx=0x7ff30000ef20 comp rx=0 tx=0).stop 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30a0c3700 1 -- 172.21.15.47:0/2284064362 reap_dead start 2024-04-20T02:20:09.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30a0c3700 1 -- 172.21.15.47:0/2284064362 reap_dead start 2024-04-20T02:20:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 shutdown_connections 2024-04-20T02:20:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.933+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 >> 172.21.15.47:0/2284064362 conn(0x7ff3040a7570 msgr2=0x7ff304071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.934+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 shutdown_connections 2024-04-20T02:20:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.934+0000 7ff30bb26700 1 -- 172.21.15.47:0/2284064362 wait complete. 2024-04-20T02:20:09.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:09.934+0000 7ff30bb26700 1 librados: shutdown 2024-04-20T02:20:09.976 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:09.976 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:20:09.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.115+0000 7fbb283cc700 10 monclient: get_monmap_and_config 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.115+0000 7fbb283cc700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.115+0000 7fbb283cc700 10 monclient: monmap: 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:10.116557+0000 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:10.116557+0000 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:10.115 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:10.116 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 Processor -- start 2024-04-20T02:20:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 -- start start 2024-04-20T02:20:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient: init 2024-04-20T02:20:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient: _add_conns ranks=[5,7,0,10,1,6,8,3,2,4,9] 2024-04-20T02:20:10.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbb200a50f0 0x7fbb20137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): picked mon.noname-h con 0x7fbb200a50f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb201378a0 0x7fbb20138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): picked mon.noname-i con 0x7fbb201378a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbb200a38f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb200751e0 con 0x7fbb200a38f0 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200a54d0 con 0x7fbb200a50f0 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201387b0 con 0x7fbb201378a0 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.116+0000 7fbb283cc700 10 monclient(hunting): authenticate will time out at 1889.517900s 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26969700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbb200a38f0 0x7fbb200a3d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50824/0 (socket says 172.21.15.47:50824) 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26969700 1 -- 172.21.15.47:0/2145822028 learned_addr learned my addr 172.21.15.47:0/2145822028 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb201378a0 0x7fbb20138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbb200a50f0 0x7fbb20137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): get_auth_request con 0x7fbb201378a0 auth_method 0 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): get_auth_request con 0x7fbb200a50f0 auth_method 0 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 1 -- 172.21.15.47:0/2145822028 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268771552 0 0) 0x7fbb200751e0 con 0x7fbb200a38f0 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 10 monclient(hunting): _negotiate 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 10 monclient(hunting): my global_id is 34974 2024-04-20T02:20:10.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 1 -- 172.21.15.47:0/2145822028 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb0c0035b0 con 0x7fbb200a38f0 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient(hunting): handle_auth_done global_id 44233 payload 274 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 -- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbb200a50f0 msgr2=0x7fbb20137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbb200a50f0 0x7fbb20137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 -- 172.21.15.47:0/2145822028 >> v1:172.21.15.47:6789/0 conn(0x7fbb200a38f0 legacy=0x7fbb200a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 monclient: found mon.noname-i 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:20:10.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 1 -- 172.21.15.47:0/2145822028 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 255132368 0 0) 0x7fbb0c0035b0 con 0x7fbb200a38f0 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 -- 172.21.15.47:0/2145822028 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2006da20 con 0x7fbb201378a0 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25166700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.117+0000 7fbb25967700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb201378a0 0x7fbb20138270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009870 tx=0x7fbb10009b80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4f83012b0578af96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 1 -- 172.21.15.47:0/2145822028 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb18001c60 con 0x7fbb201378a0 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: dump: 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.131 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 1 -- 172.21.15.47:0/2145822028 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb100044a0 con 0x7fbb201378a0 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 5 monclient: authenticate success, global_id 44233 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:10.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 1 -- 172.21.15.47:0/2145822028 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb10003b10 con 0x7fbb201378a0 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 4 set_mon_vals no callback set 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb25166700 10 monclient: dump: 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 10 monclient: shutdown 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 -- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb201378a0 msgr2=0x7fbb20138270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb201378a0 0x7fbb20138270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009870 tx=0x7fbb10009b80 comp rx=0 tx=0).stop 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 -- 172.21.15.47:0/2145822028 shutdown_connections 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbb200a50f0 0x7fbb20137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 --2- 172.21.15.47:0/2145822028 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb201378a0 0x7fbb20138270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 -- 172.21.15.47:0/2145822028 >> 172.21.15.47:0/2145822028 conn(0x7fbb200a8510 msgr2=0x7fbb200a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 -- 172.21.15.47:0/2145822028 shutdown_connections 2024-04-20T02:20:10.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.118+0000 7fbb283cc700 1 -- 172.21.15.47:0/2145822028 wait complete. 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 10 monclient: monmap: 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.135 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 1 librados: starting msgr at 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 1 librados: starting objecter 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 1 Processor -- start 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 1 -- start start 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 1 librados: setting wanted keys 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 1 librados: calling monclient init 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.119+0000 7fbb283cc700 10 monclient: init 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient: _add_conns ranks=[7,1,6,5,2,3,8,4,0] 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb200a38f0 0x7fbb20136b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): picked mon.noname-i con 0x7fbb200a38f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200a50f0 0x7fbb20137050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): picked mon.noname-f con 0x7fbb200a50f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbb201cc330 0x7fbb201cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): picked mon.noname-d con 0x7fbb201cc330 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20122950 con 0x7fbb200a50f0 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200722d0 con 0x7fbb201cc330 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20071e80 con 0x7fbb200a38f0 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb283cc700 10 monclient(hunting): authenticate will time out at 1889.521408s 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbb201cc330 0x7fbb201cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200a50f0 0x7fbb20137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb200a38f0 0x7fbb20136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbb201cc330 0x7fbb201cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41162/0 (socket says 172.21.15.47:41162) 2024-04-20T02:20:10.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 -- 172.21.15.47:0/1183053810 learned_addr learned my addr 172.21.15.47:0/1183053810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): get_auth_request con 0x7fbb201cc330 auth_method 0 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): get_auth_request con 0x7fbb200a38f0 auth_method 0 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): get_auth_request con 0x7fbb200a50f0 auth_method 0 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb25967700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient(hunting): handle_auth_done global_id 44214 payload 824 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb200a38f0 msgr2=0x7fbb20136b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbb200a38f0 0x7fbb20136b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200a50f0 msgr2=0x7fbb20137050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb200a50f0 0x7fbb20137050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 monclient: found mon.noname-d 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.120+0000 7fbb26969700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100094a0 con 0x7fbb201cc330 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb26969700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbb201cc330 0x7fbb201cc790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800cc30 tx=0x7fbb1800cf40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 client.?.objecter ms_handle_connect 0x7fbb201cc330 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb1800acd0 con 0x7fbb201cc330 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: dump: 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.140 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 5 monclient: authenticate success, global_id 44214 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800ae70 con 0x7fbb201cc330 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb18007540 con 0x7fbb201cc330 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 10 monclient: _renew_subs 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb20122100 con 0x7fbb201cc330 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb24965700 4 set_mon_vals no callback set 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 10 client.44214.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb16ffd700 10 monclient: dump: 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.142 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 10 monclient: _renew_subs 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb20123f10 con 0x7fbb201cc330 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.121+0000 7fbb283cc700 1 librados: init done 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbb180070d0 con 0x7fbb201cc330 2024-04-20T02:20:10.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb283cc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb283cc700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 0x7fbb2006aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200682f0 con 0x7fbb201378a0 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 0x7fbb2006aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient: get_auth_request con 0x7fbb201378a0 auth_method 0 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbb0403a2f0 0x7fbb0403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 7277+0+0 (secure 0 0 0) 0x7fbb1804a8a0 con 0x7fbb201cc330 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 10 client.44214.objecter ms_dispatch 0x7fbb20001550 osd_map(196..196 src has 1..196) v4 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 3 client.44214.objecter handle_osd_map got epochs [196,196] > 0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 3 client.44214.objecter handle_osd_map decoding full epoch 196 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 10 monclient(hunting): handle_auth_done global_id 44167 payload 824 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 20 client.44214.objecter dump_active .. 0 homeless 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb2006e370 con 0x7fbb201378a0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb25967700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbb0403a2f0 0x7fbb0403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb25967700 10 monclient: get_auth_request con 0x7fbb0403a2f0 auth_method 0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb26168700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 0x7fbb2006aac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0119d0 tx=0x7fbb1c011d90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 10 client.44214.objecter ms_handle_connect 0x7fbb201378a0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 10 client.44214.objecter resend_mon_ops 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb1c0036a0 con 0x7fbb201378a0 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:10.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.122+0000 7fbb16ffd700 10 monclient: dump: 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.146 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 10 client.44214.objecter ms_handle_connect 0x7fbb0403a2f0 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb25967700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbb0403a2f0 0x7fbb0403c7b0 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009870 tx=0x7fbb10008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbb1c004240 con 0x7fbb201378a0 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 msgr2=0x7fbb2006aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 0x7fbb2006aac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0119d0 tx=0x7fbb1c011d90 comp rx=0 tx=0).stop 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 msgr2=0x7fbb2006aac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.123+0000 7fbb16ffd700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201378a0 0x7fbb2006aac0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0119d0 tx=0x7fbb1c011d90 comp rx=0 tx=0).stop 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb283cc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb283cc700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 0x7fbb2009f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb283cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2006a400 con 0x7fbb201cccd0 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 0x7fbb2009f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient: get_auth_request con 0x7fbb201cccd0 auth_method 0 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 10 monclient(hunting): handle_auth_done global_id 44176 payload 824 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.131+0000 7fbb26168700 1 -- 172.21.15.47:0/1183053810 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb2006e650 con 0x7fbb201cccd0 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.132+0000 7fbb26168700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 0x7fbb2009f590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c01b000 tx=0x7fbb1c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.132+0000 7fbb16ffd700 10 client.44214.objecter ms_handle_connect 0x7fbb201cccd0 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.132+0000 7fbb16ffd700 10 client.44214.objecter resend_mon_ops 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb1c00fd90 con 0x7fbb201cccd0 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 10 monclient: dump: 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.150 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fbb1c029400 con 0x7fbb201cccd0 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:10.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 msgr2=0x7fbb2009f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 0x7fbb2009f590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c01b000 tx=0x7fbb1c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 msgr2=0x7fbb2009f590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb16ffd700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbb201cccd0 0x7fbb2009f590 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c01b000 tx=0x7fbb1c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:10.152 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.153 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.154 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:10.155 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.156 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:10.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.158 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:10.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.161 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:10.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.164 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:10.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.167 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.170 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:10.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:10.172 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:10.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 10 librados: watch_flush enter 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 10 librados: watch_flush exit 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 20 client.44214.objecter shutdown clearing up homeless session... 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 10 client.44214.objecter successfully canceled tick 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbb0403a2f0 msgr2=0x7fbb0403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbb0403a2f0 0x7fbb0403c7b0 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009870 tx=0x7fbb10008030 comp rx=0 tx=0).stop 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 10 monclient: shutdown 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbb201cc330 msgr2=0x7fbb201cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 --2- 172.21.15.47:0/1183053810 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fbb201cc330 0x7fbb201cc790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800cc30 tx=0x7fbb1800cf40 comp rx=0 tx=0).stop 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb26969700 1 -- 172.21.15.47:0/1183053810 reap_dead start 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb26969700 1 -- 172.21.15.47:0/1183053810 reap_dead start 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 shutdown_connections 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 >> 172.21.15.47:0/1183053810 conn(0x7fbb200a8510 msgr2=0x7fbb200a7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.135+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 shutdown_connections 2024-04-20T02:20:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.136+0000 7fbb283cc700 1 -- 172.21.15.47:0/1183053810 wait complete. 2024-04-20T02:20:10.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.136+0000 7fbb283cc700 1 librados: shutdown 2024-04-20T02:20:10.175 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:10.175 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:20:10.175 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.313+0000 7f2165683700 10 monclient: get_monmap_and_config 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.313+0000 7f2165683700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.313+0000 7f2165683700 10 monclient: monmap: 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:10.314793+0000 2024-04-20T02:20:10.313 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:10.314793+0000 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:10.314 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 Processor -- start 2024-04-20T02:20:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 -- start start 2024-04-20T02:20:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient: init 2024-04-20T02:20:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient: _add_conns ranks=[9,10,2,1,0,6,7,4,5,3,8] 2024-04-20T02:20:10.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2160137d90 0x7f21600a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): picked mon.noname-a con 0x7f2160137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2160138170 0x7f21600a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): picked mon.noname-f con 0x7f2160138170 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 0x7f21600a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): picked mon.noname-b con 0x7f21601385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216009fa40 con 0x7f2160137d90 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216009dfc0 con 0x7f2160138170 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600a7bc0 con 0x7f21601385b0 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f2165683700 10 monclient(hunting): authenticate will time out at 1889.715878s 2024-04-20T02:20:10.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.314+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 0x7f21600a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 0x7f21600a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:37012/0 (socket says 172.21.15.47:37012) 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2160138170 0x7f21600a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 -- 172.21.15.47:0/3720803206 learned_addr learned my addr 172.21.15.47:0/3720803206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): get_auth_request con 0x7f21601385b0 auth_method 0 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 1 --2- 172.21.15.47:0/3720803206 >> v2:172.21.15.47:3300/0 conn(0x7f2160137d90 0x7f21600a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): get_auth_request con 0x7f2160137d90 auth_method 0 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): get_auth_request con 0x7f2160138170 auth_method 0 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient(hunting): handle_auth_done global_id 23654 payload 274 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 -- 172.21.15.47:0/3720803206 >> v2:172.21.15.179:3300/0 conn(0x7f2160138170 msgr2=0x7f21600a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 --2- 172.21.15.47:0/3720803206 >> v2:172.21.15.179:3300/0 conn(0x7f2160138170 0x7f21600a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 -- 172.21.15.47:0/3720803206 >> v2:172.21.15.47:3300/0 conn(0x7f2160137d90 msgr2=0x7f21600a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 --2- 172.21.15.47:0/3720803206 >> v2:172.21.15.47:3300/0 conn(0x7f2160137d90 0x7f21600a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 monclient: found mon.noname-b 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 -- 172.21.15.47:0/3720803206 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600751e0 con 0x7f21601385b0 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215effd700 1 --2- 172.21.15.47:0/3720803206 >> v2:172.21.15.47:3300/0 conn(0x7f2160137d90 0x7f21600a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215f7fe700 1 --2- 172.21.15.47:0/3720803206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 0x7f21600a61d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f2158009e50 tx=0x7f215800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2f3ef428dfbe20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 1 -- 172.21.15.47:0/3720803206 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f215800e070 con 0x7f21601385b0 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: dump: 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.329 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 1 -- 172.21.15.47:0/3720803206 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21580099b0 con 0x7f21601385b0 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f2165683700 5 monclient: authenticate success, global_id 23654 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f2165683700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f2165683700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 1 -- 172.21.15.47:0/3720803206 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f215800bd40 con 0x7f21601385b0 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f2165683700 4 set_mon_vals no callback set 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.315+0000 7f215dffb700 10 monclient: dump: 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.331 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 10 monclient: shutdown 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 -- 172.21.15.47:0/3720803206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 msgr2=0x7f21600a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 --2- 172.21.15.47:0/3720803206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 0x7f21600a61d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f2158009e50 tx=0x7f215800b330 comp rx=0 tx=0).stop 2024-04-20T02:20:10.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 -- 172.21.15.47:0/3720803206 shutdown_connections 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 --2- 172.21.15.47:0/3720803206 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f21601385b0 0x7f21600a61d0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 --2- 172.21.15.47:0/3720803206 >> v2:172.21.15.47:3300/0 conn(0x7f2160137d90 0x7f21600a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 --2- 172.21.15.47:0/3720803206 >> v2:172.21.15.179:3300/0 conn(0x7f2160138170 0x7f21600a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 -- 172.21.15.47:0/3720803206 >> 172.21.15.47:0/3720803206 conn(0x7f21600aa8b0 msgr2=0x7f21600a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 -- 172.21.15.47:0/3720803206 shutdown_connections 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.316+0000 7f2165683700 1 -- 172.21.15.47:0/3720803206 wait complete. 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 10 monclient: monmap: 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:10.333 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 1 librados: starting msgr at 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 1 librados: starting objecter 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 1 Processor -- start 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 1 -- start start 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 1 librados: setting wanted keys 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 1 librados: calling monclient init 2024-04-20T02:20:10.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.317+0000 7f2165683700 10 monclient: init 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient: _add_conns ranks=[3,5,6,4,8,7,1,2,0] 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2160137d90 0x7f21601c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): picked mon.noname-g con 0x7f2160137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2160138170 0x7f2160136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): picked mon.noname-h con 0x7f2160138170 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 0x7f21601374a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): picked mon.noname-d con 0x7f21601385b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2160124760 con 0x7f2160137d90 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f216009ea70 con 0x7f2160138170 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f21601c9860 con 0x7f21601385b0 2024-04-20T02:20:10.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f2165683700 10 monclient(hunting): authenticate will time out at 1889.719247s 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 0x7f21601374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 0x7f21601374a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41168/0 (socket says 172.21.15.47:41168) 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 -- 172.21.15.47:0/583087566 learned_addr learned my addr 172.21.15.47:0/583087566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): get_auth_request con 0x7f21601385b0 auth_method 0 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2160137d90 0x7f21601c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215e7fc700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2160138170 0x7f2160136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): get_auth_request con 0x7f2160137d90 auth_method 0 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215e7fc700 10 monclient(hunting): get_auth_request con 0x7f2160138170 auth_method 0 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient(hunting): handle_auth_done global_id 44223 payload 824 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2160138170 msgr2=0x7f2160136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2160138170 0x7f2160136f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2160137d90 msgr2=0x7f21601c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2160137d90 0x7f21601c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 monclient: found mon.noname-d 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21500094a0 con 0x7f21601385b0 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215e7fc700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2160138170 0x7f2160136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f215f7fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 0x7f21601374a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21600a7d00 tx=0x7f2158004ad0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f21477fe700 10 client.?.objecter ms_handle_connect 0x7f21601385b0 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.318+0000 7f21477fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f21580092e0 con 0x7f21601385b0 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: dump: 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.338 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158003e10 con 0x7f21601385b0 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 5 monclient: authenticate success, global_id 44223 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 10 monclient: _renew_subs 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2158009b30 con 0x7f21601385b0 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f21580046c0 con 0x7f21601385b0 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f21477fe700 10 monclient: dump: 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.340 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f215d7fa700 4 set_mon_vals no callback set 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 10 client.44223.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:10.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 10 monclient: _renew_subs 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2160122570 con 0x7f21601385b0 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 1 librados: init done 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.319+0000 7f2165683700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 0x7f21601240a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2160068180 con 0x7f216006aaa0 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 0x7f21601240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2158004820 con 0x7f21601385b0 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient: get_auth_request con 0x7f216006aaa0 auth_method 0 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 10 monclient(hunting): handle_auth_done global_id 44232 payload 824 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f214c03a2f0 0x7f214c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f216009f760 con 0x7f216006aaa0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f215804bd70 con 0x7f21601385b0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 10 client.44223.objecter ms_dispatch 0x7f2160001550 osd_map(197..197 src has 1..197) v4 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 3 client.44223.objecter handle_osd_map got epochs [197,197] > 0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 3 client.44223.objecter handle_osd_map decoding full epoch 197 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 20 client.44223.objecter dump_active .. 0 homeless 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215effd700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 0x7f21601240a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2150011bd0 tx=0x7f2150011f90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 10 client.44223.objecter ms_handle_connect 0x7f216006aaa0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 10 client.44223.objecter resend_mon_ops 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215e7fc700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f214c03a2f0 0x7f214c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f215e7fc700 10 monclient: get_auth_request con 0x7f214c03a2f0 auth_method 0 2024-04-20T02:20:10.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2150014440 con 0x7f216006aaa0 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.320+0000 7f21477fe700 10 monclient: dump: 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.344 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f215000fa90 con 0x7f216006aaa0 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 msgr2=0x7f21601240a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 0x7f21601240a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2150011bd0 tx=0x7f2150011f90 comp rx=0 tx=0).stop 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 msgr2=0x7f21601240a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f216006aaa0 0x7f21601240a0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2150011bd0 tx=0x7f2150011f90 comp rx=0 tx=0).stop 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f21477fe700 10 client.44223.objecter ms_handle_connect 0x7f214c03a2f0 2024-04-20T02:20:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.321+0000 7f215e7fc700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f214c03a2f0 0x7f214c03c7b0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7f2160137c70 tx=0x7f2154008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.328+0000 7f2165683700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:10.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.328+0000 7f2165683700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 0x7f21601cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.328+0000 7f2165683700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.328+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f216006a400 con 0x7f21601cc190 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 0x7f21601cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient: get_auth_request con 0x7f21601cc190 auth_method 0 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 10 monclient(hunting): handle_auth_done global_id 44241 payload 824 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 1 -- 172.21.15.47:0/583087566 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f21601248a0 con 0x7f21601cc190 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f215effd700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 0x7f21601cc5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2150000c00 tx=0x7f2150004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f21477fe700 10 client.44223.objecter ms_handle_connect 0x7f21601cc190 2024-04-20T02:20:10.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f21477fe700 10 client.44223.objecter resend_mon_ops 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f215000fe80 con 0x7f21601cc190 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f21477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f21477fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.329+0000 7f21477fe700 10 monclient: dump: 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.349 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f2150004030 con 0x7f21601cc190 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 msgr2=0x7f21601cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 0x7f21601cc5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2150000c00 tx=0x7f2150004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 msgr2=0x7f21601cc5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f21477fe700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601cc190 0x7f21601cc5a0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2150000c00 tx=0x7f2150004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.350 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:10.351 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.352 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:10.354 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.355 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.358 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.360 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.361 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.363 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.366 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.369 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:10.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:10.371 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 10 librados: watch_flush enter 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 10 librados: watch_flush exit 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 20 client.44223.objecter shutdown clearing up homeless session... 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 10 client.44223.objecter successfully canceled tick 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f214c03a2f0 msgr2=0x7f214c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f214c03a2f0 0x7f214c03c7b0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7f2160137c70 tx=0x7f2154008040 comp rx=0 tx=0).stop 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 10 monclient: shutdown 2024-04-20T02:20:10.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 msgr2=0x7f21601374a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 0x7f21601374a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21600a7d00 tx=0x7f2158004ad0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f215f7fe700 1 -- 172.21.15.47:0/583087566 reap_dead start 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 shutdown_connections 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 --2- 172.21.15.47:0/583087566 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f21601385b0 0x7f21601374a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 >> 172.21.15.47:0/583087566 conn(0x7f21600aa8b0 msgr2=0x7f21601bf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 shutdown_connections 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 -- 172.21.15.47:0/583087566 wait complete. 2024-04-20T02:20:10.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.330+0000 7f2165683700 1 librados: shutdown 2024-04-20T02:20:10.374 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:10.374 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:20:10.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.514+0000 7f4d55471700 10 monclient: get_monmap_and_config 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.514+0000 7f4d55471700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.515+0000 7f4d55471700 10 monclient: monmap: 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:10.516143+0000 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:10.516143+0000 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.515+0000 7f4d55471700 1 Processor -- start 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.515+0000 7f4d55471700 1 -- start start 2024-04-20T02:20:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.515+0000 7f4d55471700 10 monclient: init 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient: _add_conns ranks=[10,6,5,0,8,1,4,2,3,7,9] 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4d50137d90 0x7f4d500a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): picked mon.noname-f con 0x7f4d50137d90 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 0x7f4d500a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): picked mon.noname-d con 0x7f4d50138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4d501385b0 0x7f4d500a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): picked mon.noname-h con 0x7f4d501385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5009fa40 con 0x7f4d50137d90 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5009dfc0 con 0x7f4d501385b0 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500a7bc0 con 0x7f4d50138170 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d55471700 10 monclient(hunting): authenticate will time out at 1889.917206s 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 0x7f4d500a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 0x7f4d500a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41188/0 (socket says 172.21.15.47:41188) 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 -- 172.21.15.47:0/1954967567 learned_addr learned my addr 172.21.15.47:0/1954967567 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): get_auth_request con 0x7f4d50138170 auth_method 0 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4f7fe700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4d501385b0 0x7f4d500a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4effd700 1 --2- 172.21.15.47:0/1954967567 >> v2:172.21.15.179:3300/0 conn(0x7f4d50137d90 0x7f4d500a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4f7fe700 10 monclient(hunting): get_auth_request con 0x7f4d501385b0 auth_method 0 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4effd700 10 monclient(hunting): get_auth_request con 0x7f4d50137d90 auth_method 0 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_done global_id 44250 payload 274 2024-04-20T02:20:10.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 -- 172.21.15.47:0/1954967567 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4d501385b0 msgr2=0x7f4d500a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4d501385b0 0x7f4d500a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 -- 172.21.15.47:0/1954967567 >> v2:172.21.15.179:3300/0 conn(0x7f4d50137d90 msgr2=0x7f4d500a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1954967567 >> v2:172.21.15.179:3300/0 conn(0x7f4d50137d90 0x7f4d500a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 monclient: found mon.noname-d 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 -- 172.21.15.47:0/1954967567 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500751e0 con 0x7f4d50138170 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4f7fe700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4d501385b0 0x7f4d500a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.516+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 0x7f4d500a5c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009920 tx=0x7f4d38009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b84863f85017d98d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 1 -- 172.21.15.47:0/1954967567 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d38009e70 con 0x7f4d50138170 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: dump: 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.531 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 1 -- 172.21.15.47:0/1954967567 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3800bc90 con 0x7f4d50138170 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 5 monclient: authenticate success, global_id 44250 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 1 -- 172.21.15.47:0/1954967567 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d3800be00 con 0x7f4d50138170 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d4dffb700 10 monclient: dump: 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 4 set_mon_vals no callback set 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 10 monclient: shutdown 2024-04-20T02:20:10.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 -- 172.21.15.47:0/1954967567 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 msgr2=0x7f4d500a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 0x7f4d500a5c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009920 tx=0x7f4d38009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 -- 172.21.15.47:0/1954967567 shutdown_connections 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 --2- 172.21.15.47:0/1954967567 >> v2:172.21.15.179:3300/0 conn(0x7f4d50137d90 0x7f4d500a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4d501385b0 0x7f4d500a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 --2- 172.21.15.47:0/1954967567 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4d50138170 0x7f4d500a5c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 -- 172.21.15.47:0/1954967567 >> 172.21.15.47:0/1954967567 conn(0x7f4d500aa8b0 msgr2=0x7f4d500a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 -- 172.21.15.47:0/1954967567 shutdown_connections 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.517+0000 7f4d55471700 1 -- 172.21.15.47:0/1954967567 wait complete. 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.518+0000 7f4d55471700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.518+0000 7f4d55471700 10 monclient: monmap: 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:10.535 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.518+0000 7f4d55471700 1 librados: starting msgr at 2024-04-20T02:20:10.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.518+0000 7f4d55471700 1 librados: starting objecter 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 Processor -- start 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 -- start start 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 librados: setting wanted keys 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 librados: calling monclient init 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient: init 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient: _add_conns ranks=[2,3,7,6,4,5,0,1,8] 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4d50137d90 0x7f4d501cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): picked mon.noname-b con 0x7f4d50137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d50138170 0x7f4d50136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): picked mon.noname-g con 0x7f4d50138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4d501385b0 0x7f4d50137240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): picked mon.noname-i con 0x7f4d501385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50123e20 con 0x7f4d50137d90 2024-04-20T02:20:10.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50122d60 con 0x7f4d50138170 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5009e9e0 con 0x7f4d501385b0 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d55471700 10 monclient(hunting): authenticate will time out at 1889.920663s 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4d50137d90 0x7f4d501cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4d50137d90 0x7f4d501cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:37016/0 (socket says 172.21.15.47:37016) 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 1 -- 172.21.15.47:0/1643685491 learned_addr learned my addr 172.21.15.47:0/1643685491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): get_auth_request con 0x7f4d50137d90 auth_method 0 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4f7fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4d501385b0 0x7f4d50137240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d50138170 0x7f4d50136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4f7fe700 10 monclient(hunting): get_auth_request con 0x7f4d501385b0 auth_method 0 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4e7fc700 10 monclient(hunting): get_auth_request con 0x7f4d50138170 auth_method 0 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.519+0000 7f4d4effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 10 monclient(hunting): handle_auth_done global_id 23663 payload 824 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4d501385b0 msgr2=0x7f4d50137240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4d501385b0 0x7f4d50137240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d50138170 msgr2=0x7f4d50136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4d50138170 0x7f4d50136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 monclient: found mon.noname-b 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38009550 con 0x7f4d50137d90 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4effd700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4d50137d90 0x7f4d501cc370 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000e840 tx=0x7f4d4000eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 client.?.objecter ms_handle_connect 0x7f4d50137d90 2024-04-20T02:20:10.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d4000c070 con 0x7f4d50137d90 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: dump: 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.541 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 5 monclient: authenticate success, global_id 23663 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d40013880 con 0x7f4d50137d90 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 10 monclient: _renew_subs 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5009dc20 con 0x7f4d50137d90 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d40013e60 con 0x7f4d50137d90 2024-04-20T02:20:10.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d4d7fa700 4 set_mon_vals no callback set 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 10 client.23663.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d477fe700 10 monclient: dump: 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.543 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 10 monclient: _renew_subs 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d50122380 con 0x7f4d50137d90 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.520+0000 7f4d55471700 1 librados: init done 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d55471700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d55471700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 0x7f4d501208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50068180 con 0x7f4d5006aac0 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4d400139f0 con 0x7f4d50137d90 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 0x7f4d501208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient: get_auth_request con 0x7f4d5006aac0 auth_method 0 2024-04-20T02:20:10.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d3003a2f0 0x7f4d3003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f4d4004a6c0 con 0x7f4d50137d90 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 10 client.23663.objecter ms_dispatch 0x7f4d50001550 osd_map(197..197 src has 1..197) v4 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 3 client.23663.objecter handle_osd_map got epochs [197,197] > 0 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 3 client.23663.objecter handle_osd_map decoding full epoch 197 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_done global_id 34811 payload 824 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4e7fc700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d5009f760 con 0x7f4d5006aac0 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d477fe700 20 client.23663.objecter dump_active .. 0 homeless 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4f7fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d3003a2f0 0x7f4d3003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.521+0000 7f4d4f7fe700 10 monclient: get_auth_request con 0x7f4d3003a2f0 auth_method 0 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 client.23663.objecter ms_handle_connect 0x7f4d5006aac0 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 client.23663.objecter resend_mon_ops 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 0x7f4d501208d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009fd0 tx=0x7f4d38004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d38020070 con 0x7f4d5006aac0 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 monclient: dump: 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.546 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 client.23663.objecter ms_handle_connect 0x7f4d3003a2f0 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d4f7fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d3003a2f0 0x7f4d3003c7b0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7f4d50138a20 tx=0x7f4d48009750 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4d38004250 con 0x7f4d5006aac0 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 msgr2=0x7f4d501208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 0x7f4d501208d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009fd0 tx=0x7f4d38004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 msgr2=0x7f4d501208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.522+0000 7f4d477fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d5006aac0 0x7f4d501208d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d55471700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d55471700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 0x7f4d50121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d55471700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5006a400 con 0x7f4d50120e10 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 0x7f4d50121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient: get_auth_request con 0x7f4d50120e10 auth_method 0 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 10 monclient(hunting): handle_auth_done global_id 34820 payload 824 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 1 -- 172.21.15.47:0/1643685491 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4d50121760 con 0x7f4d50120e10 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d4e7fc700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 0x7f4d50121220 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009840 tx=0x7f4d38011000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d477fe700 10 client.23663.objecter ms_handle_connect 0x7f4d50120e10 2024-04-20T02:20:10.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.530+0000 7f4d477fe700 10 client.23663.objecter resend_mon_ops 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4d3802bb50 con 0x7f4d50120e10 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 10 monclient: dump: 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.551 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f4d3802bcb0 con 0x7f4d50120e10 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 msgr2=0x7f4d50121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 0x7f4d50121220 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009840 tx=0x7f4d38011000 comp rx=0 tx=0).stop 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 msgr2=0x7f4d50121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d477fe700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4d50120e10 0x7f4d50121220 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009840 tx=0x7f4d38011000 comp rx=0 tx=0).stop 2024-04-20T02:20:10.552 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:10.553 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.554 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.555 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:10.556 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.557 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.559 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.562 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.565 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.565 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.565 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:10.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:10.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.568 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.570 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.574 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.576 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:10.576 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:10.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 10 librados: watch_flush enter 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 10 librados: watch_flush exit 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 20 client.23663.objecter shutdown clearing up homeless session... 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 10 client.23663.objecter successfully canceled tick 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d3003a2f0 msgr2=0x7f4d3003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d3003a2f0 0x7f4d3003c7b0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7f4d50138a20 tx=0x7f4d48009750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 10 monclient: shutdown 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4d50137d90 msgr2=0x7f4d501cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 --2- 172.21.15.47:0/1643685491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4d50137d90 0x7f4d501cc370 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000e840 tx=0x7f4d4000eb50 comp rx=0 tx=0).stop 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d4f7fe700 1 -- 172.21.15.47:0/1643685491 reap_dead start 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d4f7fe700 1 -- 172.21.15.47:0/1643685491 reap_dead start 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 shutdown_connections 2024-04-20T02:20:10.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 >> 172.21.15.47:0/1643685491 conn(0x7f4d500aa8b0 msgr2=0x7f4d500a58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 shutdown_connections 2024-04-20T02:20:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.531+0000 7f4d55471700 1 -- 172.21.15.47:0/1643685491 wait complete. 2024-04-20T02:20:10.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.532+0000 7f4d55471700 1 librados: shutdown 2024-04-20T02:20:10.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:10.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:20:10.579 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:20:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.718+0000 7f11366de700 10 monclient: get_monmap_and_config 2024-04-20T02:20:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.718+0000 7f11366de700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.718+0000 7f11366de700 10 monclient: monmap: 2024-04-20T02:20:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:10.719861+0000 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:10.719861+0000 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:10.720 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.719+0000 7f11366de700 1 Processor -- start 2024-04-20T02:20:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.719+0000 7f11366de700 1 -- start start 2024-04-20T02:20:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.719+0000 7f11366de700 10 monclient: init 2024-04-20T02:20:10.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient: _add_conns ranks=[3,6,5,8,4,1,9,2,7,10,0] 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11300a18a0 0x7f1130135e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): picked mon.noname-g con 0x7f11300a18a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 0x7f11300a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): picked mon.noname-d con 0x7f11300a1c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11300a2100 0x7f11300a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): picked mon.noname-h con 0x7f11300a2100 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1130135bd0 con 0x7f11300a18a0 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1130071eb0 con 0x7f11300a2100 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1130072190 con 0x7f11300a1c80 2024-04-20T02:20:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f11366de700 10 monclient(hunting): authenticate will time out at 1890.121169s 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 0x7f11300a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 0x7f11300a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:41190/0 (socket says 172.21.15.47:41190) 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 -- 172.21.15.47:0/4064532308 learned_addr learned my addr 172.21.15.47:0/4064532308 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): get_auth_request con 0x7f11300a1c80 auth_method 0 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11300a18a0 0x7f1130135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): get_auth_request con 0x7f11300a18a0 auth_method 0 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11300a2100 0x7f11300a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): get_auth_request con 0x7f11300a2100 auth_method 0 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f1134c7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient(hunting): handle_auth_done global_id 44259 payload 274 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 -- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11300a2100 msgr2=0x7f11300a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11300a2100 0x7f11300a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 -- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11300a18a0 msgr2=0x7f1130135e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11300a18a0 0x7f1130135e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 monclient: found mon.noname-d 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 -- 172.21.15.47:0/4064532308 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11300751e0 con 0x7f11300a1c80 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112f7fe700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 0x7f11300a3880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1118009920 tx=0x7f1118009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c641223fa35dc0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.720+0000 7f112effd700 1 -- 172.21.15.47:0/4064532308 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1118011aa0 con 0x7f11300a1c80 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: dump: 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 5 monclient: authenticate success, global_id 44259 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 1 -- 172.21.15.47:0/4064532308 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118011c00 con 0x7f11300a1c80 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 1 -- 172.21.15.47:0/4064532308 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1118011d70 con 0x7f11300a1c80 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 4 set_mon_vals no callback set 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f112effd700 10 monclient: dump: 2024-04-20T02:20:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 10 monclient: shutdown 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 -- 172.21.15.47:0/4064532308 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 msgr2=0x7f11300a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 0x7f11300a3880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1118009920 tx=0x7f1118009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 -- 172.21.15.47:0/4064532308 shutdown_connections 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11300a2100 0x7f11300a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11300a18a0 0x7f1130135e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 --2- 172.21.15.47:0/4064532308 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11300a1c80 0x7f11300a3880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 -- 172.21.15.47:0/4064532308 >> 172.21.15.47:0/4064532308 conn(0x7f11300a50b0 msgr2=0x7f1130066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 -- 172.21.15.47:0/4064532308 shutdown_connections 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.721+0000 7f11366de700 1 -- 172.21.15.47:0/4064532308 wait complete. 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.722+0000 7f11366de700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.722+0000 7f11366de700 10 monclient: monmap: 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.722+0000 7f11366de700 1 librados: starting msgr at 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.722+0000 7f11366de700 1 librados: starting objecter 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.722+0000 7f11366de700 1 Processor -- start 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 -- start start 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 librados: setting wanted keys 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 librados: calling monclient init 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient: init 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient: _add_conns ranks=[7,0,4,6,3,8,5,2,1] 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11300a18a0 0x7f1130122a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): picked mon.noname-i con 0x7f11300a18a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11300a1c80 0x7f113009b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): picked mon.noname-a con 0x7f11300a1c80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11300a2100 0x7f11301cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): picked mon.noname-c con 0x7f11300a2100 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130122f50 con 0x7f11300a1c80 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1130071450 con 0x7f11300a2100 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f113009ee50 con 0x7f11300a18a0 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f11366de700 10 monclient(hunting): authenticate will time out at 1890.124748s 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11300a1c80 0x7f113009b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11300a1c80 0x7f113009b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58504/0 (socket says 172.21.15.47:58504) 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 1 -- 172.21.15.47:0/1914795353 learned_addr learned my addr 172.21.15.47:0/1914795353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): get_auth_request con 0x7f11300a1c80 auth_method 0 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112ffff700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11300a18a0 0x7f1130122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112ffff700 10 monclient(hunting): get_auth_request con 0x7f11300a18a0 auth_method 0 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f112f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.723+0000 7f1134c7b700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11300a2100 0x7f11301cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): get_auth_request con 0x7f11300a2100 auth_method 0 2024-04-20T02:20:10.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f1134c7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 10 monclient(hunting): handle_auth_done global_id 34992 payload 824 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11300a18a0 msgr2=0x7f1130122a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11300a18a0 0x7f1130122a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11300a2100 msgr2=0x7f11301cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11300a2100 0x7f11301cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 monclient: found mon.noname-a 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:10.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1118009550 con 0x7f11300a1c80 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112f7fe700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11300a1c80 0x7f113009b650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1118000c00 tx=0x7f1118004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 client.?.objecter ms_handle_connect 0x7f11300a1c80 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1118004390 con 0x7f11300a1c80 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: dump: 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.745 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 5 monclient: authenticate success, global_id 34992 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118004550 con 0x7f11300a1c80 2024-04-20T02:20:10.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1118024470 con 0x7f11300a1c80 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112e7fc700 4 set_mon_vals no callback set 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f112cff9700 10 monclient: dump: 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 10 monclient: _renew_subs 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113009c2b0 con 0x7f11300a1c80 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 10 client.34992.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 10 monclient: _renew_subs 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113009dde0 con 0x7f11300a1c80 2024-04-20T02:20:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.724+0000 7f11366de700 1 librados: init done 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f11366de700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1118021070 con 0x7f11300a1c80 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f11366de700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 0x7f11301cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130068430 con 0x7f113006aac0 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112cff9700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f111003a2f0 0x7f111003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 0x7f11301cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient: get_auth_request con 0x7f113006aac0 auth_method 0 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7021+0+0 (secure 0 0 0) 0x7f111804bd20 con 0x7f11300a1c80 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112cff9700 10 client.34992.objecter ms_dispatch 0x7f1130001550 osd_map(197..197 src has 1..197) v4 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112cff9700 3 client.34992.objecter handle_osd_map got epochs [197,197] > 0 2024-04-20T02:20:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112cff9700 3 client.34992.objecter handle_osd_map decoding full epoch 197 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.725+0000 7f1134c7b700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f111003a2f0 0x7f111003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 20 client.34992.objecter dump_active .. 0 homeless 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f1134c7b700 10 monclient: get_auth_request con 0x7f111003a2f0 auth_method 0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112ffff700 10 monclient(hunting): handle_auth_done global_id 26425 payload 824 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112ffff700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1130070bb0 con 0x7f113006aac0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 client.34992.objecter ms_handle_connect 0x7f113006aac0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 client.34992.objecter resend_mon_ops 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112ffff700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 0x7f11301cc700 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f11200119d0 tx=0x7f1120011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 client.34992.objecter ms_handle_connect 0x7f111003a2f0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f1134c7b700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f111003a2f0 0x7f111003c7b0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7f11301cd2a0 tx=0x7f112800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11200036a0 con 0x7f113006aac0 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 monclient: dump: 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.751 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1120004240 con 0x7f113006aac0 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 msgr2=0x7f11301cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 0x7f11301cc700 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f11200119d0 tx=0x7f1120011d90 comp rx=0 tx=0).stop 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 msgr2=0x7f11301cc700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.726+0000 7f112cff9700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f113006aac0 0x7f11301cc700 secure :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f11200119d0 tx=0x7f1120011d90 comp rx=0 tx=0).stop 2024-04-20T02:20:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.734+0000 7f11366de700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.734+0000 7f11366de700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 0x7f113006c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.734+0000 7f11366de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.734+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113006a400 con 0x7f11301ccc40 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.734+0000 7f112ffff700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 0x7f113006c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient: get_auth_request con 0x7f11301ccc40 auth_method 0 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 10 monclient(hunting): handle_auth_done global_id 26434 payload 824 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 1 -- 172.21.15.47:0/1914795353 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1130067150 con 0x7f11301ccc40 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112cff9700 10 client.34992.objecter ms_handle_connect 0x7f11301ccc40 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112cff9700 10 client.34992.objecter resend_mon_ops 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112ffff700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 0x7f113006c760 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f112001b000 tx=0x7f1120004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f112000fd90 con 0x7f11301ccc40 2024-04-20T02:20:10.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112cff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.735+0000 7f112cff9700 10 monclient: dump: 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f11200283a0 con 0x7f11301ccc40 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 msgr2=0x7f113006c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 0x7f113006c760 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f112001b000 tx=0x7f1120004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 msgr2=0x7f113006c760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f112cff9700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11301ccc40 0x7f113006c760 secure :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f112001b000 tx=0x7f1120004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.757 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:10.758 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.760 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:10.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.762 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:10.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:10.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.768 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.770 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:10.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:10.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.777 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:10.778 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 10 librados: watch_flush enter 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 10 librados: watch_flush exit 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 20 client.34992.objecter shutdown clearing up homeless session... 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 10 client.34992.objecter successfully canceled tick 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f111003a2f0 msgr2=0x7f111003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f111003a2f0 0x7f111003c7b0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7f11301cd2a0 tx=0x7f112800b040 comp rx=0 tx=0).stop 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 10 monclient: shutdown 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11300a1c80 msgr2=0x7f113009b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 --2- 172.21.15.47:0/1914795353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11300a1c80 0x7f113009b650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1118000c00 tx=0x7f1118004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f1134c7b700 1 -- 172.21.15.47:0/1914795353 reap_dead start 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f1134c7b700 1 -- 172.21.15.47:0/1914795353 reap_dead start 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 shutdown_connections 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 >> 172.21.15.47:0/1914795353 conn(0x7f11300a50b0 msgr2=0x7f11300a3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 shutdown_connections 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 -- 172.21.15.47:0/1914795353 wait complete. 2024-04-20T02:20:10.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.736+0000 7f11366de700 1 librados: shutdown 2024-04-20T02:20:10.781 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:10.781 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:20:10.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.920+0000 7fcec84f0700 10 monclient: get_monmap_and_config 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.920+0000 7fcec84f0700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.920+0000 7fcec84f0700 10 monclient: monmap: 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:10.921434+0000 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:10.921434+0000 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.920 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:10.921 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.920+0000 7fcec84f0700 1 Processor -- start 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.920+0000 7fcec84f0700 1 -- start start 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.920+0000 7fcec84f0700 10 monclient: init 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient: _add_conns ranks=[3,9,8,7,5,1,2,4,6,10,0] 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec00a0780 0x7fcec00a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): picked mon.noname-g con 0x7fcec00a0780 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 0x7fcec00a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): picked mon.noname-a con 0x7fcec00a0b60 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 0x7fcec00a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): picked mon.noname-e con 0x7fcec00a2950 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:10.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec0067770 con 0x7fcec00a0b60 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec009eee0 con 0x7fcec00a0780 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcec009f1c0 con 0x7fcec00a2950 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec84f0700 10 monclient(hunting): authenticate will time out at 1890.322702s 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 0x7fcec00a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec5a8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 0x7fcec00a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 0x7fcec00a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41880/0 (socket says 172.21.15.47:41880) 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec5a8b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 0x7fcec00a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58512/0 (socket says 172.21.15.47:58512) 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 1 -- 172.21.15.47:0/2269141899 learned_addr learned my addr 172.21.15.47:0/2269141899 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): get_auth_request con 0x7fcec00a2950 auth_method 0 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec5a8b700 10 monclient(hunting): get_auth_request con 0x7fcec00a0b60 auth_method 0 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec5a8b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec5a8b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec5a8b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.921+0000 7fcec6a8d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec628c700 1 --2- 172.21.15.47:0/2269141899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec00a0780 0x7fcec00a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec628c700 10 monclient(hunting): get_auth_request con 0x7fcec00a0780 auth_method 0 2024-04-20T02:20:10.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec628c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec628c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec628c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 10 monclient(hunting): handle_auth_done global_id 34829 payload 274 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 -- 172.21.15.47:0/2269141899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec00a0780 msgr2=0x7fcec00a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/2269141899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec00a0780 0x7fcec00a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 -- 172.21.15.47:0/2269141899 >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 msgr2=0x7fcec00a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/2269141899 >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 0x7fcec00a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 monclient: found mon.noname-e 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 -- 172.21.15.47:0/2269141899 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00751e0 con 0x7fcec00a2950 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec5a8b700 1 --2- 172.21.15.47:0/2269141899 >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 0x7fcec00a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/2269141899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 0x7fcec00a2e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009e50 tx=0x7fceb800b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cd15d094720dab48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 1 -- 172.21.15.47:0/2269141899 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fceb800e070 con 0x7fcec00a2950 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: dump: 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 1 -- 172.21.15.47:0/2269141899 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb800bd80 con 0x7fcec00a2950 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec84f0700 5 monclient: authenticate success, global_id 34829 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec84f0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec84f0700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 1 -- 172.21.15.47:0/2269141899 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fceb8012860 con 0x7fcec00a2950 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec84f0700 4 set_mon_vals no callback set 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.922+0000 7fcec528a700 10 monclient: dump: 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.938 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 10 monclient: shutdown 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 -- 172.21.15.47:0/2269141899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 msgr2=0x7fcec00a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 --2- 172.21.15.47:0/2269141899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 0x7fcec00a2e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009e50 tx=0x7fceb800b330 comp rx=0 tx=0).stop 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 -- 172.21.15.47:0/2269141899 shutdown_connections 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 --2- 172.21.15.47:0/2269141899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcec00a2950 0x7fcec00a2e00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 --2- 172.21.15.47:0/2269141899 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec00a0780 0x7fcec00a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 --2- 172.21.15.47:0/2269141899 >> v2:172.21.15.47:3300/0 conn(0x7fcec00a0b60 0x7fcec00a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 -- 172.21.15.47:0/2269141899 >> 172.21.15.47:0/2269141899 conn(0x7fcec00a7570 msgr2=0x7fcec00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 -- 172.21.15.47:0/2269141899 shutdown_connections 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 1 -- 172.21.15.47:0/2269141899 wait complete. 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 10 monclient: build_initial_monmap 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.923+0000 7fcec84f0700 10 monclient: monmap: 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 librados: starting msgr at 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 librados: starting objecter 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 Processor -- start 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 -- start start 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 librados: setting wanted keys 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 librados: calling monclient init 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient: init 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient: _add_conns ranks=[4,2,1,3,0,5,7,8,6] 2024-04-20T02:20:10.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcec00a0780 0x7fcec01c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): picked mon.noname-c con 0x7fcec00a0780 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcec00a0b60 0x7fcec01bce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): picked mon.noname-b con 0x7fcec00a0b60 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec00a2950 0x7fcec01cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): picked mon.noname-f con 0x7fcec00a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec01ac8e0 con 0x7fcec00a2950 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec009e970 con 0x7fcec00a0b60 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcec009e440 con 0x7fcec00a0780 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec84f0700 10 monclient(hunting): authenticate will time out at 1890.325846s 2024-04-20T02:20:10.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcec00a0780 0x7fcec01c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec5a8b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcec00a0b60 0x7fcec01bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcec00a0780 0x7fcec01c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43092/0 (socket says 172.21.15.47:43092) 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 1 -- 172.21.15.47:0/3029900349 learned_addr learned my addr 172.21.15.47:0/3029900349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 10 monclient(hunting): get_auth_request con 0x7fcec00a0780 auth_method 0 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.924+0000 7fcec628c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): get_auth_request con 0x7fcec00a0b60 auth_method 0 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec00a2950 0x7fcec01cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec6a8d700 10 monclient(hunting): get_auth_request con 0x7fcec00a2950 auth_method 0 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec6a8d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec6a8d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec6a8d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient(hunting): handle_auth_done global_id 23672 payload 824 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcec00a0780 msgr2=0x7fcec01c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcec00a0780 0x7fcec01c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec00a2950 msgr2=0x7fcec01cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec00a2950 0x7fcec01cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient: _finish_hunting 0 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 monclient: found mon.noname-b 2024-04-20T02:20:10.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00094a0 con 0x7fcec00a0b60 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcec00a2950 0x7fcec01cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec5a8b700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcec00a0b60 0x7fcec01bce20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9b0 tx=0x7fceb000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 client.?.objecter ms_handle_connect 0x7fcec00a0b60 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fceb000ccc0 con 0x7fcec00a0b60 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: dump: 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.945 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:10.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 5 monclient: authenticate success, global_id 23672 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb000ce40 con 0x7fcec00a0b60 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 10 monclient: _renew_subs 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb8009b30 con 0x7fcec00a0b60 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fceb00138e0 con 0x7fcec00a0b60 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec4a89700 4 set_mon_vals no callback set 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 10 client.23672.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fceb6ffd700 10 monclient: dump: 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 10 monclient: _renew_subs 2024-04-20T02:20:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec01ac240 con 0x7fcec00a0b60 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.925+0000 7fcec84f0700 1 librados: init done 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.926+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fceb0013470 con 0x7fcec00a0b60 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.926+0000 7fcec84f0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.926+0000 7fcec84f0700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 0x7fcec002bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.926+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.926+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec01ac600 con 0x7fcec006aac0 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.926+0000 7fceb6ffd700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcea403a2f0 0x7fcea403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7021+0+0 (secure 0 0 0) 0x7fceb004ac00 con 0x7fcec00a0b60 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 client.23672.objecter ms_dispatch 0x7fcec0001550 osd_map(197..197 src has 1..197) v4 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 3 client.23672.objecter handle_osd_map got epochs [197,197] > 0 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 3 client.23672.objecter handle_osd_map decoding full epoch 197 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 0x7fcec002bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient: get_auth_request con 0x7fcec006aac0 auth_method 0 2024-04-20T02:20:10.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcea403a2f0 0x7fcea403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec6a8d700 10 monclient: get_auth_request con 0x7fcea403a2f0 auth_method 0 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 20 client.23672.objecter dump_active .. 0 homeless 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 10 monclient(hunting): handle_auth_done global_id 21711 payload 824 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcec00a3fb0 con 0x7fcec006aac0 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 client.23672.objecter ms_handle_connect 0x7fcec006aac0 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec628c700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 0x7fcec002bbc0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011830 tx=0x7fcebc011bf0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 client.23672.objecter resend_mon_ops 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 client.23672.objecter ms_handle_connect 0x7fcea403a2f0 2024-04-20T02:20:10.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fcec6a8d700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcea403a2f0 0x7fcea403c7b0 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b370 tx=0x7fceb8009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcebc00c070 con 0x7fcec006aac0 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.927+0000 7fceb6ffd700 10 monclient: dump: 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcebc0157d0 con 0x7fcec006aac0 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 msgr2=0x7fcec002bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 0x7fcec002bbc0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011830 tx=0x7fcebc011bf0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 msgr2=0x7fcec002bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.928+0000 7fceb6ffd700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec006aac0 0x7fcec002bbc0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.935+0000 7fcec84f0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.935+0000 7fcec84f0700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 0x7fcec002c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.935+0000 7fcec84f0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.935+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0068310 con 0x7fcec01ab240 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 0x7fcec002c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient: get_auth_request con 0x7fcec01ab240 auth_method 0 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:10.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 10 monclient(hunting): handle_auth_done global_id 21720 payload 824 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 1 -- 172.21.15.47:0/3029900349 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcec006b960 con 0x7fcec01ab240 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fcec628c700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 0x7fcec002c100 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011830 tx=0x7fcebc004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fceb6ffd700 10 client.23672.objecter ms_handle_connect 0x7fcec01ab240 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.936+0000 7fceb6ffd700 10 client.23672.objecter resend_mon_ops 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcebc024070 con 0x7fcec01ab240 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 10 monclient: dump: 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7fcebc015e10 con 0x7fcec01ab240 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:10.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 msgr2=0x7fcec002c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 0x7fcec002c100 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011830 tx=0x7fcebc004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 msgr2=0x7fcec002c100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fceb6ffd700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcec01ab240 0x7fcec002c100 secure :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011830 tx=0x7fcebc004750 comp rx=0 tx=0).stop 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:10.957 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:10.958 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:10.960 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:10.961 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:10.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.964 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.967 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:10.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.972 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:10.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:10.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:10.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:10.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:10.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:10.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:10.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:10.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:10.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:10.979 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:10.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:10.981 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:10.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 10 librados: watch_flush enter 2024-04-20T02:20:10.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 10 librados: watch_flush exit 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 20 client.23672.objecter shutdown clearing up homeless session... 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 10 client.23672.objecter successfully canceled tick 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcea403a2f0 msgr2=0x7fcea403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcea403a2f0 0x7fcea403c7b0 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b370 tx=0x7fceb8009000 comp rx=0 tx=0).stop 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 10 monclient: shutdown 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcec00a0b60 msgr2=0x7fcec01bce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 --2- 172.21.15.47:0/3029900349 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcec00a0b60 0x7fcec01bce20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9b0 tx=0x7fceb000ecc0 comp rx=0 tx=0).stop 2024-04-20T02:20:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec6a8d700 1 -- 172.21.15.47:0/3029900349 reap_dead start 2024-04-20T02:20:10.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec6a8d700 1 -- 172.21.15.47:0/3029900349 reap_dead start 2024-04-20T02:20:10.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 shutdown_connections 2024-04-20T02:20:10.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 >> 172.21.15.47:0/3029900349 conn(0x7fcec00a7570 msgr2=0x7fcec00a4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:10.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 shutdown_connections 2024-04-20T02:20:10.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 -- 172.21.15.47:0/3029900349 wait complete. 2024-04-20T02:20:10.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:10.937+0000 7fcec84f0700 1 librados: shutdown 2024-04-20T02:20:10.988 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:10.988 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:20:10.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:20:11.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.127+0000 7fe4dd570700 10 monclient: get_monmap_and_config 2024-04-20T02:20:11.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.127+0000 7fe4dd570700 10 monclient: build_initial_monmap 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.127+0000 7fe4dd570700 10 monclient: monmap: 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:11.128915+0000 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:11.128915+0000 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:11.129 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:11.129 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.128+0000 7fe4dd570700 1 Processor -- start 2024-04-20T02:20:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.128+0000 7fe4dd570700 1 -- start start 2024-04-20T02:20:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.128+0000 7fe4dd570700 10 monclient: init 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.128+0000 7fe4dd570700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.128+0000 7fe4dd570700 10 monclient: _add_conns ranks=[7,2,5,8,10,1,4,9,0,3,6] 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.128+0000 7fe4dd570700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d80a1510 0x7fe4d80a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): picked mon.noname-i con 0x7fe4d80a1510 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 0x7fe4d80a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): picked mon.noname-b con 0x7fe4d809f9a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809fe00 0x7fe4d80a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): picked mon.noname-h con 0x7fe4d809fe00 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80720a0 con 0x7fe4d809f9a0 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8070640 con 0x7fe4d809fe00 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8071d30 con 0x7fe4d80a1510 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 10 monclient(hunting): authenticate will time out at 1890.530127s 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 0x7fe4d80a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 0x7fe4d80a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:37036/0 (socket says 172.21.15.47:37036) 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 -- 172.21.15.47:0/912524652 learned_addr learned my addr 172.21.15.47:0/912524652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): get_auth_request con 0x7fe4d809f9a0 auth_method 0 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d6ffd700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d80a1510 0x7fe4d80a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d6ffd700 10 monclient(hunting): get_auth_request con 0x7fe4d80a1510 auth_method 0 2024-04-20T02:20:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d77fe700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809fe00 0x7fe4d80a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d77fe700 10 monclient(hunting): get_auth_request con 0x7fe4d809fe00 auth_method 0 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_done global_id 23681 payload 274 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 -- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d80a1510 msgr2=0x7fe4d80a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d80a1510 0x7fe4d80a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 -- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809fe00 msgr2=0x7fe4d80a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809fe00 0x7fe4d80a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient: _finish_hunting 0 2024-04-20T02:20:11.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 monclient: found mon.noname-b 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 -- 172.21.15.47:0/912524652 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80751e0 con 0x7fe4d809f9a0 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d6ffd700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d80a1510 0x7fe4d80a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d77fe700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809fe00 0x7fe4d80a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 0x7fe4d80a38f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009920 tx=0x7fe4cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6296b97ba34e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 1 -- 172.21.15.47:0/912524652 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4cc009e70 con 0x7fe4d809f9a0 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: dump: 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.139 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 1 -- 172.21.15.47:0/912524652 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00bcd0 con 0x7fe4d809f9a0 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 5 monclient: authenticate success, global_id 23681 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4dd570700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 1 -- 172.21.15.47:0/912524652 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4cc00be40 con 0x7fe4d809f9a0 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.129+0000 7fe4d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4d5ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4d5ffb700 10 monclient: dump: 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 4 set_mon_vals no callback set 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 10 monclient: shutdown 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 -- 172.21.15.47:0/912524652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 msgr2=0x7fe4d80a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 0x7fe4d80a38f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009920 tx=0x7fe4cc009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 -- 172.21.15.47:0/912524652 shutdown_connections 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d80a1510 0x7fe4d80a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809fe00 0x7fe4d80a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 --2- 172.21.15.47:0/912524652 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4d809f9a0 0x7fe4d80a38f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 -- 172.21.15.47:0/912524652 >> 172.21.15.47:0/912524652 conn(0x7fe4d80a8510 msgr2=0x7fe4d80a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 -- 172.21.15.47:0/912524652 shutdown_connections 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.130+0000 7fe4dd570700 1 -- 172.21.15.47:0/912524652 wait complete. 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.131+0000 7fe4dd570700 10 monclient: build_initial_monmap 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.131+0000 7fe4dd570700 10 monclient: monmap: 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:11.143 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.131+0000 7fe4dd570700 1 librados: starting msgr at 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.131+0000 7fe4dd570700 1 librados: starting objecter 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.131+0000 7fe4dd570700 1 Processor -- start 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 -- start start 2024-04-20T02:20:11.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 librados: setting wanted keys 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 librados: calling monclient init 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient: init 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient: _add_conns ranks=[5,7,0,6,1,4,8,3,2] 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809f9a0 0x7fe4d81cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): picked mon.noname-h con 0x7fe4d809f9a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d809fe00 0x7fe4d81ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): picked mon.noname-i con 0x7fe4d809fe00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d80a1510 0x7fe4d8124050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): picked mon.noname-a con 0x7fe4d80a1510 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8127740 con 0x7fe4d80a1510 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80a2130 con 0x7fe4d809f9a0 2024-04-20T02:20:11.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8071040 con 0x7fe4d809fe00 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4dd570700 10 monclient(hunting): authenticate will time out at 1890.533734s 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809f9a0 0x7fe4d81cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d67fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d809fe00 0x7fe4d81ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d80a1510 0x7fe4d8124050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809f9a0 0x7fe4d81cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:57562/0 (socket says 172.21.15.47:57562) 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d67fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d809fe00 0x7fe4d81ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:33594/0 (socket says 172.21.15.47:33594) 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 1 -- 172.21.15.47:0/374005602 learned_addr learned my addr 172.21.15.47:0/374005602 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 10 monclient(hunting): get_auth_request con 0x7fe4d809f9a0 auth_method 0 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d67fc700 10 monclient(hunting): get_auth_request con 0x7fe4d809fe00 auth_method 0 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d77fe700 10 monclient(hunting): get_auth_request con 0x7fe4d80a1510 auth_method 0 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.132+0000 7fe4d77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 10 monclient(hunting): handle_auth_done global_id 44213 payload 824 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d809fe00 msgr2=0x7fe4d81ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe4d809fe00 0x7fe4d81ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d80a1510 msgr2=0x7fe4d8124050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4d80a1510 0x7fe4d8124050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 monclient: found mon.noname-h 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009550 con 0x7fe4d809f9a0 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4d6ffd700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809f9a0 0x7fe4d81cc810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800e790 tx=0x7fe4c800eaa0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4bf7fe700 10 client.?.objecter ms_handle_connect 0x7fe4d809f9a0 2024-04-20T02:20:11.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.133+0000 7fe4bf7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4c800c070 con 0x7fe4d809f9a0 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: dump: 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8013880 con 0x7fe4d809f9a0 2024-04-20T02:20:11.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4c8013e60 con 0x7fe4d809f9a0 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4bf7fe700 10 monclient: dump: 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.154 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4dd570700 5 monclient: authenticate success, global_id 44213 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4dd570700 10 monclient: _renew_subs 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4dd570700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d8127fe0 con 0x7fe4d809f9a0 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.139+0000 7fe4d57fa700 4 set_mon_vals no callback set 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.140+0000 7fe4dd570700 10 client.44213.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.140+0000 7fe4dd570700 10 monclient: _renew_subs 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.140+0000 7fe4dd570700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:20:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.140+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d81265a0 con 0x7fe4d809f9a0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.140+0000 7fe4dd570700 1 librados: init done 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe4c80139f0 con 0x7fe4d809f9a0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4dd570700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4dd570700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 0x7fe4d8124590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80682f0 con 0x7fe4d806aac0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4bf7fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe4c403a2f0 0x7fe4c403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7021+0+0 (secure 0 0 0) 0x7fe4c804a630 con 0x7fe4d809f9a0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4bf7fe700 10 client.44213.objecter ms_dispatch 0x7fe4d8001550 osd_map(197..197 src has 1..197) v4 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4bf7fe700 3 client.44213.objecter handle_osd_map got epochs [197,197] > 0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4bf7fe700 3 client.44213.objecter handle_osd_map decoding full epoch 197 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.141+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 0x7fe4d8124590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient: get_auth_request con 0x7fe4d806aac0 auth_method 0 2024-04-20T02:20:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 20 client.44213.objecter dump_active .. 0 homeless 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d77fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe4c403a2f0 0x7fe4c403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d77fe700 10 monclient: get_auth_request con 0x7fe4c403a2f0 auth_method 0 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_done global_id 44222 payload 824 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4d80a7590 con 0x7fe4d806aac0 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 10 client.44213.objecter ms_handle_connect 0x7fe4d806aac0 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 0x7fe4d8124590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009fd0 tx=0x7fe4cc004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 10 client.44213.objecter resend_mon_ops 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 10 client.44213.objecter ms_handle_connect 0x7fe4c403a2f0 2024-04-20T02:20:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4d77fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe4c403a2f0 0x7fe4c403c7b0 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80a56c0 tx=0x7fe4d0009870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4cc020070 con 0x7fe4d806aac0 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.142+0000 7fe4bf7fe700 10 monclient: dump: 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe4cc00bb60 con 0x7fe4d806aac0 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 msgr2=0x7fe4d8124590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 0x7fe4d8124590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009fd0 tx=0x7fe4cc004750 comp rx=0 tx=0).stop 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 msgr2=0x7fe4d8124590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.143+0000 7fe4bf7fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d806aac0 0x7fe4d8124590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.155+0000 7fe4dd570700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.155+0000 7fe4dd570700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 0x7fe4d8124ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.155+0000 7fe4dd570700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.155+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d806a400 con 0x7fe4d8128400 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 0x7fe4d8124ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient: get_auth_request con 0x7fe4d8128400 auth_method 0 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.156+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.157+0000 7fe4d67fc700 10 monclient(hunting): handle_auth_done global_id 44231 payload 824 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.157+0000 7fe4d67fc700 1 -- 172.21.15.47:0/374005602 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe4d80a1c20 con 0x7fe4d8128400 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.157+0000 7fe4d67fc700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 0x7fe4d8124ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009840 tx=0x7fe4cc0092c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.157+0000 7fe4bf7fe700 10 client.44213.objecter ms_handle_connect 0x7fe4d8128400 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.158+0000 7fe4bf7fe700 10 client.44213.objecter resend_mon_ops 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.158+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4cc00bec0 con 0x7fe4d8128400 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.158+0000 7fe4bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.158+0000 7fe4bf7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.158+0000 7fe4bf7fe700 10 monclient: dump: 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fe4cc0045e0 con 0x7fe4d8128400 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 msgr2=0x7fe4d8124ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 0x7fe4d8124ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009840 tx=0x7fe4cc0092c0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 msgr2=0x7fe4d8124ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.159+0000 7fe4bf7fe700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d8128400 0x7fe4d8124ad0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009840 tx=0x7fe4cc0092c0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:11.164 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:11.165 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:11.166 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:11.167 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:11.168 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:11.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.172 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:11.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.175 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:11.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:11.183 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:11.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:11.185 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 10 librados: watch_flush enter 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 10 librados: watch_flush exit 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 20 client.44213.objecter shutdown clearing up homeless session... 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 10 client.44213.objecter successfully canceled tick 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe4c403a2f0 msgr2=0x7fe4c403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe4c403a2f0 0x7fe4c403c7b0 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80a56c0 tx=0x7fe4d0009870 comp rx=0 tx=0).stop 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 10 monclient: shutdown 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809f9a0 msgr2=0x7fe4d81cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 1 --2- 172.21.15.47:0/374005602 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe4d809f9a0 0x7fe4d81cc810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800e790 tx=0x7fe4c800eaa0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4d77fe700 1 -- 172.21.15.47:0/374005602 reap_dead start 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4d77fe700 1 -- 172.21.15.47:0/374005602 reap_dead start 2024-04-20T02:20:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 shutdown_connections 2024-04-20T02:20:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.160+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 >> 172.21.15.47:0/374005602 conn(0x7fe4d80a8510 msgr2=0x7fe4d8135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.161+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 shutdown_connections 2024-04-20T02:20:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.161+0000 7fe4dd570700 1 -- 172.21.15.47:0/374005602 wait complete. 2024-04-20T02:20:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.161+0000 7fe4dd570700 1 librados: shutdown 2024-04-20T02:20:11.188 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:11.188 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:20:11.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.325+0000 7fcbdfedf700 10 monclient: get_monmap_and_config 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.325+0000 7fcbdfedf700 10 monclient: build_initial_monmap 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.325+0000 7fcbdfedf700 10 monclient: monmap: 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:11.326720+0000 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:11.326720+0000 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:11.325 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:11.326 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 Processor -- start 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 -- start start 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient: init 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient: _add_conns ranks=[0,5,8,10,9,7,4,1,3,2,6] 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcbd809f690 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd813ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): picked mon.noname-h con 0x7fcbd80a4070 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcbd80a44f0 0x7fcbd813d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): picked mon.noname-e con 0x7fcbd80a44f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80751e0 con 0x7fcbd809f690 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8070ce0 con 0x7fcbd80a4070 2024-04-20T02:20:11.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8072780 con 0x7fcbd80a44f0 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.326+0000 7fcbdfedf700 10 monclient(hunting): authenticate will time out at 1890.727989s 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcbd80a44f0 0x7fcbd813d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbddc7b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcbd809f690 0x7fcbd80a3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50830/0 (socket says 172.21.15.47:50830) 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbddc7b700 1 -- 172.21.15.47:0/2636433727 learned_addr learned my addr 172.21.15.47:0/2636433727 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): get_auth_request con 0x7fcbd80a44f0 auth_method 0 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 1 --2- 172.21.15.47:0/2636433727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd813ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): get_auth_request con 0x7fcbd80a4070 auth_method 0 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdd47a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient(hunting): handle_auth_done global_id 34847 payload 274 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 1 -- 172.21.15.47:0/2636433727 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513006331 0 0) 0x7fcbd80751e0 con 0x7fcbd809f690 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 -- 172.21.15.47:0/2636433727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 msgr2=0x7fcbd813ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 --2- 172.21.15.47:0/2636433727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd813ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 -- 172.21.15.47:0/2636433727 >> v1:172.21.15.47:6789/0 conn(0x7fcbd809f690 legacy=0x7fcbd80a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient: _finish_hunting 0 2024-04-20T02:20:11.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 monclient: found mon.noname-e 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 -- 172.21.15.47:0/2636433727 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd809fa70 con 0x7fcbd80a44f0 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbde47c700 1 --2- 172.21.15.47:0/2636433727 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcbd80a44f0 0x7fcbd813d060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009dc0 tx=0x7fcbd000b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c917dc3fd69d879d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 1 -- 172.21.15.47:0/2636433727 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbd000e070 con 0x7fcbd80a44f0 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: dump: 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.341 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: _finish_auth 0 2024-04-20T02:20:11.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: _check_auth_tickets 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdfedf700 5 monclient: authenticate success, global_id 34847 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdfedf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 1 -- 172.21.15.47:0/2636433727 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd0012430 con 0x7fcbd80a44f0 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdfedf700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdfedf700 4 set_mon_vals no callback set 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 1 -- 172.21.15.47:0/2636433727 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbd0012a10 con 0x7fcbd80a44f0 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.327+0000 7fcbdcc79700 10 monclient: dump: 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.343 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 10 monclient: shutdown 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:11.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 -- 172.21.15.47:0/2636433727 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcbd80a44f0 msgr2=0x7fcbd813d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/2636433727 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcbd80a44f0 0x7fcbd813d060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009dc0 tx=0x7fcbd000b330 comp rx=0 tx=0).stop 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 -- 172.21.15.47:0/2636433727 shutdown_connections 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/2636433727 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcbd80a44f0 0x7fcbd813d060 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/2636433727 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd813ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 -- 172.21.15.47:0/2636433727 >> 172.21.15.47:0/2636433727 conn(0x7fcbd80a8610 msgr2=0x7fcbd80a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 -- 172.21.15.47:0/2636433727 shutdown_connections 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.328+0000 7fcbdfedf700 1 -- 172.21.15.47:0/2636433727 wait complete. 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 10 monclient: build_initial_monmap 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 10 monclient: monmap: 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:11.345 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 1 librados: starting msgr at 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 1 librados: starting objecter 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 1 Processor -- start 2024-04-20T02:20:11.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 1 -- start start 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 1 librados: setting wanted keys 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 1 librados: calling monclient init 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.329+0000 7fcbdfedf700 10 monclient: init 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient: _add_conns ranks=[1,5,0,3,8,2,4,6,7] 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd809f690 0x7fcbd81c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): picked mon.noname-f con 0x7fcbd809f690 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd8132bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): picked mon.noname-h con 0x7fcbd80a4070 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80a44f0 0x7fcbd81330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): picked mon.noname-a con 0x7fcbd80a44f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd809edc0 con 0x7fcbd80a44f0 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd809e520 con 0x7fcbd809f690 2024-04-20T02:20:11.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd806dcf0 con 0x7fcbd80a4070 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdfedf700 10 monclient(hunting): authenticate will time out at 1890.731331s 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80a44f0 0x7fcbd81330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80a44f0 0x7fcbd81330f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58522/0 (socket says 172.21.15.47:58522) 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 -- 172.21.15.47:0/337611836 learned_addr learned my addr 172.21.15.47:0/337611836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): get_auth_request con 0x7fcbd80a44f0 auth_method 0 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd8132bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): get_auth_request con 0x7fcbd80a4070 auth_method 0 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd809f690 0x7fcbd81c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): get_auth_request con 0x7fcbd809f690 auth_method 0 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbdd47a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient(hunting): handle_auth_done global_id 35046 payload 824 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 msgr2=0x7fcbd8132bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcbd80a4070 0x7fcbd8132bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd809f690 msgr2=0x7fcbd81c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbd809f690 0x7fcbd81c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient: _finish_hunting 0 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 monclient: found mon.noname-a 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009a70 con 0x7fcbd80a44f0 2024-04-20T02:20:11.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbde47c700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80a44f0 0x7fcbd81330f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b370 tx=0x7fcbd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbce7fc700 10 client.?.objecter ms_handle_connect 0x7fcbd80a44f0 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.330+0000 7fcbce7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbd0012c20 con 0x7fcbd80a44f0 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: dump: 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.351 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: _finish_auth 0 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd0012dc0 con 0x7fcbd80a44f0 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 5 monclient: authenticate success, global_id 35046 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 10 monclient: _renew_subs 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd806dfd0 con 0x7fcbd80a44f0 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:11.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbd00044b0 con 0x7fcbd80a44f0 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbce7fc700 10 monclient: dump: 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.353 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 10 client.35046.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbcffff700 4 set_mon_vals no callback set 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 10 monclient: _renew_subs 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd81c91e0 con 0x7fcbd80a44f0 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 1 librados: init done 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.331+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 0x7fcbd81cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8068380 con 0x7fcbd806aac0 2024-04-20T02:20:11.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcbd0004040 con 0x7fcbd80a44f0 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 0x7fcbd81cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient: get_auth_request con 0x7fcbd806aac0 auth_method 0 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbbc03a2f0 0x7fcbbc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcbd004c6c0 con 0x7fcbd80a44f0 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 10 client.35046.objecter ms_dispatch 0x7fcbd8001550 osd_map(198..198 src has 1..198) v4 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 3 client.35046.objecter handle_osd_map got epochs [198,198] > 0 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 3 client.35046.objecter handle_osd_map decoding full epoch 198 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 20 client.35046.objecter dump_active .. 0 homeless 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbdd47a700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbbc03a2f0 0x7fcbbc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_done global_id 44296 payload 824 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbd80721c0 con 0x7fcbd806aac0 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbdd47a700 10 monclient: get_auth_request con 0x7fcbbc03a2f0 auth_method 0 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 10 client.35046.objecter ms_handle_connect 0x7fcbd806aac0 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbddc7b700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 0x7fcbd81cc6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8000f60 tx=0x7fcbc80018f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.332+0000 7fcbce7fc700 10 client.35046.objecter resend_mon_ops 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbc8017070 con 0x7fcbd806aac0 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 10 monclient: dump: 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.356 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 10 client.35046.objecter ms_handle_connect 0x7fcbbc03a2f0 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbdd47a700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbbc03a2f0 0x7fcbbc03c7b0 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7fcbd809e0f0 tx=0x7fcbd4009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcbc8012560 con 0x7fcbd806aac0 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 msgr2=0x7fcbd81cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 0x7fcbd81cc6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8000f60 tx=0x7fcbc80018f0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 msgr2=0x7fcbd81cc6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:11.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.333+0000 7fcbce7fc700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd806aac0 0x7fcbd81cc6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbdfedf700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:11.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 0x7fcbd802bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbdfedf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8072930 con 0x7fcbd802bbc0 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 0x7fcbd802bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient: get_auth_request con 0x7fcbd802bbc0 auth_method 0 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 10 monclient(hunting): handle_auth_done global_id 44305 payload 824 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.341+0000 7fcbddc7b700 1 -- 172.21.15.47:0/337611836 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcbd80724a0 con 0x7fcbd802bbc0 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 client.35046.objecter ms_handle_connect 0x7fcbd802bbc0 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 client.35046.objecter resend_mon_ops 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbddc7b700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 0x7fcbd802bfd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8004b20 tx=0x7fcbc8000ed0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcbc8023070 con 0x7fcbd802bbc0 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 monclient: dump: 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fcbc8012e70 con 0x7fcbd802bbc0 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 msgr2=0x7fcbd802bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 0x7fcbd802bfd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8004b20 tx=0x7fcbc8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 msgr2=0x7fcbd802bfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbce7fc700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcbd802bbc0 0x7fcbd802bfd0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8004b20 tx=0x7fcbc8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 28, 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:20:11.363 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:11.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:11.365 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:11.366 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:11.367 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:11.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:11.370 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.373 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.374 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.376 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:11.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.379 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:11.381 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:11.382 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbdfedf700 10 librados: watch_flush enter 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbdfedf700 10 librados: watch_flush exit 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbdfedf700 20 client.35046.objecter shutdown clearing up homeless session... 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.342+0000 7fcbdfedf700 10 client.35046.objecter successfully canceled tick 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbbc03a2f0 msgr2=0x7fcbbc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcbbc03a2f0 0x7fcbbc03c7b0 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7fcbd809e0f0 tx=0x7fcbd4009210 comp rx=0 tx=0).stop 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 10 monclient: shutdown 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80a44f0 msgr2=0x7fcbd81330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 --2- 172.21.15.47:0/337611836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd80a44f0 0x7fcbd81330f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b370 tx=0x7fcbd0004ad0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbde47c700 1 -- 172.21.15.47:0/337611836 reap_dead start 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbde47c700 1 -- 172.21.15.47:0/337611836 reap_dead start 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 shutdown_connections 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 >> 172.21.15.47:0/337611836 conn(0x7fcbd80a8610 msgr2=0x7fcbd809fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 shutdown_connections 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 -- 172.21.15.47:0/337611836 wait complete. 2024-04-20T02:20:11.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.343+0000 7fcbdfedf700 1 librados: shutdown 2024-04-20T02:20:11.385 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:20:11.385 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['d', 'i', 'c', 'f'] 2024-04-20T02:20:11.385 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['h'] 2024-04-20T02:20:11.386 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-04-20T02:20:11.386 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-04-20T02:20:11.386 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-04-20T02:20:11.386 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:20:11.397 INFO:tasks.ceph.mon.d:Stopped 2024-04-20T02:20:11.397 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-04-20T02:20:11.397 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i store 2024-04-20T02:20:11.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i sync_force --yes-i-really-mean-it 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.581+0000 7f4a9faf2700 10 monclient: get_monmap_and_config 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.581+0000 7f4a9faf2700 10 monclient: build_initial_monmap 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.581+0000 7f4a9faf2700 10 monclient: monmap: 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:11.582291+0000 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:11.582291+0000 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:11.581 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:11.582 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.581+0000 7f4a9faf2700 1 Processor -- start 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 1 -- start start 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient: init 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient: _add_conns ranks=[10,6,1,7,2,8,9,3,4,0,5] 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4a980a0a70 0x7f4a980a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): picked mon.noname-f con 0x7f4a980a0a70 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:11.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4a980a2870 0x7f4a980a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): picked mon.noname-d con 0x7f4a980a2870 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4a980a31f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980751e0 con 0x7f4a980a31f0 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a980a1390 con 0x7f4a980a0a70 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980a3680 con 0x7f4a980a2870 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9d08d700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4a980a2870 msgr2=0x7f4a980a2cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9faf2700 10 monclient(hunting): authenticate will time out at 1890.983714s 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.582+0000 7f4a9d08d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4a980a2870 0x7f4a980a2cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4a980a0a70 0x7f4a980a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4a980a0a70 0x7f4a980a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:53486/0 (socket says 172.21.15.47:53486) 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 -- 172.21.15.47:0/1125374551 learned_addr learned my addr 172.21.15.47:0/1125374551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:11.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): get_auth_request con 0x7f4a980a0a70 auth_method 0 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9e08f700 1 --1- 172.21.15.47:0/1125374551 >> v1:172.21.15.179:6789/0 conn(0x7f4a980a31f0 0x7f4a9813ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:50194/0 (socket says 172.21.15.47:50194) 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_done global_id 26470 payload 274 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 -- 172.21.15.47:0/1125374551 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4a980a2870 msgr2=0x7f4a980a2cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 --2- 172.21.15.47:0/1125374551 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4a980a2870 0x7f4a980a2cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 -- 172.21.15.47:0/1125374551 >> v1:172.21.15.179:6789/0 conn(0x7f4a980a31f0 legacy=0x7f4a9813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient: _finish_hunting 0 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 monclient: found mon.noname-f 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 1 -- 172.21.15.47:0/1125374551 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357236856 0 0) 0x7f4a980751e0 con 0x7f4a980a31f0 2024-04-20T02:20:11.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 -- 172.21.15.47:0/1125374551 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98072560 con 0x7f4a980a0a70 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9d88e700 1 --2- 172.21.15.47:0/1125374551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a980a0a70 0x7f4a980a0e50 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f4a880098f0 tx=0x7f4a88009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779e0df9f4d8bb35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 1 -- 172.21.15.47:0/1125374551 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8800bb10 con 0x7f4a980a0a70 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: dump: 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.595 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: _finish_auth 0 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: _check_auth_tickets 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 1 -- 172.21.15.47:0/1125374551 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8800bc70 con 0x7f4a980a0a70 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9faf2700 5 monclient: authenticate success, global_id 26470 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9faf2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:11.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9faf2700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 1 -- 172.21.15.47:0/1125374551 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8800bde0 con 0x7f4a980a0a70 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9faf2700 4 set_mon_vals no callback set 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.583+0000 7f4a9c88c700 10 monclient: dump: 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.597 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 10 monclient: shutdown 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 -- 172.21.15.47:0/1125374551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a980a0a70 msgr2=0x7f4a980a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/1125374551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a980a0a70 0x7f4a980a0e50 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f4a880098f0 tx=0x7f4a88009c00 comp rx=0 tx=0).stop 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 -- 172.21.15.47:0/1125374551 shutdown_connections 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/1125374551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a980a0a70 0x7f4a980a0e50 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/1125374551 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4a980a2870 0x7f4a980a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 -- 172.21.15.47:0/1125374551 >> 172.21.15.47:0/1125374551 conn(0x7f4a980a4080 msgr2=0x7f4a9806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 -- 172.21.15.47:0/1125374551 shutdown_connections 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.584+0000 7f4a9faf2700 1 -- 172.21.15.47:0/1125374551 wait complete. 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 10 monclient: build_initial_monmap 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 10 monclient: monmap: 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:11.599 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 1 librados: starting msgr at 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 1 librados: starting objecter 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 1 Processor -- start 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 1 -- start start 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 1 librados: setting wanted keys 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 1 librados: calling monclient init 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.585+0000 7f4a9faf2700 10 monclient: init 2024-04-20T02:20:11.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient: _add_conns ranks=[0,7,3,5,2,1,6,4,8] 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 0x7f4a981cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): picked mon.noname-a con 0x7f4a980a0a70 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a980a2870 0x7f4a981baa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): picked mon.noname-i con 0x7f4a980a2870 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a980a31f0 0x7f4a981baf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): picked mon.noname-g con 0x7f4a980a31f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a981cd240 con 0x7f4a980a0a70 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a981a79e0 con 0x7f4a980a31f0 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a98071a00 con 0x7f4a980a2870 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9faf2700 10 monclient(hunting): authenticate will time out at 1890.987302s 2024-04-20T02:20:11.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 0x7f4a981cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 0x7f4a981cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58524/0 (socket says 172.21.15.47:58524) 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 -- 172.21.15.47:0/3527011323 learned_addr learned my addr 172.21.15.47:0/3527011323 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): get_auth_request con 0x7f4a980a0a70 auth_method 0 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a980a31f0 0x7f4a981baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a980a2870 0x7f4a981baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): get_auth_request con 0x7f4a980a31f0 auth_method 0 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): get_auth_request con 0x7f4a980a2870 auth_method 0 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9e08f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient(hunting): handle_auth_done global_id 35055 payload 824 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a980a2870 msgr2=0x7f4a981baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a980a2870 0x7f4a981baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a980a31f0 msgr2=0x7f4a981baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4a980a31f0 0x7f4a981baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient: _finish_hunting 0 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 monclient: found mon.noname-a 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:11.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.586+0000 7f4a9d88e700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a88009550 con 0x7f4a980a0a70 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9d88e700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 0x7f4a981cc500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806d640 tx=0x7f4a88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 client.?.objecter ms_handle_connect 0x7f4a980a0a70 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8801a980 con 0x7f4a980a0a70 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: dump: 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.605 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 5 monclient: authenticate success, global_id 35055 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8801f070 con 0x7f4a980a0a70 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:11.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a8801b430 con 0x7f4a980a0a70 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8ffff700 4 set_mon_vals no callback set 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a8e7fc700 10 monclient: dump: 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 10 monclient: _renew_subs 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a98072230 con 0x7f4a980a0a70 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 10 client.35055.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 10 monclient: _renew_subs 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a981a7ec0 con 0x7f4a980a0a70 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.587+0000 7f4a9faf2700 1 librados: init done 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9faf2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 0x7f4a9806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980752f0 con 0x7f4a981a8080 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4a88021030 con 0x7f4a980a0a70 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 0x7f4a9806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient: get_auth_request con 0x7f4a981a8080 auth_method 0 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4a7c03a2f0 0x7f4a7c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4a8804bae0 con 0x7f4a980a0a70 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 10 client.35055.objecter ms_dispatch 0x7f4a98001550 osd_map(198..198 src has 1..198) v4 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 3 client.35055.objecter handle_osd_map got epochs [198,198] > 0 2024-04-20T02:20:11.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 3 client.35055.objecter handle_osd_map decoding full epoch 198 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9e08f700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4a7c03a2f0 0x7f4a7c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 20 client.35055.objecter dump_active .. 0 homeless 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9e08f700 10 monclient: get_auth_request con 0x7f4a7c03a2f0 auth_method 0 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_done global_id 44323 payload 824 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a981cca40 con 0x7f4a981a8080 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 10 client.35055.objecter ms_handle_connect 0x7f4a981a8080 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a9d08d700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 0x7f4a9806fec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806cf50 tx=0x7f4a94009a00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.588+0000 7f4a8e7fc700 10 client.35055.objecter resend_mon_ops 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a94012a70 con 0x7f4a981a8080 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a9e08f700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4a7c03a2f0 0x7f4a7c03c7b0 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7f4a90001c70 tx=0x7f4a90001c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 10 monclient: dump: 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.610 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.611 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 10 client.35055.objecter ms_handle_connect 0x7f4a7c03a2f0 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4a94012bd0 con 0x7f4a981a8080 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 msgr2=0x7f4a9806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 0x7f4a9806fec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806cf50 tx=0x7f4a94009a00 comp rx=0 tx=0).stop 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 msgr2=0x7f4a9806fec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:11.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.589+0000 7f4a8e7fc700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a981a8080 0x7f4a9806fec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9faf2700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:20:11.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 0x7f4a980707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9faf2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4a98068430 con 0x7f4a98070400 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9d08d700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 0x7f4a980707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9d08d700 10 monclient: get_auth_request con 0x7f4a98070400 auth_method 0 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9d08d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9d08d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.598+0000 7f4a9d08d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a9d08d700 10 monclient(hunting): handle_auth_done global_id 44332 payload 824 2024-04-20T02:20:11.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a9d08d700 1 -- 172.21.15.47:0/3527011323 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f4a98067af0 con 0x7f4a98070400 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a9d08d700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 0x7f4a980707e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806cf50 tx=0x7f4a94005a30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a8e7fc700 10 client.35055.objecter ms_handle_connect 0x7f4a98070400 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a8e7fc700 10 client.35055.objecter resend_mon_ops 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a94023070 con 0x7f4a98070400 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a8e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a8e7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.599+0000 7f4a8e7fc700 10 monclient: dump: 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:11.616 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f4a9401ee80 con 0x7f4a98070400 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 msgr2=0x7f4a980707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 0x7f4a980707e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806cf50 tx=0x7f4a94005a30 comp rx=0 tx=0).stop 2024-04-20T02:20:11.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 msgr2=0x7f4a980707e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:11.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a8e7fc700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4a98070400 0x7f4a980707e0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806cf50 tx=0x7f4a94005a30 comp rx=0 tx=0).stop 2024-04-20T02:20:11.618 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:11.619 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:20:11.619 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:20:11.619 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:11.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 10 librados: watch_flush enter 2024-04-20T02:20:11.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 10 librados: watch_flush exit 2024-04-20T02:20:11.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 20 client.35055.objecter shutdown clearing up homeless session... 2024-04-20T02:20:11.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 10 client.35055.objecter successfully canceled tick 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4a7c03a2f0 msgr2=0x7f4a7c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4a7c03a2f0 0x7f4a7c03c7b0 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7f4a90001c70 tx=0x7f4a90001c00 comp rx=0 tx=0).stop 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 10 monclient: shutdown 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 msgr2=0x7f4a981cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9e08f700 1 -- 172.21.15.47:0/3527011323 reap_dead start 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 0x7f4a981cc500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a9806d640 tx=0x7f4a88004750 comp rx=0 tx=0).stop 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 shutdown_connections 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 --2- 172.21.15.47:0/3527011323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a980a0a70 0x7f4a981cc500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 >> 172.21.15.47:0/3527011323 conn(0x7f4a980a4080 msgr2=0x7f4a980a66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 shutdown_connections 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 -- 172.21.15.47:0/3527011323 wait complete. 2024-04-20T02:20:11.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:11.600+0000 7f4a9faf2700 1 librados: shutdown 2024-04-20T02:20:11.623 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:20:11.623 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-04-20T02:20:11.623 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:20:11.636 INFO:tasks.ceph.mon.i:Stopped 2024-04-20T02:20:11.636 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-20T02:20:11.636 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:20:11.636 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T02:20:11.636 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:20:11.648 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T02:20:11.648 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-04-20T02:20:11.648 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-04-20T02:20:11.648 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-04-20T02:20:11.648 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:20:11.692 INFO:tasks.ceph.mon.f:Stopped 2024-04-20T02:20:11.693 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-04-20T02:20:11.693 INFO:tasks.ceph.mon.h:Sent signal 19 2024-04-20T02:20:11.693 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:20:26.694 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-04-20T02:20:26.694 INFO:tasks.ceph.mon.h:Sent signal 18 2024-04-20T02:20:26.695 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-04-20T02:20:26.695 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:20:26.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.837+0000 7f38feee7700 10 monclient: get_monmap_and_config 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.837+0000 7f38feee7700 10 monclient: build_initial_monmap 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.838+0000 7f38feee7700 10 monclient: monmap: 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:26.839137+0000 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:26.839137+0000 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:26.838 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:26.839 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.838+0000 7f38feee7700 1 Processor -- start 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.838+0000 7f38feee7700 1 -- start start 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.838+0000 7f38feee7700 10 monclient: init 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient: _add_conns ranks=[10,0,9,7,8,6,3,2,5,1,4] 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 0x7f38f8134f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): picked mon.noname-f con 0x7f38f8134b20 addr v2:172.21.15.179:3300/0 2024-04-20T02:20:26.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38f80721e0 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): picked mon.noname-a con 0x7f38f80a2f10 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80751e0 con 0x7f38f80721e0 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f809ff90 con 0x7f38f80a2f10 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80a12f0 con 0x7f38f8134b20 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38feee7700 10 monclient(hunting): authenticate will time out at 1906.240512s 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fcc83700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 msgr2=0x7f38f8134f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38f7fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f38f80721e0 0x7f38f80a2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46862/0 (socket says 172.21.15.47:46862) 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fcc83700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 0x7f38f8134f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 learned_addr learned my addr 172.21.15.47:0/1392983574 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:26.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38f80a2f10 auth_method 0 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:26.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:26.839+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:27.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 msgr2=0x7f38f8134f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2024-04-20T02:20:27.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 0x7f38f8134f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:27.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:27.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38f80a2f10 auth_method 0 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:27.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.040+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 msgr2=0x7f38f8134f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 0x7f38f8134f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38f80a2f10 auth_method 0 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:27.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:27.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:27.441+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 msgr2=0x7f38f8134f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2024-04-20T02:20:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 0x7f38f8134f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:28.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38f80a2f10 auth_method 0 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:28.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:28.242+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): tick 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 msgr2=0x7f38f8134f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.179:3300/0 conn(0x7f38f8134b20 0x7f38f8134f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 msgr2=0x7f38f813c9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38f80a2f10 0x7f38f813c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.47:6789/0 conn(0x7f38f80721e0 legacy=0x7f38f80a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient: _add_conns ranks=[8,3,2,1,6,7,5,4] 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-e con 0x7f38d8003870 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-g con 0x7f38d80062f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-b con 0x7f38d8008e70 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f809ff90 con 0x7f38d8008e70 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80a12f0 con 0x7f38d80062f0 2024-04-20T02:20:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800bb50 con 0x7f38d8003870 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f67fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38d8008e70 auth_method 0 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003870 auth_method 0 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:29.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 10 monclient(hunting): get_auth_request con 0x7f38d80062f0 auth_method 0 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:29.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:29.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:29.840+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:30.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38d8008e70 auth_method 0 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003870 auth_method 0 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:30.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 10 monclient(hunting): get_auth_request con 0x7f38d80062f0 auth_method 0 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:30.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.041+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003870 auth_method 0 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38d8008e70 auth_method 0 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 10 monclient(hunting): get_auth_request con 0x7f38d80062f0 auth_method 0 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:30.442+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:31.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:31.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:31.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003870 auth_method 0 2024-04-20T02:20:31.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 10 monclient(hunting): get_auth_request con 0x7f38d8008e70 auth_method 0 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 10 monclient(hunting): get_auth_request con 0x7f38d80062f0 auth_method 0 2024-04-20T02:20:31.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:31.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:31.243+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.840+0000 7f38f67fc700 10 monclient(hunting): tick 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.840+0000 7f38f67fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.840+0000 7f38f67fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.840+0000 7f38f67fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 msgr2=0x7f38d8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8003870 0x7f38d8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 msgr2=0x7f38d8008780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d80062f0 0x7f38d8008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 msgr2=0x7f38d800b300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f38d8008e70 0x7f38d800b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 reap_dead start 2024-04-20T02:20:32.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient: _add_conns ranks=[5,6,4,7,1] 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 reap_dead start 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-h con 0x7f38d8003c90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-d con 0x7f38d8006c30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 0x7f38d80074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-c con 0x7f38d8007070 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38f809ff90 con 0x7f38d8007070 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80a12f0 con 0x7f38d8003c90 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800bb50 con 0x7f38d8006c30 2024-04-20T02:20:32.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f67fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 msgr2=0x7f38d80074f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 0x7f38d80074f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003c90 auth_method 0 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:32.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:32.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 msgr2=0x7f38d80074f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:20:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 0x7f38d80074f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003c90 auth_method 0 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.042+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:33.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:33.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 msgr2=0x7f38d80074f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:20:33.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:33.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 0x7f38d80074f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003c90 auth_method 0 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:33.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:33.443+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 msgr2=0x7f38d80074f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 0x7f38d80074f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8003c90 auth_method 0 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:34.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:34.244+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:35.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): tick 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38d8003c90 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 msgr2=0x7f38d80074f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38d8007070 0x7f38d80074f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient: _add_conns ranks=[1,7] 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f38d8007e40 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 0x7f38d8015720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:35.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-i con 0x7f38d8013330 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80a0200 con 0x7f38d8007e40 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f38f809ff90 con 0x7f38d8013330 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f67fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.179:6789/0 conn(0x7f38d8007e40 legacy=0x7f38d8008250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 msgr2=0x7f38d8015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:35.841+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 0x7f38d8015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:36.042+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 msgr2=0x7f38d8015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:36.042+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.179:6789/0 conn(0x7f38d8007e40 legacy=0x7f38d8008250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2024-04-20T02:20:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:36.042+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 0x7f38d8015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:36.443+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.179:6789/0 conn(0x7f38d8007e40 legacy=0x7f38d8008250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2024-04-20T02:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:36.443+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 msgr2=0x7f38d8015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:36.443+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 0x7f38d8015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:37.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:37.243+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.179:6789/0 conn(0x7f38d8007e40 legacy=0x7f38d8008250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2024-04-20T02:20:37.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:37.243+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 msgr2=0x7f38d8015720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:37.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:37.243+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 0x7f38d8015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): tick 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 msgr2=0x7f38d8015720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8013330 0x7f38d8015720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.179:6789/0 conn(0x7f38d8007e40 legacy=0x7f38d8008250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient: _add_conns ranks=[3,6,7,8,1,0,5,10,9,2,4] 2024-04-20T02:20:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 reap_dead start 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-g con 0x7f38d8015d00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-d con 0x7f38d8006c30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 0x7f38d80066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-i con 0x7f38d8007070 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f809ff90 con 0x7f38d8015d00 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80a12f0 con 0x7f38d8006c30 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800bb50 con 0x7f38d8007070 2024-04-20T02:20:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f67fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.840+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8015d00 auth_method 0 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:38.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8015d00 auth_method 0 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:39.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.042+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:20:39.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8015d00 auth_method 0 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:39.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:39.443+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:20:40.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fd484700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8015d00 auth_method 0 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.244+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:20:40.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:40.245+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): tick 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 msgr2=0x7f38d80066f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f38d8007070 0x7f38d80066f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 msgr2=0x7f38d800b260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:41.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f38d8006c30 0x7f38d800b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 msgr2=0x7f38d80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38d8015d00 0x7f38d80160e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient: _add_conns ranks=[9,8,0,5,10,1,4,2] 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38d8007e40 0x7f38d80074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-a con 0x7f38d8007e40 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8005ca0 0x7f38d8015720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-e con 0x7f38d8005ca0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38d8008220 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80a0200 con 0x7f38d8008220 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f809ff90 con 0x7f38d8007e40 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80a12f0 con 0x7f38d8005ca0 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f67fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:41.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8005ca0 0x7f38d8015720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): get_auth_request con 0x7f38d8005ca0 auth_method 0 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38d8007e40 0x7f38d80074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8007e40 auth_method 0 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38fcc83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient(hunting): handle_auth_done global_id 34856 payload 274 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38d8007e40 msgr2=0x7f38d80074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38d8007e40 0x7f38d80074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 >> v1:172.21.15.47:6789/0 conn(0x7f38d8008220 legacy=0x7f38d80122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f7fff700 1 monclient: found mon.noname-e 2024-04-20T02:20:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38fd484700 1 -- 172.21.15.47:0/1392983574 reap_dead start 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f7fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.841+0000 7f38f77fe700 1 -- 172.21.15.47:0/1392983574 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403572688 0 0) 0x7f38f80a0200 con 0x7f38d8008220 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f7fff700 1 -- 172.21.15.47:0/1392983574 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8008750 con 0x7f38d8005ca0 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38fcc83700 1 --2- 172.21.15.47:0/1392983574 >> v2:172.21.15.47:3300/0 conn(0x7f38d8007e40 0x7f38d80074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f7fff700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8005ca0 0x7f38d8015720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001050 tx=0x7f38e0007330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9b77e32049e3b3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 1 -- 172.21.15.47:0/1392983574 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38e0005e70 con 0x7f38d8005ca0 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: dump: 2024-04-20T02:20:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 1 -- 172.21.15.47:0/1392983574 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e00178d0 con 0x7f38d8005ca0 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38feee7700 5 monclient: authenticate success, global_id 34856 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38feee7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38feee7700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 1 -- 172.21.15.47:0/1392983574 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38e00163c0 con 0x7f38d8005ca0 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38feee7700 4 set_mon_vals no callback set 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.842+0000 7f38f77fe700 10 monclient: dump: 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 10 monclient: shutdown 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 -- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8005ca0 msgr2=0x7f38d8015720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8005ca0 0x7f38d8015720 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001050 tx=0x7f38e0007330 comp rx=0 tx=0).stop 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 -- 172.21.15.47:0/1392983574 shutdown_connections 2024-04-20T02:20:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 --2- 172.21.15.47:0/1392983574 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f38d8005ca0 0x7f38d8015720 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 -- 172.21.15.47:0/1392983574 >> 172.21.15.47:0/1392983574 conn(0x7f38f80a7420 msgr2=0x7f38f80a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 -- 172.21.15.47:0/1392983574 shutdown_connections 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.843+0000 7f38feee7700 1 -- 172.21.15.47:0/1392983574 wait complete. 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.844+0000 7f38feee7700 10 monclient: build_initial_monmap 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.844+0000 7f38feee7700 10 monclient: monmap: 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.844+0000 7f38feee7700 1 librados: starting msgr at 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.844+0000 7f38feee7700 1 librados: starting objecter 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.845+0000 7f38feee7700 1 Processor -- start 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.845+0000 7f38feee7700 1 -- start start 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.845+0000 7f38feee7700 1 librados: setting wanted keys 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.845+0000 7f38feee7700 1 librados: calling monclient init 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.845+0000 7f38feee7700 10 monclient: init 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient: _add_conns ranks=[0,5,4,6,2,3,1,8,7] 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 0x7f38f8131180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): picked mon.noname-a con 0x7f38d8005ca0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38f81316c0 0x7f38f811dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): picked mon.noname-h con 0x7f38f81316c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38f8131ac0 0x7f38f811e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): picked mon.noname-c con 0x7f38f8131ac0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f811f8b0 con 0x7f38d8005ca0 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8072630 con 0x7f38f8131ac0 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38f809f3c0 con 0x7f38f81316c0 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38feee7700 10 monclient(hunting): authenticate will time out at 1921.247171s 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fd484700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38f8131ac0 msgr2=0x7f38f811e1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fd484700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38f8131ac0 0x7f38f811e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 0x7f38f8131180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 0x7f38f8131180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48390/0 (socket says 172.21.15.47:48390) 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 -- 172.21.15.47:0/3674402468 learned_addr learned my addr 172.21.15.47:0/3674402468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): get_auth_request con 0x7f38d8005ca0 auth_method 0 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f7fff700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38f81316c0 0x7f38f811dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f7fff700 10 monclient(hunting): get_auth_request con 0x7f38f81316c0 auth_method 0 2024-04-20T02:20:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient(hunting): handle_auth_done global_id 35082 payload 824 2024-04-20T02:20:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 -- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38f81316c0 msgr2=0x7f38f811dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38f81316c0 0x7f38f811dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 -- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38f8131ac0 msgr2=0x7f38f811e1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38f8131ac0 0x7f38f811e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient: _finish_hunting 0 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 monclient: found mon.noname-a 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 -- 172.21.15.47:0/3674402468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e000f040 con 0x7f38d8005ca0 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f7fff700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38f81316c0 0x7f38f811dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38fcc83700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 0x7f38f8131180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38e800a010 tx=0x7f38e801f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f57fa700 10 client.?.objecter ms_handle_connect 0x7f38d8005ca0 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.846+0000 7f38f57fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38e8024070 con 0x7f38d8005ca0 2024-04-20T02:20:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: dump: 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: _finish_auth 0 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: _check_auth_tickets 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e801fe90 con 0x7f38d8005ca0 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38e8029420 con 0x7f38d8005ca0 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 5 monclient: authenticate success, global_id 35082 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:41.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f57fa700 10 monclient: dump: 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38f6ffd700 4 set_mon_vals no callback set 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 10 monclient: _renew_subs 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f81212d0 con 0x7f38d8005ca0 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 10 client.35082.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 10 monclient: _renew_subs 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f811ff60 con 0x7f38d8005ca0 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.847+0000 7f38feee7700 1 librados: init done 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f38e800f620 con 0x7f38d8005ca0 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f38d803eb40 0x7f38d8041000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f38e804cdc0 con 0x7f38d8005ca0 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 10 client.35082.objecter ms_dispatch 0x7f38f8001550 osd_map(199..199 src has 1..199) v4 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 3 client.35082.objecter handle_osd_map got epochs [199,199] > 0 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 3 client.35082.objecter handle_osd_map decoding full epoch 199 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38feee7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38feee7700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38feee7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f806acc0 con 0x7f38d8005ca0 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f7fff700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f38d803eb40 0x7f38d8041000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f7fff700 10 monclient: get_auth_request con 0x7f38d803eb40 auth_method 0 2024-04-20T02:20:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.848+0000 7f38f57fa700 20 client.35082.objecter dump_active .. 0 homeless 2024-04-20T02:20:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.849+0000 7f38f7fff700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f38d803eb40 0x7f38d8041000 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f38d80086a0 tx=0x7f38e000a000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.849+0000 7f38f57fa700 10 client.35082.objecter ms_handle_connect 0x7f38d803eb40 2024-04-20T02:20:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.854+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f38e802e030 con 0x7f38d8005ca0 2024-04-20T02:20:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.854+0000 7f38f57fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:41.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:41.854+0000 7f38f57fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38feee7700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38feee7700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38feee7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f38f8120fc0 con 0x7f38d8005ca0 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38f57fa700 1 -- 172.21.15.47:0/3674402468 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3151 (secure 0 0 0) 0x7f38e802e020 con 0x7f38d8005ca0 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38f57fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:20:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.164+0000 7f38f57fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:42.165 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":40,"quorum":[0,2,3,5,8],"quorum_names":["a","b","g","h","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:20:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 10 librados: watch_flush enter 2024-04-20T02:20:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 10 librados: watch_flush exit 2024-04-20T02:20:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 20 client.35082.objecter shutdown clearing up homeless session... 2024-04-20T02:20:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 10 client.35082.objecter successfully canceled tick 2024-04-20T02:20:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f38d803eb40 msgr2=0x7f38d8041000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f38d803eb40 0x7f38d8041000 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f38d80086a0 tx=0x7f38e000a000 comp rx=0 tx=0).stop 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.166+0000 7f38feee7700 10 monclient: shutdown 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 msgr2=0x7f38f8131180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 0x7f38f8131180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38e800a010 tx=0x7f38e801f8a0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 shutdown_connections 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f38d803eb40 0x7f38d8041000 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f38f8131ac0 0x7f38f811e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38d8005ca0 0x7f38f8131180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 --2- 172.21.15.47:0/3674402468 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f38f81316c0 0x7f38f811dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 >> 172.21.15.47:0/3674402468 conn(0x7f38f80a7420 msgr2=0x7f38f8133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 shutdown_connections 2024-04-20T02:20:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 -- 172.21.15.47:0/3674402468 wait complete. 2024-04-20T02:20:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.167+0000 7f38feee7700 1 librados: shutdown 2024-04-20T02:20:42.180 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-04-20T02:20:42.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-04-20T02:20:42.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:20:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.329+0000 7fea671a9700 10 monclient: get_monmap_and_config 2024-04-20T02:20:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.329+0000 7fea671a9700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.329+0000 7fea671a9700 10 monclient: monmap: 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:42.330989+0000 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:42.330989+0000 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:42.331 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.330+0000 7fea671a9700 1 Processor -- start 2024-04-20T02:20:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.330+0000 7fea671a9700 1 -- start start 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.330+0000 7fea671a9700 10 monclient: init 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient: _add_conns ranks=[0,6,1,10,7,3,5,4,8,2,9] 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fea60137d90 addr v1:172.21.15.47:6789/0 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea600a09e0 0x7fea6013cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient(hunting): picked mon.noname-d con 0x7fea600a09e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fea6013d1c0 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea600751e0 con 0x7fea60137d90 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6009dfc0 con 0x7fea6013d1c0 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea6009fa40 con 0x7fea600a09e0 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 10 monclient(hunting): authenticate will time out at 1921.732311s 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5ffff700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea600a09e0 msgr2=0x7fea6013cc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea600a09e0 0x7fea6013cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea65746700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7fea6013d1c0 legacy=0x7fea6013f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea64f45700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fea60137d90 0x7fea60138170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50652/0 (socket says 172.21.15.47:50652) 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea64f45700 1 -- 172.21.15.47:0/2605847403 learned_addr learned my addr 172.21.15.47:0/2605847403 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876753184 0 0) 0x7fea600751e0 con 0x7fea60137d90 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient(hunting): _negotiate 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient(hunting): my global_id is 35100 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea4c0035b0 con 0x7fea60137d90 2024-04-20T02:20:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1260383582 0 0) 0x7fea4c0035b0 con 0x7fea60137d90 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea600a09e0 msgr2=0x7fea6013cc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea600a09e0 0x7fea6013cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 >> v1:172.21.15.179:6789/0 conn(0x7fea6013d1c0 legacy=0x7fea6013f5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea60138880 con 0x7fea60137d90 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 5 monclient: authenticate success, global_id 35100 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea671a9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7fea500018e0 con 0x7fea60137d90 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.331+0000 7fea5f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: dump: 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 >> v1:172.21.15.47:6789/0 conn(0x7fea60137d90 legacy=0x7fea60138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: _add_conns ranks=[1,2,0,7,4,3,6,8,5] 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c004280 0x7fea4c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): picked mon.f con 0x7fea4c004280 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:20:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea4c004f90 0x7fea4c009a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): picked mon.b con 0x7fea4c004f90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c009f40 0x7fea4c00c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): picked mon.a con 0x7fea4c009f40 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6009fa40 con 0x7fea4c009f40 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c00cba0 con 0x7fea4c004280 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c00cd10 con 0x7fea4c004f90 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea671a9700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c009f40 0x7fea4c00c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea64f45700 1 -- 172.21.15.47:0/2605847403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c004280 msgr2=0x7fea4c004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea4c004f90 0x7fea4c009a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea64f45700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c004280 0x7fea4c004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): get_auth_request con 0x7fea4c009f40 auth_method 0 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): get_auth_request con 0x7fea4c004f90 auth_method 0 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:20:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5ffff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient(hunting): handle_auth_done global_id 35100 payload 293 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 -- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea4c004f90 msgr2=0x7fea4c009a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea4c004f90 0x7fea4c009a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 -- 172.21.15.47:0/2605847403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c004280 msgr2=0x7fea4c004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea4c004280 0x7fea4c004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 monclient: found mon.a 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 -- 172.21.15.47:0/2605847403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0035b0 con 0x7fea4c009f40 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 -- 172.21.15.47:0/2605847403 reap_dead start 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea65746700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c009f40 0x7fea4c00c330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea58009da0 tx=0x7fea4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb40fd654277b1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea58009760 con 0x7fea4c009f40 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: dump: 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5800e400 con 0x7fea4c009f40 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 1 -- 172.21.15.47:0/2605847403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea5800e9e0 con 0x7fea4c009f40 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea5f7fe700 10 monclient: dump: 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea671a9700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.332+0000 7fea671a9700 4 set_mon_vals no callback set 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 10 monclient: shutdown 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 -- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c009f40 msgr2=0x7fea4c00c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c009f40 0x7fea4c00c330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea58009da0 tx=0x7fea4c005880 comp rx=0 tx=0).stop 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 -- 172.21.15.47:0/2605847403 shutdown_connections 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 --2- 172.21.15.47:0/2605847403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c009f40 0x7fea4c00c330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 -- 172.21.15.47:0/2605847403 >> 172.21.15.47:0/2605847403 conn(0x7fea600aa8b0 msgr2=0x7fea600a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 -- 172.21.15.47:0/2605847403 shutdown_connections 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.333+0000 7fea671a9700 1 -- 172.21.15.47:0/2605847403 wait complete. 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 10 monclient: monmap: 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 1 librados: starting msgr at 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 1 librados: starting objecter 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 1 Processor -- start 2024-04-20T02:20:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 1 -- start start 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 1 librados: setting wanted keys 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 1 librados: calling monclient init 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.334+0000 7fea671a9700 10 monclient: init 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient: _add_conns ranks=[5,3,6,2,0,7,8,4,1] 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fea4c009f40 0x7fea60131c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): picked mon.noname-h con 0x7fea4c009f40 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea60132190 0x7fea600a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): picked mon.noname-g con 0x7fea60132190 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea601325f0 0x7fea600a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): picked mon.noname-d con 0x7fea601325f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea60139bf0 con 0x7fea60132190 2024-04-20T02:20:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea6009dc20 con 0x7fea4c009f40 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea6013a120 con 0x7fea601325f0 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea671a9700 10 monclient(hunting): authenticate will time out at 1921.736468s 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea65746700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea601325f0 msgr2=0x7fea600a2470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea65746700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea601325f0 0x7fea600a2470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea60132190 0x7fea600a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea60132190 0x7fea600a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:33944/0 (socket says 172.21.15.47:33944) 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 1 -- 172.21.15.47:0/3199885978 learned_addr learned my addr 172.21.15.47:0/3199885978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): get_auth_request con 0x7fea60132190 auth_method 0 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fea4c009f40 0x7fea60131c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): get_auth_request con 0x7fea4c009f40 auth_method 0 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea5ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.335+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 10 monclient(hunting): handle_auth_done global_id 21738 payload 824 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea601325f0 msgr2=0x7fea600a2470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea601325f0 0x7fea600a2470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fea4c009f40 msgr2=0x7fea60131c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fea4c009f40 0x7fea60131c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 monclient: found mon.noname-g 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6009e9e0 con 0x7fea60132190 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea64f45700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fea4c009f40 0x7fea60131c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5ffff700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea60132190 0x7fea600a1f30 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fea48003fd0 tx=0x7fea48009ae0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 client.?.objecter ms_handle_connect 0x7fea60132190 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea48012a70 con 0x7fea60132190 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:20:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: dump: 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.363 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea48012bd0 con 0x7fea60132190 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 5 monclient: authenticate success, global_id 21738 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 10 monclient: _renew_subs 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6013c0c0 con 0x7fea60132190 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea48012d40 con 0x7fea60132190 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5d7fa700 10 monclient: dump: 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 10 client.21738.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea5effd700 4 set_mon_vals no callback set 2024-04-20T02:20:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 10 monclient: _renew_subs 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea60139880 con 0x7fea60132190 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.336+0000 7fea671a9700 1 librados: init done 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea671a9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea671a9700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 0x7fea600a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea671a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60068180 con 0x7fea6006aac0 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 0x7fea600a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fea48012600 con 0x7fea60132190 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient: get_auth_request con 0x7fea6006aac0 auth_method 0 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea4003a2f0 0x7fea4003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 10 monclient(hunting): handle_auth_done global_id 35109 payload 824 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea600a6bc0 con 0x7fea6006aac0 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6670+0+0 (secure 0 0 0) 0x7fea4804a690 con 0x7fea60132190 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 10 client.21738.objecter ms_dispatch 0x7fea60001550 osd_map(200..200 src has 1..200) v4 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 3 client.21738.objecter handle_osd_map got epochs [200,200] > 0 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 3 client.21738.objecter handle_osd_map decoding full epoch 200 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 20 client.21738.objecter dump_active .. 0 homeless 2024-04-20T02:20:42.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea65746700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea4003a2f0 0x7fea4003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea65746700 10 monclient: get_auth_request con 0x7fea4003a2f0 auth_method 0 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea5d7fa700 10 client.21738.objecter ms_handle_connect 0x7fea6006aac0 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 client.21738.objecter resend_mon_ops 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.337+0000 7fea64f45700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 0x7fea600a29b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea500072a0 tx=0x7fea50007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea5000fcc0 con 0x7fea6006aac0 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 monclient: dump: 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 client.21738.objecter ms_handle_connect 0x7fea4003a2f0 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea65746700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea4003a2f0 0x7fea4003c7b0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00cb70 tx=0x7fea58005000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fea5000fe20 con 0x7fea6006aac0 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 msgr2=0x7fea600a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 0x7fea600a29b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea500072a0 tx=0x7fea50007660 comp rx=0 tx=0).stop 2024-04-20T02:20:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 msgr2=0x7fea600a29b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.338+0000 7fea5d7fa700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea6006aac0 0x7fea600a29b0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea500072a0 tx=0x7fea50007660 comp rx=0 tx=0).stop 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea671a9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea671a9700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 0x7fea601362a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea671a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6006a400 con 0x7fea600a2ef0 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 0x7fea601362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient: get_auth_request con 0x7fea600a2ef0 auth_method 0 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 10 monclient(hunting): handle_auth_done global_id 35118 payload 824 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.346+0000 7fea64f45700 1 -- 172.21.15.47:0/3199885978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea600a7bc0 con 0x7fea600a2ef0 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea64f45700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 0x7fea601362a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea50015040 tx=0x7fea50000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 client.21738.objecter ms_handle_connect 0x7fea600a2ef0 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 client.21738.objecter resend_mon_ops 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea50023070 con 0x7fea600a2ef0 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:42.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 monclient: dump: 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.375 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7fea50013930 con 0x7fea600a2ef0 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 msgr2=0x7fea601362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 0x7fea601362a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea50015040 tx=0x7fea50000f50 comp rx=0 tx=0).stop 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 msgr2=0x7fea601362a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea5d7fa700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea600a2ef0 0x7fea601362a0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea50015040 tx=0x7fea50000f50 comp rx=0 tx=0).stop 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 40, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:42.397 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.399 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.400 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:42.400 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea671a9700 10 librados: watch_flush enter 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea671a9700 10 librados: watch_flush exit 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea671a9700 20 client.21738.objecter shutdown clearing up homeless session... 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea671a9700 10 client.21738.objecter successfully canceled tick 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.347+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea4003a2f0 msgr2=0x7fea4003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fea4003a2f0 0x7fea4003c7b0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00cb70 tx=0x7fea58005000 comp rx=0 tx=0).stop 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 10 monclient: shutdown 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea60132190 msgr2=0x7fea600a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 --2- 172.21.15.47:0/3199885978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea60132190 0x7fea600a1f30 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fea48003fd0 tx=0x7fea48009ae0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea65746700 1 -- 172.21.15.47:0/3199885978 reap_dead start 2024-04-20T02:20:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea65746700 1 -- 172.21.15.47:0/3199885978 reap_dead start 2024-04-20T02:20:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 shutdown_connections 2024-04-20T02:20:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 >> 172.21.15.47:0/3199885978 conn(0x7fea600aa8b0 msgr2=0x7fea60134f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 shutdown_connections 2024-04-20T02:20:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 -- 172.21.15.47:0/3199885978 wait complete. 2024-04-20T02:20:42.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.348+0000 7fea671a9700 1 librados: shutdown 2024-04-20T02:20:42.403 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:20:42.403 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon b 2024-04-20T02:20:42.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:20:42.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.544+0000 7f71780e1700 10 monclient: get_monmap_and_config 2024-04-20T02:20:42.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.544+0000 7f71780e1700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.544+0000 7f71780e1700 10 monclient: monmap: 2024-04-20T02:20:42.544 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:42.545539+0000 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:42.545539+0000 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.545 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 Processor -- start 2024-04-20T02:20:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 -- start start 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient: init 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient: _add_conns ranks=[3,8,7,0,1,9,6,5,2,4,10] 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71700a1510 0x7f71700a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): picked mon.noname-g con 0x7f71700a1510 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 0x7f71700a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): picked mon.noname-e con 0x7f717009f9a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f717009fe00 0x7f71700a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): picked mon.noname-i con 0x7f717009fe00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700720a0 con 0x7f71700a1510 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7170070640 con 0x7f717009fe00 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7170071d30 con 0x7f717009f9a0 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f71780e1700 10 monclient(hunting): authenticate will time out at 1921.946827s 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f717667e700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f717009fe00 msgr2=0x7f71700a3e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f717667e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f717009fe00 0x7f71700a3e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.545+0000 7f717567c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 0x7f71700a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 0x7f71700a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:54396/0 (socket says 172.21.15.47:54396) 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 -- 172.21.15.47:0/2366702927 learned_addr learned my addr 172.21.15.47:0/2366702927 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): get_auth_request con 0x7f717009f9a0 auth_method 0 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71700a1510 0x7f71700a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): get_auth_request con 0x7f71700a1510 auth_method 0 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient(hunting): handle_auth_done global_id 34865 payload 274 2024-04-20T02:20:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 -- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f717009fe00 msgr2=0x7f71700a3e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f717009fe00 0x7f71700a3e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 -- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71700a1510 msgr2=0x7f71700a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71700a1510 0x7f71700a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 monclient: found mon.noname-e 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 -- 172.21.15.47:0/2366702927 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700751e0 con 0x7f717009f9a0 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f717567c700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 0x7f71700a38f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7160009920 tx=0x7f7160009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ed814a0ad046cdf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 1 -- 172.21.15.47:0/2366702927 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7160011aa0 con 0x7f717009f9a0 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: dump: 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 1 -- 172.21.15.47:0/2366702927 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160011c00 con 0x7f717009f9a0 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f71780e1700 5 monclient: authenticate success, global_id 34865 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f71780e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 1 -- 172.21.15.47:0/2366702927 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7160011d70 con 0x7f717009f9a0 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f7174e7b700 10 monclient: dump: 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f71780e1700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.546+0000 7f71780e1700 4 set_mon_vals no callback set 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 10 monclient: shutdown 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 -- 172.21.15.47:0/2366702927 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 msgr2=0x7f71700a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 0x7f71700a38f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7160009920 tx=0x7f7160009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 -- 172.21.15.47:0/2366702927 shutdown_connections 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f717009fe00 0x7f71700a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71700a1510 0x7f71700a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 --2- 172.21.15.47:0/2366702927 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f717009f9a0 0x7f71700a38f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 -- 172.21.15.47:0/2366702927 >> 172.21.15.47:0/2366702927 conn(0x7f71700a8510 msgr2=0x7f71700a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 -- 172.21.15.47:0/2366702927 shutdown_connections 2024-04-20T02:20:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.547+0000 7f71780e1700 1 -- 172.21.15.47:0/2366702927 wait complete. 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient: monmap: 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 librados: starting msgr at 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 librados: starting objecter 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 Processor -- start 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 -- start start 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 librados: setting wanted keys 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 librados: calling monclient init 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient: init 2024-04-20T02:20:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient: _add_conns ranks=[0,2,6,1,8,7,3,5,4] 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f717009f9a0 0x7f71701cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): picked mon.noname-a con 0x7f717009f9a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717009fe00 0x7f71701cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): picked mon.noname-b con 0x7f717009fe00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f71700a1510 0x7f71701242a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): picked mon.noname-d con 0x7f71700a1510 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701279b0 con 0x7f717009f9a0 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700a2130 con 0x7f717009fe00 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7170071040 con 0x7f71700a1510 2024-04-20T02:20:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.548+0000 7f71780e1700 10 monclient(hunting): authenticate will time out at 1921.949998s 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717667e700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f71700a1510 msgr2=0x7f71701242a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717667e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f71700a1510 0x7f71701242a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f717009f9a0 0x7f71701cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717009fe00 0x7f71701cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f717009f9a0 0x7f71701cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48438/0 (socket says 172.21.15.47:48438) 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 1 -- 172.21.15.47:0/1301542712 learned_addr learned my addr 172.21.15.47:0/1301542712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717009fe00 0x7f71701cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54346/0 (socket says 172.21.15.47:54346) 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): get_auth_request con 0x7f717009f9a0 auth_method 0 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): get_auth_request con 0x7f717009fe00 auth_method 0 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient(hunting): handle_auth_done global_id 23708 payload 824 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f71700a1510 msgr2=0x7f71701242a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f71700a1510 0x7f71701242a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f717009f9a0 msgr2=0x7f71701cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f717009f9a0 0x7f71701cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 monclient: found mon.noname-b 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009550 con 0x7f717009fe00 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7175e7d700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f717009f9a0 0x7f71701cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f717567c700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717009fe00 0x7f71701cccf0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7160016000 tx=0x7f7160004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 client.?.objecter ms_handle_connect 0x7f717009fe00 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7160004580 con 0x7f717009fe00 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: dump: 2024-04-20T02:20:42.577 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71780e1700 5 monclient: authenticate success, global_id 23708 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160003b10 con 0x7f717009fe00 2024-04-20T02:20:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7160003c80 con 0x7f717009fe00 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71780e1700 10 monclient: _renew_subs 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71780e1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7170128250 con 0x7f717009fe00 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f71667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.549+0000 7f7167fff700 4 set_mon_vals no callback set 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71667fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 10 client.23708.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71667fc700 10 monclient: dump: 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 10 monclient: _renew_subs 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7170125e80 con 0x7f717009fe00 2024-04-20T02:20:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 1 librados: init done 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 0x7f71701247e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.550+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700682f0 con 0x7f717006aac0 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 0x7f71701247e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7160021070 con 0x7f717009fe00 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient: get_auth_request con 0x7f717006aac0 auth_method 0 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f715403a2f0 0x7f715403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6670+0+0 (secure 0 0 0) 0x7f716004bd00 con 0x7f717009fe00 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 10 client.23708.objecter ms_dispatch 0x7f7170001550 osd_map(200..200 src has 1..200) v4 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 3 client.23708.objecter handle_osd_map got epochs [200,200] > 0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 3 client.23708.objecter handle_osd_map decoding full epoch 200 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 10 monclient(hunting): handle_auth_done global_id 23717 payload 824 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7170062590 con 0x7f717006aac0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 20 client.23708.objecter dump_active .. 0 homeless 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 10 client.23708.objecter ms_handle_connect 0x7f717006aac0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 10 client.23708.objecter resend_mon_ops 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f7175e7d700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 0x7f71701247e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f716c011d90 tx=0x7f716c00f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f717667e700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f715403a2f0 0x7f715403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f717667e700 10 monclient: get_auth_request con 0x7f715403a2f0 auth_method 0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f716c00fcc0 con 0x7f717006aac0 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.551+0000 7f71667fc700 10 monclient: dump: 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.584 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f716c00fe20 con 0x7f717006aac0 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 msgr2=0x7f71701247e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 0x7f71701247e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f716c011d90 tx=0x7f716c00f520 comp rx=0 tx=0).stop 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 msgr2=0x7f71701247e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717006aac0 0x7f71701247e0 secure :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f716c011d90 tx=0x7f716c00f520 comp rx=0 tx=0).stop 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f71667fc700 10 client.23708.objecter ms_handle_connect 0x7f715403a2f0 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.552+0000 7f717667e700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f715403a2f0 0x7f715403c7b0 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7f7168003cf0 tx=0x7f71680091f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.559+0000 7f71780e1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.559+0000 7f71780e1700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 0x7f7170125130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.559+0000 7f71780e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.559+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717006a400 con 0x7f7170124d20 2024-04-20T02:20:42.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 0x7f7170125130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient: get_auth_request con 0x7f7170124d20 auth_method 0 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 10 monclient(hunting): handle_auth_done global_id 23726 payload 824 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 1 -- 172.21.15.47:0/1301542712 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f71700a13a0 con 0x7f7170124d20 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 client.23708.objecter ms_handle_connect 0x7f7170124d20 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 client.23708.objecter resend_mon_ops 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f7175e7d700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 0x7f7170125130 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f716c011d90 tx=0x7f716c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f716c02b070 con 0x7f7170124d20 2024-04-20T02:20:42.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 monclient: dump: 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.587 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9481 (secure 0 0 0) 0x7f716c023040 con 0x7f7170124d20 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.560+0000 7f71667fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 msgr2=0x7f7170125130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71667fc700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 0x7f7170125130 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f716c011d90 tx=0x7f716c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71667fc700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 msgr2=0x7f7170125130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71667fc700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7170124d20 0x7f7170125130 secure :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f716c011d90 tx=0x7f716c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:42.588 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 40, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.604 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:42.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:42.607 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:42.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.609 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:42.610 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 10 librados: watch_flush enter 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 10 librados: watch_flush exit 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 20 client.23708.objecter shutdown clearing up homeless session... 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 10 client.23708.objecter successfully canceled tick 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f715403a2f0 msgr2=0x7f715403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f715403a2f0 0x7f715403c7b0 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7f7168003cf0 tx=0x7f71680091f0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 10 monclient: shutdown 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717009fe00 msgr2=0x7f71701cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 --2- 172.21.15.47:0/1301542712 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f717009fe00 0x7f71701cccf0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7160016000 tx=0x7f7160004750 comp rx=0 tx=0).stop 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f717667e700 1 -- 172.21.15.47:0/1301542712 reap_dead start 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f717667e700 1 -- 172.21.15.47:0/1301542712 reap_dead start 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 shutdown_connections 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 >> 172.21.15.47:0/1301542712 conn(0x7f71700a8510 msgr2=0x7f7170135e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 shutdown_connections 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 -- 172.21.15.47:0/1301542712 wait complete. 2024-04-20T02:20:42.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.561+0000 7f71780e1700 1 librados: shutdown 2024-04-20T02:20:42.613 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:20:42.614 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-04-20T02:20:42.614 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:20:42.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.754+0000 7f46cb950700 10 monclient: get_monmap_and_config 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.754+0000 7f46cb950700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.754+0000 7f46cb950700 10 monclient: monmap: 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:42.755200+0000 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:42.755200+0000 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:42.754 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:42.755 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.754+0000 7f46cb950700 1 Processor -- start 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.754+0000 7f46cb950700 1 -- start start 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.754+0000 7f46cb950700 10 monclient: init 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient: _add_conns ranks=[2,3,7,5,6,4,10,9,0,8,1] 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 0x7f46c40a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): picked mon.noname-b con 0x7f46c40a07f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46c40a0bd0 0x7f46c40a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): picked mon.noname-g con 0x7f46c40a0bd0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f46c40a1050 0x7f46c40a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): picked mon.noname-i con 0x7f46c40a1050 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4067800 con 0x7f46c40a07f0 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40672f0 con 0x7f46c40a0bd0 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f46c409ed90 con 0x7f46c40a1050 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46cb950700 10 monclient(hunting): authenticate will time out at 1922.156577s 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c9eed700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f46c40a1050 msgr2=0x7f46c40a2db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 0x7f46c40a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c9eed700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f46c40a1050 0x7f46c40a2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 0x7f46c40a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54368/0 (socket says 172.21.15.47:54368) 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 1 -- 172.21.15.47:0/1834800907 learned_addr learned my addr 172.21.15.47:0/1834800907 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): get_auth_request con 0x7f46c40a07f0 auth_method 0 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46c40a0bd0 0x7f46c40a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): get_auth_request con 0x7f46c40a0bd0 auth_method 0 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c96ec700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.755+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 10 monclient(hunting): handle_auth_done global_id 23735 payload 274 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 -- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f46c40a1050 msgr2=0x7f46c40a2db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f46c40a1050 0x7f46c40a2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 -- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46c40a0bd0 msgr2=0x7f46c40a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46c40a0bd0 0x7f46c40a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 monclient: found mon.noname-b 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 -- 172.21.15.47:0/1834800907 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c40751e0 con 0x7f46c40a07f0 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46c96ec700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 0x7f46c40a0010 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009920 tx=0x7f46c0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568b0744a8e9c851 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 1 -- 172.21.15.47:0/1834800907 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46c0011aa0 con 0x7f46c40a07f0 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: dump: 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46cb950700 5 monclient: authenticate success, global_id 23735 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 1 -- 172.21.15.47:0/1834800907 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0011c00 con 0x7f46c40a07f0 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46cb950700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46cb950700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46cb950700 4 set_mon_vals no callback set 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 1 -- 172.21.15.47:0/1834800907 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46c0011d70 con 0x7f46c40a07f0 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46bbfff700 10 monclient: dump: 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46cb950700 10 monclient: shutdown 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.756+0000 7f46cb950700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 -- 172.21.15.47:0/1834800907 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 msgr2=0x7f46c40a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 0x7f46c40a0010 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009920 tx=0x7f46c0009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 -- 172.21.15.47:0/1834800907 shutdown_connections 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f46c40a1050 0x7f46c40a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f46c40a07f0 0x7f46c40a0010 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 --2- 172.21.15.47:0/1834800907 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f46c40a0bd0 0x7f46c40a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 -- 172.21.15.47:0/1834800907 >> 172.21.15.47:0/1834800907 conn(0x7f46c40a3f90 msgr2=0x7f46c406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 -- 172.21.15.47:0/1834800907 shutdown_connections 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 1 -- 172.21.15.47:0/1834800907 wait complete. 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.757+0000 7f46cb950700 10 monclient: monmap: 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 librados: starting msgr at 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 librados: starting objecter 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 Processor -- start 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 -- start start 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 librados: setting wanted keys 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 librados: calling monclient init 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient: init 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient: _add_conns ranks=[0,6,8,4,5,7,1,3,2] 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 0x7f46c41bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): picked mon.noname-a con 0x7f46c40a07f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f46c40a0bd0 0x7f46c41bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): picked mon.noname-d con 0x7f46c40a0bd0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c40a1050 0x7f46c41cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): picked mon.noname-e con 0x7f46c40a1050 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41aa530 con 0x7f46c40a07f0 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41a8160 con 0x7f46c40a0bd0 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41a9c90 con 0x7f46c40a1050 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46cb950700 10 monclient(hunting): authenticate will time out at 1922.159933s 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46c8eeb700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f46c40a0bd0 msgr2=0x7f46c41bd0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.758+0000 7f46c8eeb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f46c40a0bd0 0x7f46c41bd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 0x7f46c41bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 0x7f46c41bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48444/0 (socket says 172.21.15.47:48444) 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 -- 172.21.15.47:0/2546519724 learned_addr learned my addr 172.21.15.47:0/2546519724 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): get_auth_request con 0x7f46c40a07f0 auth_method 0 2024-04-20T02:20:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c40a1050 0x7f46c41cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): get_auth_request con 0x7f46c40a1050 auth_method 0 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c9eed700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient(hunting): handle_auth_done global_id 35136 payload 824 2024-04-20T02:20:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c40a1050 msgr2=0x7f46c41cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c40a1050 0x7f46c41cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f46c40a0bd0 msgr2=0x7f46c41bd0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f46c40a0bd0 0x7f46c41bd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 monclient: found mon.noname-a 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009550 con 0x7f46c40a07f0 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46c96ec700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 0x7f46c41bcb70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009840 tx=0x7f46c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 client.?.objecter ms_handle_connect 0x7f46c40a07f0 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46c0004580 con 0x7f46c40a07f0 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: dump: 2024-04-20T02:20:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0003b10 con 0x7f46c40a07f0 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46c0003c80 con 0x7f46c40a07f0 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46cb950700 5 monclient: authenticate success, global_id 35136 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46cb950700 10 monclient: _renew_subs 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46cb950700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c41a9f70 con 0x7f46c40a07f0 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46bb7fe700 4 set_mon_vals no callback set 2024-04-20T02:20:42.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.759+0000 7f46b9ffb700 10 monclient: dump: 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.760+0000 7f46cb950700 10 client.35136.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.760+0000 7f46cb950700 10 monclient: _renew_subs 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.760+0000 7f46cb950700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.760+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c41a8f50 con 0x7f46c40a07f0 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.760+0000 7f46cb950700 1 librados: init done 2024-04-20T02:20:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46cb950700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46cb950700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 0x7f46c41cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4068280 con 0x7f46c406aac0 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f46c0021070 con 0x7f46c40a07f0 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 0x7f46c41cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient: get_auth_request con 0x7f46c406aac0 auth_method 0 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46a803a2f0 0x7f46a803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_done global_id 34883 payload 824 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6670+0+0 (secure 0 0 0) 0x7f46c004bdb0 con 0x7f46c40a07f0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 10 client.35136.objecter ms_dispatch 0x7f46c4001550 osd_map(200..200 src has 1..200) v4 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46c409f350 con 0x7f46c406aac0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 3 client.35136.objecter handle_osd_map got epochs [200,200] > 0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 3 client.35136.objecter handle_osd_map decoding full epoch 200 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 20 client.35136.objecter dump_active .. 0 homeless 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 10 client.35136.objecter ms_handle_connect 0x7f46c406aac0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 10 client.35136.objecter resend_mon_ops 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c8eeb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 0x7f46c41cc770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f46c406ce10 tx=0x7f46b400eb90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c9eed700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46a803a2f0 0x7f46a803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46b4013af0 con 0x7f46c406aac0 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46b9ffb700 10 monclient: dump: 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.788 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.761+0000 7f46c9eed700 10 monclient: get_auth_request con 0x7f46a803a2f0 auth_method 0 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f46b4013c50 con 0x7f46c406aac0 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 msgr2=0x7f46c41cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 0x7f46c41cc770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f46c406ce10 tx=0x7f46b400eb90 comp rx=0 tx=0).stop 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 msgr2=0x7f46c41cc770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c406aac0 0x7f46c41cc770 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f46c406ce10 tx=0x7f46b400eb90 comp rx=0 tx=0).stop 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46b9ffb700 10 client.35136.objecter ms_handle_connect 0x7f46a803a2f0 2024-04-20T02:20:42.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.762+0000 7f46c9eed700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46a803a2f0 0x7f46a803c7b0 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f46bc003d40 tx=0x7f46bc0095f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46cb950700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46cb950700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 0x7f46c409a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46cb950700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f46c406b960 con 0x7f46c41cccb0 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 0x7f46c409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient: get_auth_request con 0x7f46c41cccb0 auth_method 0 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 10 monclient(hunting): handle_auth_done global_id 34892 payload 824 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 1 -- 172.21.15.47:0/2546519724 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f46c406d160 con 0x7f46c41cccb0 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46b9ffb700 10 client.35136.objecter ms_handle_connect 0x7f46c41cccb0 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46c8eeb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 0x7f46c409a930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46b4018000 tx=0x7f46b40047c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46b9ffb700 10 client.35136.objecter resend_mon_ops 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f46b4022070 con 0x7f46c41cccb0 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46b9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46b9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.770+0000 7f46b9ffb700 10 monclient: dump: 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.793 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.794 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f46b40039e0 con 0x7f46c41cccb0 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 msgr2=0x7f46c409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 0x7f46c409a930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46b4018000 tx=0x7f46b40047c0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 msgr2=0x7f46c409a930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46b9ffb700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f46c41cccb0 0x7f46c409a930 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46b4018000 tx=0x7f46b40047c0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 40, 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:42.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.815 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.817 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:42.820 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:42.821 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 10 librados: watch_flush enter 2024-04-20T02:20:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 10 librados: watch_flush exit 2024-04-20T02:20:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 20 client.35136.objecter shutdown clearing up homeless session... 2024-04-20T02:20:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 10 client.35136.objecter successfully canceled tick 2024-04-20T02:20:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46a803a2f0 msgr2=0x7f46a803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f46a803a2f0 0x7f46a803c7b0 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f46bc003d40 tx=0x7f46bc0095f0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 10 monclient: shutdown 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 msgr2=0x7f46c41bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 0x7f46c41bcb70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009840 tx=0x7f46c0004750 comp rx=0 tx=0).stop 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46c9eed700 1 -- 172.21.15.47:0/2546519724 reap_dead start 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 shutdown_connections 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 --2- 172.21.15.47:0/2546519724 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f46c40a07f0 0x7f46c41bcb70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 >> 172.21.15.47:0/2546519724 conn(0x7f46c40a3f90 msgr2=0x7f46c41bf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 shutdown_connections 2024-04-20T02:20:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 -- 172.21.15.47:0/2546519724 wait complete. 2024-04-20T02:20:42.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.771+0000 7f46cb950700 1 librados: shutdown 2024-04-20T02:20:42.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:20:42.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon g 2024-04-20T02:20:42.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:20:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.965+0000 7f86a0d86700 10 monclient: get_monmap_and_config 2024-04-20T02:20:42.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.965+0000 7f86a0d86700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.965+0000 7f86a0d86700 10 monclient: monmap: 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:42.966743+0000 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:42.966743+0000 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 1 Processor -- start 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 1 -- start start 2024-04-20T02:20:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 10 monclient: init 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 10 monclient: _add_conns ranks=[1,6,9,7,5,8,4,2,10,3,0] 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f869c0a07f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.966+0000 7f86a0d86700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f869c0a0010 0x7f869c13cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient(hunting): picked mon.noname-d con 0x7f869c0a0010 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f869c13d1a0 0x7f869c13f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient(hunting): picked mon.noname-a con 0x7f869c13d1a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c0751e0 con 0x7f869c0a07f0 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c067800 con 0x7f869c13d1a0 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0672f0 con 0x7f869c0a0010 2024-04-20T02:20:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient(hunting): authenticate will time out at 1922.368089s 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f8699d9b700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f869c0a0010 msgr2=0x7f869c13cc60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f8699d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f869c0a0010 0x7f869c13cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869a59c700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f869c0a07f0 legacy=0x7f869c0a0bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f869c13d1a0 0x7f869c13f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f869c13d1a0 0x7f869c13f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48454/0 (socket says 172.21.15.47:48454) 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 -- 172.21.15.47:0/2599623702 learned_addr learned my addr 172.21.15.47:0/2599623702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): get_auth_request con 0x7f869c13d1a0 auth_method 0 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient(hunting): handle_auth_done global_id 35145 payload 274 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 -- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f869c0a0010 msgr2=0x7f869c13cc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 --2- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f869c0a0010 0x7f869c13cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 -- 172.21.15.47:0/2599623702 >> v1:172.21.15.179:6789/0 conn(0x7f869c0a07f0 legacy=0x7f869c0a0bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 monclient: found mon.noname-a 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 -- 172.21.15.47:0/2599623702 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0a12e0 con 0x7f869c13d1a0 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869ad9d700 1 --2- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f869c13d1a0 0x7f869c13f590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8694009e50 tx=0x7f869400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e99be9d6518fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 1 -- 172.21.15.47:0/2599623702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f869400e070 con 0x7f869c13d1a0 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: dump: 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 1 -- 172.21.15.47:0/2599623702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694003e90 con 0x7f869c13d1a0 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 5 monclient: authenticate success, global_id 35145 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 1 -- 172.21.15.47:0/2599623702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f869400bdf0 con 0x7f869c13d1a0 2024-04-20T02:20:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f86a0d86700 4 set_mon_vals no callback set 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.967+0000 7f869959a700 10 monclient: dump: 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 10 monclient: shutdown 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 -- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f869c13d1a0 msgr2=0x7f869c13f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 --2- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f869c13d1a0 0x7f869c13f590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8694009e50 tx=0x7f869400b330 comp rx=0 tx=0).stop 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 -- 172.21.15.47:0/2599623702 shutdown_connections 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 --2- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f869c13d1a0 0x7f869c13f590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 --2- 172.21.15.47:0/2599623702 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f869c0a0010 0x7f869c13cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 -- 172.21.15.47:0/2599623702 >> 172.21.15.47:0/2599623702 conn(0x7f869c0a3f90 msgr2=0x7f869c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 -- 172.21.15.47:0/2599623702 shutdown_connections 2024-04-20T02:20:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.968+0000 7f86a0d86700 1 -- 172.21.15.47:0/2599623702 wait complete. 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.969+0000 7f86a0d86700 10 monclient: build_initial_monmap 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.969+0000 7f86a0d86700 10 monclient: monmap: 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.969+0000 7f86a0d86700 1 librados: starting msgr at 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.969+0000 7f86a0d86700 1 librados: starting objecter 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 Processor -- start 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 -- start start 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 librados: setting wanted keys 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 librados: calling monclient init 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient: init 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient: _add_conns ranks=[7,4,8,0,5,1,2,3,6] 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f869c0a0010 0x7f869c1cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): picked mon.noname-i con 0x7f869c0a0010 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f869c0a07f0 0x7f869c138b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): picked mon.noname-c con 0x7f869c0a07f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 0x7f869c1390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): picked mon.noname-e con 0x7f869c13d1a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1cc9e0 con 0x7f869c0a07f0 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f869c125270 con 0x7f869c0a0010 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f869c124f90 con 0x7f869c13d1a0 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f86a0d86700 10 monclient(hunting): authenticate will time out at 1922.371979s 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.970+0000 7f8699d9b700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f869c0a07f0 msgr2=0x7f869c138b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f8699d9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f869c0a07f0 0x7f869c138b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869a59c700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f869c0a0010 msgr2=0x7f869c1cc4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869a59c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f869c0a0010 0x7f869c1cc4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:42.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 0x7f869c1390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 0x7f869c1390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:54436/0 (socket says 172.21.15.47:54436) 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 -- 172.21.15.47:0/3583589891 learned_addr learned my addr 172.21.15.47:0/3583589891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): get_auth_request con 0x7f869c13d1a0 auth_method 0 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient(hunting): handle_auth_done global_id 34901 payload 824 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f869c0a0010 msgr2=0x7f869c1cc4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f869c0a0010 0x7f869c1cc4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f869c0a07f0 msgr2=0x7f869c138b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f869c0a07f0 0x7f869c138b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 monclient: found mon.noname-e 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694009b30 con 0x7f869c13d1a0 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f869ad9d700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 0x7f869c1390c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f869400b730 tx=0x7f8694004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 client.?.objecter ms_handle_connect 0x7f869c13d1a0 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f86940046b0 con 0x7f869c13d1a0 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: dump: 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: _finish_auth 0 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: _check_auth_tickets 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694004850 con 0x7f869c13d1a0 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 5 monclient: authenticate success, global_id 34901 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 10 monclient: _renew_subs 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c1231c0 con 0x7f869c13d1a0 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f869401c8b0 con 0x7f869c13d1a0 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f868affd700 10 monclient: dump: 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 10 client.34901.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 10 monclient: _renew_subs 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f8698d99700 4 set_mon_vals no callback set 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c124cb0 con 0x7f869c13d1a0 2024-04-20T02:20:42.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.971+0000 7f86a0d86700 1 librados: init done 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.972+0000 7f86a0d86700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.972+0000 7f86a0d86700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 0x7f869c1218c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.972+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.972+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c068280 con 0x7f869c06aac0 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.972+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f869401c440 con 0x7f869c13d1a0 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f867c03a2f0 0x7f867c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6670+0+0 (secure 0 0 0) 0x7f869404bc20 con 0x7f869c13d1a0 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 client.34901.objecter ms_dispatch 0x7f869c001550 osd_map(200..200 src has 1..200) v4 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 3 client.34901.objecter handle_osd_map got epochs [200,200] > 0 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 3 client.34901.objecter handle_osd_map decoding full epoch 200 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 0x7f869c1218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient: get_auth_request con 0x7f869c06aac0 auth_method 0 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 20 client.34901.objecter dump_active .. 0 homeless 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f8699d9b700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f867c03a2f0 0x7f867c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f8699d9b700 10 monclient: get_auth_request con 0x7f867c03a2f0 auth_method 0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 10 monclient(hunting): handle_auth_done global_id 21765 payload 824 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f869c09f070 con 0x7f869c06aac0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 client.34901.objecter ms_handle_connect 0x7f869c06aac0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 client.34901.objecter resend_mon_ops 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f869a59c700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 0x7f869c1218c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b450 tx=0x7f868c00b810 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 client.34901.objecter ms_handle_connect 0x7f867c03a2f0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f8699d9b700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f867c03a2f0 0x7f867c03c7b0 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f8690006fd0 tx=0x7f8690006d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f868c003c50 con 0x7f869c06aac0 2024-04-20T02:20:42.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.973+0000 7f868affd700 10 monclient: dump: 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:42.995 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f868c003db0 con 0x7f869c06aac0 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 msgr2=0x7f869c1218c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 0x7f869c1218c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b450 tx=0x7f868c00b810 comp rx=0 tx=0).stop 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 msgr2=0x7f869c1218c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:42.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.974+0000 7f868affd700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c06aac0 0x7f869c1218c0 secure :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b450 tx=0x7f868c00b810 comp rx=0 tx=0).stop 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f86a0d86700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f86a0d86700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 0x7f869c122210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f86a0d86700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c075710 con 0x7f869c121e00 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 0x7f869c122210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient: get_auth_request con 0x7f869c121e00 auth_method 0 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:42.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 10 monclient(hunting): handle_auth_done global_id 21774 payload 824 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.982+0000 7f869a59c700 1 -- 172.21.15.47:0/3583589891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f869c09f350 con 0x7f869c121e00 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 client.34901.objecter ms_handle_connect 0x7f869c121e00 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 client.34901.objecter resend_mon_ops 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f869a59c700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 0x7f869c122210 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f868c000c00 tx=0x7f868c005a60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f868c01e070 con 0x7f869c121e00 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 monclient: dump: 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:42.999 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9139 (secure 0 0 0) 0x7f868c0056d0 con 0x7f869c121e00 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 msgr2=0x7f869c122210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 0x7f869c122210 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f868c000c00 tx=0x7f868c005a60 comp rx=0 tx=0).stop 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 msgr2=0x7f869c122210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f868affd700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f869c121e00 0x7f869c122210 secure :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f868c000c00 tx=0x7f868c005a60 comp rx=0 tx=0).stop 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 40, 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.013 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.015 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:43.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.020 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.021 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:43.022 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:43.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f86a0d86700 10 librados: watch_flush enter 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f86a0d86700 10 librados: watch_flush exit 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f86a0d86700 20 client.34901.objecter shutdown clearing up homeless session... 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f86a0d86700 10 client.34901.objecter successfully canceled tick 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f867c03a2f0 msgr2=0x7f867c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.983+0000 7f86a0d86700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f867c03a2f0 0x7f867c03c7b0 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f8690006fd0 tx=0x7f8690006d40 comp rx=0 tx=0).stop 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 10 monclient: shutdown 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 msgr2=0x7f869c1390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 0x7f869c1390c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f869400b730 tx=0x7f8694004ad0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f869ad9d700 1 -- 172.21.15.47:0/3583589891 reap_dead start 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 shutdown_connections 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 --2- 172.21.15.47:0/3583589891 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f869c13d1a0 0x7f869c1390c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 >> 172.21.15.47:0/3583589891 conn(0x7f869c0a3f90 msgr2=0x7f869c09c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:43.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 shutdown_connections 2024-04-20T02:20:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 -- 172.21.15.47:0/3583589891 wait complete. 2024-04-20T02:20:43.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:42.984+0000 7f86a0d86700 1 librados: shutdown 2024-04-20T02:20:43.025 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:20:43.025 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-04-20T02:20:43.025 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:20:43.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.166+0000 7f513dcd4700 10 monclient: get_monmap_and_config 2024-04-20T02:20:43.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.166+0000 7f513dcd4700 10 monclient: build_initial_monmap 2024-04-20T02:20:43.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.166+0000 7f513dcd4700 10 monclient: monmap: 2024-04-20T02:20:43.166 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:43.166 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:20:43.167659+0000 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:20:43.167659+0000 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:20:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:20:43.168 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 1 Processor -- start 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 1 -- start start 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient: init 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient: _add_conns ranks=[6,3,7,4,9,5,1,10,2,8,0] 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a07f0 0x7f51380a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient(hunting): picked mon.noname-d con 0x7f51380a07f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 0x7f51380a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient(hunting): picked mon.noname-g con 0x7f51380a0bd0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51380a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient(hunting): picked mon.noname-i con 0x7f51380a1050 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5138067800 con 0x7f51380a0bd0 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f513dcd4700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f51380672f0 con 0x7f51380a07f0 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f513dcd4700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f513809ed90 con 0x7f51380a1050 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f513dcd4700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.167+0000 7f51377fe700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a07f0 msgr2=0x7f51380a0010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f513dcd4700 10 monclient(hunting): authenticate will time out at 1922.569024s 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f51377fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a07f0 0x7f51380a0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5137fff700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 msgr2=0x7f51380a2db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5137fff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51380a2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 0x7f51380a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 0x7f51380a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:33978/0 (socket says 172.21.15.47:33978) 2024-04-20T02:20:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 -- 172.21.15.47:0/2886034778 learned_addr learned my addr 172.21.15.47:0/2886034778 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): get_auth_request con 0x7f51380a0bd0 auth_method 0 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient(hunting): handle_auth_done global_id 21783 payload 274 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 -- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 msgr2=0x7f51380a2db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51380a2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 -- 172.21.15.47:0/2886034778 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a07f0 msgr2=0x7f51380a0010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a07f0 0x7f51380a0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:20:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 monclient: found mon.noname-g 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 -- 172.21.15.47:0/2886034778 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380751e0 con 0x7f51380a0bd0 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.168+0000 7f5136ffd700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 0x7f51380a2870 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f512c009920 tx=0x7f512c009c30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=992019cfb0819da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 1 -- 172.21.15.47:0/2886034778 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f512c009e70 con 0x7f51380a0bd0 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: dump: 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: _finish_auth 0 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: _check_auth_tickets 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 1 -- 172.21.15.47:0/2886034778 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c00bcd0 con 0x7f51380a0bd0 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 5 monclient: authenticate success, global_id 21783 2024-04-20T02:20:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 1 -- 172.21.15.47:0/2886034778 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f512c00be40 con 0x7f51380a0bd0 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f51367fc700 10 monclient: dump: 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 10 monclient: get_monmap_and_config success 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 4 set_mon_vals no callback set 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 10 monclient: shutdown 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 -- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 msgr2=0x7f51380a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 0x7f51380a2870 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f512c009920 tx=0x7f512c009c30 comp rx=0 tx=0).stop 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 -- 172.21.15.47:0/2886034778 shutdown_connections 2024-04-20T02:20:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51380a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a07f0 0x7f51380a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 --2- 172.21.15.47:0/2886034778 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a0bd0 0x7f51380a2870 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 -- 172.21.15.47:0/2886034778 >> 172.21.15.47:0/2886034778 conn(0x7f51380a3f90 msgr2=0x7f513806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 -- 172.21.15.47:0/2886034778 shutdown_connections 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.169+0000 7f513dcd4700 1 -- 172.21.15.47:0/2886034778 wait complete. 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.170+0000 7f513dcd4700 10 monclient: build_initial_monmap 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.170+0000 7f513dcd4700 10 monclient: monmap: 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:20:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.170+0000 7f513dcd4700 1 librados: starting msgr at 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.170+0000 7f513dcd4700 1 librados: starting objecter 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.170+0000 7f513dcd4700 1 Processor -- start 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 -- start start 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 librados: setting wanted keys 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 librados: calling monclient init 2024-04-20T02:20:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient: init 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient: _reopen_session rank -1 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient: _add_conns ranks=[3,6,7,2,1,5,4,0,8] 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a07f0 0x7f51381cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): picked mon.noname-g con 0x7f51380a07f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a0bd0 0x7f51381bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): picked mon.noname-d con 0x7f51380a0bd0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51381bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): picked mon.noname-i con 0x7f51380a1050 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51381a9a60 con 0x7f51380a07f0 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f513809e800 con 0x7f51380a0bd0 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f513809e360 con 0x7f51380a1050 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): _renew_subs 2024-04-20T02:20:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f513dcd4700 10 monclient(hunting): authenticate will time out at 1922.572757s 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f5136ffd700 1 -- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a0bd0 msgr2=0x7f51381bcb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3303/0 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a0bd0 0x7f51381bcb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f5137fff700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 msgr2=0x7f51381bd0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f5137fff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51381bd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.171+0000 7f51377fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a07f0 0x7f51381cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a07f0 0x7f51381cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:33994/0 (socket says 172.21.15.47:33994) 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 -- 172.21.15.47:0/3738141913 learned_addr learned my addr 172.21.15.47:0/3738141913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): get_auth_request con 0x7f51380a07f0 auth_method 0 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:43.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient(hunting): handle_auth_done global_id 21792 payload 824 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 msgr2=0x7f51381bd0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f51380a1050 0x7f51381bd0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a0bd0 msgr2=0x7f51381bcb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f51380a0bd0 0x7f51381bcb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient: _finish_hunting 0 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 monclient: found mon.noname-g 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009550 con 0x7f51380a07f0 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f51377fe700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a07f0 0x7f51381cc2e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f512800b550 tx=0x7f512800b910 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 client.?.objecter ms_handle_connect 0x7f51380a07f0 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:20:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5128003c50 con 0x7f51380a07f0 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: dump: 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: _finish_auth 0 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 5 monclient: authenticate success, global_id 21792 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128003df0 con 0x7f51380a07f0 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51280103a0 con 0x7f51380a07f0 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 10 monclient: _renew_subs 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:20:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51381ccf90 con 0x7f51380a07f0 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f5135ffb700 4 set_mon_vals no callback set 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f511ffff700 10 monclient: dump: 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 10 client.21792.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 10 monclient: _renew_subs 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51381a90e0 con 0x7f51380a07f0 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.172+0000 7f513dcd4700 1 librados: init done 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f513dcd4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5128005070 con 0x7f51380a07f0 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f513dcd4700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 0x7f51381a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5138068280 con 0x7f513806aac0 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f511ffff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f512403a320 0x7f512403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.173+0000 7f5136ffd700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 0x7f51381a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient: get_auth_request con 0x7f513806aac0 auth_method 0 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6670+0+0 (secure 0 0 0) 0x7f5128049950 con 0x7f51380a07f0 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 client.21792.objecter ms_dispatch 0x7f5138001550 osd_map(200..200 src has 1..200) v4 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 3 client.21792.objecter handle_osd_map got epochs [200,200] > 0 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 3 client.21792.objecter handle_osd_map decoding full epoch 200 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:43.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5137fff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f512403a320 0x7f512403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 20 client.21792.objecter dump_active .. 0 homeless 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5137fff700 10 monclient: get_auth_request con 0x7f512403a320 auth_method 0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 10 monclient(hunting): handle_auth_done global_id 44285 payload 824 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f513809f350 con 0x7f513806aac0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 client.21792.objecter ms_handle_connect 0x7f513806aac0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 client.21792.objecter resend_mon_ops 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5136ffd700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 0x7f51381a6da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f512c009fd0 tx=0x7f512c004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 client.21792.objecter ms_handle_connect 0x7f512403a320 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f5137fff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f512403a320 0x7f512403c7e0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7f5130003cf0 tx=0x7f51300091f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f512c004390 con 0x7f513806aac0 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.174+0000 7f511ffff700 10 monclient: dump: 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:43.196 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f512c0044f0 con 0x7f513806aac0 2024-04-20T02:20:43.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:20:43.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:20:43.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 msgr2=0x7f51381a6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 0x7f51381a6da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f512c009fd0 tx=0x7f512c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:43.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 msgr2=0x7f51381a6da0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:43.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.175+0000 7f511ffff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513806aac0 0x7f51381a6da0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f512c009fd0 tx=0x7f512c004750 comp rx=0 tx=0).stop 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.182+0000 7f513dcd4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.182+0000 7f513dcd4700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 0x7f513802bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.182+0000 7f513dcd4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.182+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f513806a400 con 0x7f513802bbc0 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 0x7f513802bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient: get_auth_request con 0x7f513802bbc0 auth_method 0 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:20:43.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 10 monclient(hunting): handle_auth_done global_id 44294 payload 824 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 1 -- 172.21.15.47:0/3738141913 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f513806d160 con 0x7f513802bbc0 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f511ffff700 10 client.21792.objecter ms_handle_connect 0x7f513802bbc0 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f511ffff700 10 client.21792.objecter resend_mon_ops 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f5136ffd700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 0x7f513802bfa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f512c009920 tx=0x7f512c0092c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f512c02c070 con 0x7f513802bbc0 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f511ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f511ffff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.183+0000 7f511ffff700 10 monclient: dump: 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:20:43.201 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:20:43.202 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f512c026430 con 0x7f513802bbc0 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 msgr2=0x7f513802bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 0x7f513802bfa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f512c009920 tx=0x7f512c0092c0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 msgr2=0x7f513802bfa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f511ffff700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f513802bbc0 0x7f513802bfa0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f512c009920 tx=0x7f512c0092c0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 40, 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:20:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:20:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:20:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:20:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.214 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.219 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:20:43.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:20:43.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:20:43.222 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:20:43.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:20:43.224 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:20:43.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 10 librados: watch_flush enter 2024-04-20T02:20:43.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 10 librados: watch_flush exit 2024-04-20T02:20:43.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 20 client.21792.objecter shutdown clearing up homeless session... 2024-04-20T02:20:43.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 10 client.21792.objecter successfully canceled tick 2024-04-20T02:20:43.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f512403a320 msgr2=0x7f512403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f512403a320 0x7f512403c7e0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7f5130003cf0 tx=0x7f51300091f0 comp rx=0 tx=0).stop 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 10 monclient: shutdown 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a07f0 msgr2=0x7f51381cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 --2- 172.21.15.47:0/3738141913 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51380a07f0 0x7f51381cc2e0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f512800b550 tx=0x7f512800b910 comp rx=0 tx=0).stop 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f5137fff700 1 -- 172.21.15.47:0/3738141913 reap_dead start 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f5137fff700 1 -- 172.21.15.47:0/3738141913 reap_dead start 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 shutdown_connections 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 >> 172.21.15.47:0/3738141913 conn(0x7f51380a3f90 msgr2=0x7f51380a22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 shutdown_connections 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 -- 172.21.15.47:0/3738141913 wait complete. 2024-04-20T02:20:43.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:20:43.184+0000 7f513dcd4700 1 librados: shutdown 2024-04-20T02:20:43.228 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:20:43.228 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosA api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: seed 46984 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (65 ms) 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10835 ms) 2024-04-20T02:21:00.018 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (16 ms) 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (10916 ms total) 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7217 ms) 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7858 ms) 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10167 ms) 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: my_snaps [4] 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: my_snaps [5,4] 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: my_snaps [6,5,4] 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-04-20T02:21:00.019 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: promoting some heads 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: promoting from clones for snap 7 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: promoting from clones for snap 6 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: promoting from clones for snap 5 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: promoting from clones for snap 4 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: waiting for scrubs... 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: done waiting 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (54781 ms) 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10005 ms) 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8004 ms) 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8053 ms) 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (7980 ms) 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-04-20T02:21:00.020 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (9975 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (9002 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10027 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (13015 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7982 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (16840 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (42106 ms) 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-04-20T02:21:00.021 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7977 ms) 2024-04-20T02:21:00.022 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-04-20T02:21:00.022 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (7993 ms) 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSe 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7008 ms) 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6992 ms) 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7021 ms) 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6997 ms) 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-04-20T02:21:06.047 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (6994 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7013 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (10005 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7024 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (9971 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (9009 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (11393 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (11619 ms) 2024-04-20T02:21:06.048 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (36155 ms) 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (6752 ms) 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7011 ms) 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (10017 ms) 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (160981 ms total) 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [----------] Global test environment tear-down 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (264012 ms total) 2024-04-20T02:21:06.049 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio: [ PASSED ] 42 tests. 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout:tRead 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: hmm, no HitSet yet 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: ok, hit_set contains 185:602f83fe:::foo:head 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9036 ms) 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg_num = 32 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 0 ls 1713579676,0 2024-04-20T02:21:15.380 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 1 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 2 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 3 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 4 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 5 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 6 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 7 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 8 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 9 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 10 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 11 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 12 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 13 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 14 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 15 ls 1713579676,0 2024-04-20T02:21:15.381 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 16 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 17 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 18 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 19 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 20 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 21 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 22 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 23 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 24 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 25 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 26 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 27 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 28 ls 1713579676,0 2024-04-20T02:21:15.382 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 29 ls 1713579676,0 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 30 ls 1713579676,0 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg 31 ls 1713579676,0 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: pg_num = 32 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6cac518f:::0:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:02547ec2:::1:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f905c69b:::2:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:cfc208b3:::3:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d83876eb:::4:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b29083e3:::5:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c4fdafeb:::6:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:5c6b0b28:::7:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:bd63b0f1:::8:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e960b815:::9:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:52ea6a34:::10:head 2024-04-20T02:21:15.383 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:89d3ae78:::11:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:de5d7c5f:::12:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:566253c9:::13:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:62a1935d:::14:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:863748b0:::15:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3958e169:::16:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4d4dabf9:::17:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:8391935d:::18:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:28883081:::19:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:69259c59:::20:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4bdb80b7:::21:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a11c5d71:::22:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:271af37b:::23:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:95b121be:::24:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:58d1031b:::25:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:0a050783:::26:head 2024-04-20T02:21:15.384 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c709704c:::27:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:cbe56eaf:::28:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:86b4b162:::29:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:70d89383:::30:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:dd450c7c:::31:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6d5729b1:::32:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c388f3fb:::33:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:56cfea31:::34:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9dbc1bf7:::35:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:40b74ccd:::36:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4d5aaf42:::37:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:920f362c:::38:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6cc53222:::39:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9cad833f:::40:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:1ea84d41:::41:head 2024-04-20T02:21:15.385 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c4480ef6:::42:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a694361e:::43:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d1bd33e9:::44:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ddc2cd5d:::45:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:2b782207:::46:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7b187fca:::47:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:90ecdf6f:::48:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a5ed95fe:::49:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ea0eaa55:::50:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f33ef17b:::51:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a0d1b2f6:::52:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:60c5229e:::53:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:edcbc575:::54:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:102cf253:::55:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:efb7fb0b:::56:head 2024-04-20T02:21:15.386 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:50d0a326:::57:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d4dc5daf:::58:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3a130462:::59:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ec87ed71:::60:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d5bc9454:::61:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3ddfe313:::62:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7c2816b9:::63:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:47e00e4d:::64:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c6410c18:::65:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b48ed237:::66:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:cd63ad31:::67:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b179e92b:::68:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:0d9f741a:::69:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6d3352ae:::70:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c6d5c19e:::71:head 2024-04-20T02:21:15.387 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:bc4729c3:::72:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:77e930b9:::73:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:0abeecfd:::74:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b7c37e15:::75:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b6378398:::76:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:02bd68de:::77:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:cc795d2d:::78:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:630d4fea:::79:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e0d29ef5:::80:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:fd6f13d2:::81:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:606461d5:::82:head 2024-04-20T02:21:15.389 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:eadbdc43:::83:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:8761d0bb:::84:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9ef0186f:::85:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e0d41294:::86:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:961de695:::87:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:1423148f:::88:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:633a8fa2:::89:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a8653809:::90:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3dac8b33:::91:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:35aad435:::92:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f6dcc343:::93:head 2024-04-20T02:21:15.390 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:dbbdad87:::94:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:1cb48ce0:::95:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:03cd461c:::96:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:17a4ea99:::97:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9993c9a7:::98:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6394211c:::99:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:94c7ae57:::100:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6fdee5bb:::101:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9a477fd1:::102:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:eb850916:::103:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:affc56b9:::104:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b42dc814:::105:head 2024-04-20T02:21:15.391 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f319f8f0:::106:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9a40b9de:::107:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:8b524f28:::108:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e3de589f:::109:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:90f90a5b:::110:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a7b4f1d7:::111:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:af51766e:::112:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b6f90bd1:::113:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e0261208:::114:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c9569ef7:::115:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:61bebe50:::116:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:fe93412b:::117:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d3d38bee:::118:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3100ba0c:::119:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d0560ada:::120:head 2024-04-20T02:21:15.392 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f0ea8b35:::121:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:766f231a:::122:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a07a2582:::123:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:bd7c6b3a:::124:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:fb2ddaff:::125:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4408e1fe:::126:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ee1df7a7:::127:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c3002909:::128:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4f48ffa9:::129:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:edf38733:::130:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c08425c0:::131:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:5f902d98:::132:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:41ea2c93:::133:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:813cee13:::134:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:0131818d:::135:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:26ba5a85:::136:head 2024-04-20T02:21:15.393 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:381b8a5a:::137:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:28797e47:::138:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:bfca7f22:::139:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:36807075:::140:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:80b03975:::141:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:5c15709b:::142:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f39ea15e:::143:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ea992956:::144:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:48887b1c:::145:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:9f24a9dd:::146:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:987f100b:::147:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d2dd3581:::148:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7fed1808:::149:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c80b70e9:::150:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:85ed90f9:::151:head 2024-04-20T02:21:15.394 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:36428b24:::152:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d044c34a:::153:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7c18bf58:::154:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d1c21232:::155:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a7a3c575:::156:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:87da0633:::157:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d5ac3822:::158:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3f20522d:::159:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6ca26563:::160:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:532ce135:::161:head 2024-04-20T02:21:15.395 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:c78863e6:::162:head 2024-04-20T02:21:56.109 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7462ddf6:::io.RoundTripAppendPP (2982 ms) 2024-04-20T02:21:56.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-04-20T02:21:56.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3008 ms) 2024-04-20T02:21:56.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-04-20T02:21:56.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3027 ms) 2024-04-20T02:21:56.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-04-20T02:21:56.110 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3001 ms) 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: setting pool EIO 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: max_success 99, min_failed 100 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (3992 ms) 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (2989 ms) 2024-04-20T02:21:56.111 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (143048 ms total) 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3024 ms) 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (8986 ms) 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14985 ms) 2024-04-20T02:21:56.112 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3041 ms) 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30036 ms total) 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3028 ms) 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3028 ms total) 2024-04-20T02:21:56.113 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (22806 ms) 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (10207 ms) 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (35873 ms) 2024-04-20T02:21:56.114 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (6996 ms) 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (5994 ms) 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7045 ms) 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6994 ms) 2024-04-20T02:21:56.115 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6979 ms) 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7056 ms) 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6977 ms) 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (6962 ms) 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-04-20T02:21:56.116 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (6993 ms) 2024-04-20T02:21:56.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-04-20T02:21:56.117 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7079 ms) 2024-04-20T02:22:10.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFull163:head 2024-04-20T02:22:10.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:5d165639:::164:head 2024-04-20T02:22:10.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f43765fc:::165:head 2024-04-20T02:22:10.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b4c720e9:::166:head 2024-04-20T02:22:10.038 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e694b040:::167:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:afa38db2:::168:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:77ba9f53:::169:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:87495034:::170:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7c96bf0e:::171:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:dbe346cc:::172:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:e943ec24:::173:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f97a9c0c:::174:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:6f26e74d:::175:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4f95e106:::176:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:0e6f2f8f:::177:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:05db05f1:::178:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:38a78d66:::179:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:d095610b:::180:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:a1a9d709:::181:head 2024-04-20T02:22:10.039 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:1e5d39db:::182:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:f7df4fb9:::183:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:03a7f161:::184:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ba70721e:::185:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:28e5662d:::186:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:973d52de:::187:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:4303eb1c:::188:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:b990b48e:::189:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:29b8165b:::190:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:3547f197:::191:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:7e260936:::192:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:1abec7b1:::193:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:10fdda93:::194:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:15817eea:::195:head 2024-04-20T02:22:10.040 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:770bab57:::196:head 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:ed9e13e7:::197:head 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:71471a8f:::198:head 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: checking for 189:10fb1d02:::199:head 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8021 ms) 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: first is 1713579685 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,1713579690,1713579691,0 2024-04-20T02:22:10.041 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,1713579690,1713579691,0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,1713579690,1713579691,0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,1713579690,1713579691,1713579693,1713579694,0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,1713579690,1713579691,1713579693,1713579694,0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579685,1713579687,1713579688,1713579690,1713579691,1713579693,1713579694,0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713579688,1713579690,1713579691,1713579693,1713579694,1713579696,1713579697,0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: first now 1713579688, trimmed 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21023 ms) 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: foo0 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-04-20T02:22:10.042 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (13985 ms) 2024-04-20T02:22:10.043 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-04-20T02:22:13.229 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-04-20T02:22:13.229 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-04-20T02:22:13.229 INFO:tasks.ceph.mon.d:Restarting daemon 2024-04-20T02:22:13.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-04-20T02:22:13.232 INFO:tasks.ceph.mon.d:Started 2024-04-20T02:22:13.233 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:22:13.233 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-04-20T02:22:13.233 INFO:tasks.ceph.mon.i:Restarting daemon 2024-04-20T02:22:13.233 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-04-20T02:22:13.235 INFO:tasks.ceph.mon.i:Started 2024-04-20T02:22:13.235 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:22:13.236 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-20T02:22:13.236 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T02:22:13.236 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T02:22:13.238 INFO:tasks.ceph.mon.c:Started 2024-04-20T02:22:13.238 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-04-20T02:22:13.238 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-04-20T02:22:13.239 INFO:tasks.ceph.mon.f:Restarting daemon 2024-04-20T02:22:13.239 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-04-20T02:22:13.241 INFO:tasks.ceph.mon.f:Started 2024-04-20T02:22:13.241 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-04-20T02:22:13.241 INFO:tasks.ceph.mon.h:Sent signal 19 2024-04-20T02:22:13.241 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:22:13.354 INFO:tasks.ceph.mon.i.smithi179.stderr:2024-04-20T02:22:13.353+0000 7fc66c3f8cc0 -1 mon.i@-1(???).log v0 log_external_backlog rewinding external_log_to from 197 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:22:28.242 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-04-20T02:22:28.243 INFO:tasks.ceph.mon.h:Sent signal 18 2024-04-20T02:22:28.243 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:22:28.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.386+0000 7f3693fac700 10 monclient: get_monmap_and_config 2024-04-20T02:22:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.386+0000 7f3693fac700 10 monclient: build_initial_monmap 2024-04-20T02:22:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.386+0000 7f3693fac700 10 monclient: monmap: 2024-04-20T02:22:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:28.387708+0000 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:28.387708+0000 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:28.388 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 1 Processor -- start 2024-04-20T02:22:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 1 -- start start 2024-04-20T02:22:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 10 monclient: init 2024-04-20T02:22:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 10 monclient: _add_conns ranks=[7,10,3,8,0,9,4,6,2,5,1] 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.387+0000 7f3693fac700 10 monclient(hunting): picked mon.noname-i con 0x7f368c0a17e0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): picked mon.noname-f con 0x7f368c0709d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): picked mon.noname-g con 0x7f368c1360a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c071e80 con 0x7f368c0709d0 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c072160 con 0x7f368c1360a0 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f368c067070 con 0x7f368c0a17e0 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3693fac700 10 monclient(hunting): authenticate will time out at 2027.789084s 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:56554/0 (socket says 172.21.15.47:56554) 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:57162/0 (socket says 172.21.15.47:57162) 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 learned_addr learned my addr 172.21.15.47:0/3173899749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 10 monclient(hunting): get_auth_request con 0x7f368c1360a0 auth_method 0 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 10 monclient(hunting): get_auth_request con 0x7f368c0a17e0 auth_method 0 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 10 monclient(hunting): get_auth_request con 0x7f368c0709d0 auth_method 0 2024-04-20T02:22:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.388+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 10 monclient(hunting): get_auth_request con 0x7f368c1360a0 auth_method 0 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 10 monclient(hunting): get_auth_request con 0x7f368c0709d0 auth_method 0 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 10 monclient(hunting): get_auth_request con 0x7f368c0a17e0 auth_method 0 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.589+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 10 monclient(hunting): get_auth_request con 0x7f368c1360a0 auth_method 0 2024-04-20T02:22:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 10 monclient(hunting): get_auth_request con 0x7f368c0709d0 auth_method 0 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 10 monclient(hunting): get_auth_request con 0x7f368c0a17e0 auth_method 0 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:28.990+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 10 monclient(hunting): get_auth_request con 0x7f368c0709d0 auth_method 0 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 10 monclient(hunting): get_auth_request con 0x7f368c1360a0 auth_method 0 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 10 monclient(hunting): get_auth_request con 0x7f368c0a17e0 auth_method 0 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3692549700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:22:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:29.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:29.791+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): tick 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 monclient(hunting): continuing hunt 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 msgr2=0x7f368c0a1bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f368c0a17e0 0x7f368c0a1bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 msgr2=0x7f368c0a38f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c1360a0 0x7f368c0a38f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 -- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 msgr2=0x7f368c135b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 --2- 172.21.15.47:0/3173899749 >> v2:172.21.15.179:3300/0 conn(0x7f368c0709d0 0x7f368c135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient: _add_conns ranks=[4,2,1,0,8,9,5,6] 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 0x7f3678005cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): picked mon.noname-c con 0x7f3678003880 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36780062b0 0x7f3678008720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): picked mon.noname-b con 0x7f36780062b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3678008df0 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 -- 172.21.15.47:0/3173899749 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0751e0 con 0x7f3678008df0 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 -- 172.21.15.47:0/3173899749 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c071e80 con 0x7f36780062b0 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 1 -- 172.21.15.47:0/3173899749 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f368c072160 con 0x7f3678003880 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f36837fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36780062b0 0x7f3678008720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:31.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 0x7f3678005cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 10 monclient(hunting): get_auth_request con 0x7f3678003880 auth_method 0 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): get_auth_request con 0x7f36780062b0 auth_method 0 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 msgr2=0x7f3678005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 msgr2=0x7f3678005cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 0x7f3678005cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691d48700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 0x7f3678005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:31.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient(hunting): handle_auth_done global_id 23816 payload 274 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 msgr2=0x7f3678005cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3678003880 0x7f3678005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 >> v1:172.21.15.179:6789/0 conn(0x7f3678008df0 legacy=0x7f367800b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient: _finish_hunting 0 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 1 monclient: found mon.noname-b 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3692549700 1 -- 172.21.15.47:0/3173899749 reap_dead start 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.388+0000 7f3691547700 1 -- 172.21.15.47:0/3173899749 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367800dd00 con 0x7f36780062b0 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3691547700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36780062b0 0x7f3678008720 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f368c071060 tx=0x7f367c010c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=951c244200b2a592 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 1 -- 172.21.15.47:0/3173899749 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f367c010e90 con 0x7f36780062b0 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:31.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: dump: 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:31.395 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: _finish_auth 0 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: _check_auth_tickets 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 1 -- 172.21.15.47:0/3173899749 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c00c070 con 0x7f36780062b0 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 1 -- 172.21.15.47:0/3173899749 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f367c014c20 con 0x7f36780062b0 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3693fac700 5 monclient: authenticate success, global_id 23816 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3693fac700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3693fac700 4 set_mon_vals no callback set 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.389+0000 7f3690d46700 10 monclient: dump: 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:31.396 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:31.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 10 monclient: shutdown 2024-04-20T02:22:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 -- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36780062b0 msgr2=0x7f3678008720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36780062b0 0x7f3678008720 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f368c071060 tx=0x7f367c010c50 comp rx=0 tx=0).stop 2024-04-20T02:22:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 -- 172.21.15.47:0/3173899749 shutdown_connections 2024-04-20T02:22:31.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 --2- 172.21.15.47:0/3173899749 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f36780062b0 0x7f3678008720 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 -- 172.21.15.47:0/3173899749 >> 172.21.15.47:0/3173899749 conn(0x7f368c0a5040 msgr2=0x7f368c066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 -- 172.21.15.47:0/3173899749 shutdown_connections 2024-04-20T02:22:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.390+0000 7f3693fac700 1 -- 172.21.15.47:0/3173899749 wait complete. 2024-04-20T02:22:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.391+0000 7f3693fac700 10 monclient: build_initial_monmap 2024-04-20T02:22:31.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.391+0000 7f3693fac700 10 monclient: monmap: 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:31.401 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:31.402 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.391+0000 7f3693fac700 1 librados: starting msgr at 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.391+0000 7f3693fac700 1 librados: starting objecter 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 Processor -- start 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 -- start start 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 librados: setting wanted keys 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 librados: calling monclient init 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient: init 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient: _add_conns ranks=[5,3,8,4,2,0,6,1,7] 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:31.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): picked mon.noname-h con 0x7f36780062b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c134bc0 0x7f368c135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): picked mon.noname-g con 0x7f368c134bc0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f368c1c8a90 0x7f368c1c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): picked mon.noname-e con 0x7f368c1c8a90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c09ed30 con 0x7f368c134bc0 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f368c135540 con 0x7f36780062b0 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1230f0 con 0x7f368c1c8a90 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.392+0000 7f3693fac700 10 monclient(hunting): authenticate will time out at 2030.793808s 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f368c1c8a90 0x7f368c1c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c134bc0 0x7f368c135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:39156/0 (socket says 172.21.15.47:39156) 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 -- 172.21.15.47:0/410307087 learned_addr learned my addr 172.21.15.47:0/410307087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): get_auth_request con 0x7f368c1c8a90 auth_method 0 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): get_auth_request con 0x7f368c134bc0 auth_method 0 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 10 monclient(hunting): get_auth_request con 0x7f36780062b0 auth_method 0 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:31.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 -- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 msgr2=0x7f368c1cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 -- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 msgr2=0x7f368c1cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691d48700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient(hunting): handle_auth_done global_id 21909 payload 824 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 -- 172.21.15.47:0/410307087 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f368c1c8a90 msgr2=0x7f368c1c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f368c1c8a90 0x7f368c1c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 -- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 msgr2=0x7f368c1cbfd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient: _finish_hunting 0 2024-04-20T02:22:31.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 monclient: found mon.noname-g 2024-04-20T02:22:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:22:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 -- 172.21.15.47:0/410307087 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c00e040 con 0x7f368c134bc0 2024-04-20T02:22:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3692549700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f368c1c8a90 0x7f368c1c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f3691547700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c134bc0 0x7f368c135000 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f3678008dc0 tx=0x7f367c004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f36827fc700 10 client.?.objecter ms_handle_connect 0x7f368c134bc0 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.393+0000 7f36827fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f367c026070 con 0x7f368c134bc0 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: dump: 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.411 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:31.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c010e90 con 0x7f368c134bc0 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 5 monclient: authenticate success, global_id 21909 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 10 monclient: _renew_subs 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c123d20 con 0x7f368c134bc0 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 10 client.21909.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 10 monclient: _renew_subs 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1242e0 con 0x7f368c134bc0 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3693fac700 1 librados: init done 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f367c017cc0 con 0x7f368c134bc0 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f3683fff700 4 set_mon_vals no callback set 2024-04-20T02:22:31.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.395+0000 7f36827fc700 10 monclient: dump: 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:31.414 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.396+0000 7f3693fac700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.396+0000 7f3693fac700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.396+0000 7f3693fac700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.396+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36740050e0 con 0x7f368c134bc0 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.396+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f367c017850 con 0x7f368c134bc0 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f367803b650 0x7f367803db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(292..292 src has 1..292) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f367c04bd10 con 0x7f368c134bc0 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 10 client.21909.objecter ms_dispatch 0x7f368c001550 osd_map(292..292 src has 1..292) v4 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 3 client.21909.objecter handle_osd_map got epochs [292,292] > 0 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 3 client.21909.objecter handle_osd_map decoding full epoch 292 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 20 client.21909.objecter dump_active .. 0 homeless 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f3691d48700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f367803b650 0x7f367803db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:31.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f3691d48700 10 monclient: get_auth_request con 0x7f367803b650 auth_method 0 2024-04-20T02:22:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f36827fc700 10 client.21909.objecter ms_handle_connect 0x7f367803b650 2024-04-20T02:22:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.397+0000 7f3691d48700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f367803b650 0x7f367803db10 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7f3688014370 tx=0x7f3688008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.402+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f367c017b00 con 0x7f368c134bc0 2024-04-20T02:22:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.402+0000 7f36827fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:31.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.402+0000 7f36827fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:31.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.709+0000 7f3693fac700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:31.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.709+0000 7f3693fac700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.709+0000 7f3693fac700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:22:31.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.709+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3674005390 con 0x7f368c134bc0 2024-04-20T02:22:31.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.710+0000 7f36827fc700 1 -- 172.21.15.47:0/410307087 <== mon.3 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3169 (secure 0 0 0) 0x7f367c029820 con 0x7f368c134bc0 2024-04-20T02:22:31.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.710+0000 7f36827fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:31.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.710+0000 7f36827fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:31.711 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":48,"quorum":[0,1,2,3,4,6,7,8],"quorum_names":["a","f","b","g","c","d","i","e"],"quorum_leader_name":"g","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 10 librados: watch_flush enter 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 10 librados: watch_flush exit 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 20 client.21909.objecter shutdown clearing up homeless session... 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 10 client.21909.objecter successfully canceled tick 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f367803b650 msgr2=0x7f367803db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f367803b650 0x7f367803db10 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7f3688014370 tx=0x7f3688008040 comp rx=0 tx=0).stop 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 10 monclient: shutdown 2024-04-20T02:22:31.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c134bc0 msgr2=0x7f368c135000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c134bc0 0x7f368c135000 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f3678008dc0 tx=0x7f367c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 shutdown_connections 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f367803b650 0x7f367803db10 unknown :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f368c1c8a90 0x7f368c1c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f36780062b0 0x7f368c1cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 --2- 172.21.15.47:0/410307087 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f368c134bc0 0x7f368c135000 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 >> 172.21.15.47:0/410307087 conn(0x7f368c0a5040 msgr2=0x7f368c09fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 shutdown_connections 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 -- 172.21.15.47:0/410307087 wait complete. 2024-04-20T02:22:31.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:31.712+0000 7f3693fac700 1 librados: shutdown 2024-04-20T02:22:34.726 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.870+0000 7fc0400af700 10 monclient: get_monmap_and_config 2024-04-20T02:22:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.870+0000 7fc0400af700 10 monclient: build_initial_monmap 2024-04-20T02:22:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.870+0000 7fc0400af700 10 monclient: monmap: 2024-04-20T02:22:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:34.870 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:34.871316+0000 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:34.871316+0000 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:34.871 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.870+0000 7fc0400af700 1 Processor -- start 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 -- start start 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient: init 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient: _add_conns ranks=[9,6,8,4,3,2,7,5,10,1,0] 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 0x7fc0380a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): picked mon.noname-a con 0x7fc0380a5080 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0380a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): picked mon.noname-d con 0x7fc0380a1a80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0380a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): picked mon.noname-e con 0x7fc0380a3dc0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:34.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc038072380 con 0x7fc0380a5080 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380a5460 con 0x7fc0380a1a80 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc038070b10 con 0x7fc0380a3dc0 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc0400af700 10 monclient(hunting): authenticate will time out at 2034.272611s 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 0x7fc0380a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0380a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0380a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0380a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48122/0 (socket says 172.21.15.47:48122) 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 0x7fc0380a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60842/0 (socket says 172.21.15.47:60842) 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0380a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58218/0 (socket says 172.21.15.47:58218) 2024-04-20T02:22:34.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 1 -- 172.21.15.47:0/91230508 learned_addr learned my addr 172.21.15.47:0/91230508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): get_auth_request con 0x7fc0380a1a80 auth_method 0 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): get_auth_request con 0x7fc0380a5080 auth_method 0 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): get_auth_request con 0x7fc0380a3dc0 auth_method 0 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03e64c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:34.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03de4b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.871+0000 7fc03d64a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 10 monclient(hunting): handle_auth_done global_id 35000 payload 274 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 -- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 msgr2=0x7fc0380a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 --2- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0380a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 -- 172.21.15.47:0/91230508 >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 msgr2=0x7fc0380a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 --2- 172.21.15.47:0/91230508 >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 0x7fc0380a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 10 monclient: _finish_hunting 0 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 monclient: found mon.noname-e 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:34.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 -- 172.21.15.47:0/91230508 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380751e0 con 0x7fc0380a3dc0 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03de4b700 1 --2- 172.21.15.47:0/91230508 >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 0x7fc0380a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03e64c700 1 --2- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0380a4270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03000b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=67c16fadfa1204f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 1 -- 172.21.15.47:0/91230508 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc03000e070 con 0x7fc0380a3dc0 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: dump: 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:34.885 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: _finish_auth 0 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: _check_auth_tickets 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 1 -- 172.21.15.47:0/91230508 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0300098f0 con 0x7fc0380a3dc0 2024-04-20T02:22:34.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc0400af700 5 monclient: authenticate success, global_id 35000 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc0400af700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 1 -- 172.21.15.47:0/91230508 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0300128a0 con 0x7fc0380a3dc0 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc03ce49700 10 monclient: dump: 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:34.887 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:34.888 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc0400af700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.872+0000 7fc0400af700 4 set_mon_vals no callback set 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 10 monclient: shutdown 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 -- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 msgr2=0x7fc0380a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 --2- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0380a4270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03000b330 comp rx=0 tx=0).stop 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 -- 172.21.15.47:0/91230508 shutdown_connections 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 --2- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0380a4270 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 --2- 172.21.15.47:0/91230508 >> v2:172.21.15.47:3300/0 conn(0x7fc0380a5080 0x7fc0380a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 --2- 172.21.15.47:0/91230508 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0380a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 -- 172.21.15.47:0/91230508 >> 172.21.15.47:0/91230508 conn(0x7fc0380a84a0 msgr2=0x7fc0380a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 -- 172.21.15.47:0/91230508 shutdown_connections 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 1 -- 172.21.15.47:0/91230508 wait complete. 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 10 monclient: build_initial_monmap 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.873+0000 7fc0400af700 10 monclient: monmap: 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:34.890 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 librados: starting msgr at 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 librados: starting objecter 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 Processor -- start 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 -- start start 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 librados: setting wanted keys 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 librados: calling monclient init 2024-04-20T02:22:34.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient: init 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient: _add_conns ranks=[6,8,4,7,2,3,5,1,0] 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0381cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): picked mon.noname-d con 0x7fc0380a1a80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0381ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): picked mon.noname-e con 0x7fc0380a3dc0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc0380a5080 0x7fc03809d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): picked mon.noname-c con 0x7fc0380a5080 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc038124700 con 0x7fc0380a5080 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc038124c30 con 0x7fc0380a1a80 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380a21e0 con 0x7fc0380a3dc0 2024-04-20T02:22:34.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc0400af700 10 monclient(hunting): authenticate will time out at 2034.275857s 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc03de4b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0381cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc03d64a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0381ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc03e64c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc0380a5080 0x7fc03809d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc03de4b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0381cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48128/0 (socket says 172.21.15.47:48128) 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc03d64a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0381ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58220/0 (socket says 172.21.15.47:58220) 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.874+0000 7fc03de4b700 1 -- 172.21.15.47:0/1220817331 learned_addr learned my addr 172.21.15.47:0/1220817331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): get_auth_request con 0x7fc0380a1a80 auth_method 0 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): get_auth_request con 0x7fc0380a3dc0 auth_method 0 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:34.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): get_auth_request con 0x7fc0380a5080 auth_method 0 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:34.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03e64c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient(hunting): handle_auth_done global_id 54114 payload 824 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 -- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 msgr2=0x7fc0381ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0381ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 -- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc0380a5080 msgr2=0x7fc03809d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc0380a5080 0x7fc03809d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient: _finish_hunting 0 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 monclient: found mon.noname-d 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 -- 172.21.15.47:0/1220817331 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340094a0 con 0x7fc0380a1a80 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03d64a700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0381ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc03de4b700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0381cc840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc03400ebe0 tx=0x7fc03400eef0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 client.?.objecter ms_handle_connect 0x7fc0380a1a80 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:34.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc03400ccc0 con 0x7fc0380a1a80 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: dump: 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:34.896 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400ce40 con 0x7fc0380a1a80 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 5 monclient: authenticate success, global_id 54114 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0340138e0 con 0x7fc0380a1a80 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02ffff700 4 set_mon_vals no callback set 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc02e7fc700 10 monclient: dump: 2024-04-20T02:22:34.897 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:34.898 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 10 monclient: _renew_subs 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc030009a70 con 0x7fc0380a1a80 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 10 client.54114.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 10 monclient: _renew_subs 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038124f10 con 0x7fc0380a1a80 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.875+0000 7fc0400af700 1 librados: init done 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.876+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc034013470 con 0x7fc0380a1a80 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.876+0000 7fc0400af700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:34.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.876+0000 7fc0400af700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.876+0000 7fc0400af700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.876+0000 7fc02e7fc700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc01c03a320 0x7fc01c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.876+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0381249e0 con 0x7fc0380a1a80 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(296..296 src has 1..296) v4 ==== 7028+0+0 (secure 0 0 0) 0x7fc03404abd0 con 0x7fc0380a1a80 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc02e7fc700 10 client.54114.objecter ms_dispatch 0x7fc038001550 osd_map(296..296 src has 1..296) v4 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc02e7fc700 3 client.54114.objecter handle_osd_map got epochs [296,296] > 0 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc02e7fc700 3 client.54114.objecter handle_osd_map decoding full epoch 296 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc02e7fc700 20 client.54114.objecter dump_active .. 0 homeless 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc03d64a700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc01c03a320 0x7fc01c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc03d64a700 10 monclient: get_auth_request con 0x7fc01c03a320 auth_method 0 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc03d64a700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc01c03a320 0x7fc01c03c7e0 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 crypto rx=0x7fc03806a700 tx=0x7fc0280092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:34.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.877+0000 7fc02e7fc700 10 client.54114.objecter ms_handle_connect 0x7fc01c03a320 2024-04-20T02:22:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.882+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc03401b3b0 con 0x7fc0380a1a80 2024-04-20T02:22:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.882+0000 7fc02e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:34.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:34.882+0000 7fc02e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:35.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.185+0000 7fc0400af700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.185+0000 7fc0400af700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.185+0000 7fc0400af700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.185+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc038123490 con 0x7fc0380a1a80 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.186+0000 7fc02e7fc700 1 -- 172.21.15.47:0/1220817331 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fc034017080 con 0x7fc0380a1a80 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.186+0000 7fc02e7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.186+0000 7fc02e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:35.186 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 10 librados: watch_flush enter 2024-04-20T02:22:35.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 10 librados: watch_flush exit 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 20 client.54114.objecter shutdown clearing up homeless session... 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 10 client.54114.objecter successfully canceled tick 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc01c03a320 msgr2=0x7fc01c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc01c03a320 0x7fc01c03c7e0 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 crypto rx=0x7fc03806a700 tx=0x7fc0280092c0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 10 monclient: shutdown 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 msgr2=0x7fc0381cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0381cc840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc03400ebe0 tx=0x7fc03400eef0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 shutdown_connections 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc01c03a320 0x7fc01c03c7e0 unknown :-1 s=CLOSED pgs=3172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc0380a5080 0x7fc03809d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0380a1a80 0x7fc0381cc840 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 --2- 172.21.15.47:0/1220817331 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc0380a3dc0 0x7fc0381ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 >> 172.21.15.47:0/1220817331 conn(0x7fc0380a84a0 msgr2=0x7fc038132d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 shutdown_connections 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 -- 172.21.15.47:0/1220817331 wait complete. 2024-04-20T02:22:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.188+0000 7fc0400af700 1 librados: shutdown 2024-04-20T02:22:35.201 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:22:35.201 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:22:35.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.338+0000 7f0bf8ab3700 10 monclient: get_monmap_and_config 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.338+0000 7f0bf8ab3700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.338+0000 7f0bf8ab3700 10 monclient: monmap: 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:35.339233+0000 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:35.339233+0000 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.338 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:35.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.338+0000 7f0bf8ab3700 1 Processor -- start 2024-04-20T02:22:35.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.338+0000 7f0bf8ab3700 1 -- start start 2024-04-20T02:22:35.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.338+0000 7f0bf8ab3700 10 monclient: init 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient: _add_conns ranks=[10,6,0,7,2,9,5,3,1,8,4] 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0bf4137d90 0x7f0bf40a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): picked mon.noname-f con 0x7f0bf4137d90 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bf4138170 0x7f0bf40a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): picked mon.noname-d con 0x7f0bf4138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bf41385b0 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40751e0 con 0x7f0bf41385b0 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf409fa40 con 0x7f0bf4137d90 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf409dfc0 con 0x7f0bf4138170 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf8ab3700 10 monclient(hunting): authenticate will time out at 2034.740496s 2024-04-20T02:22:35.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bf4138170 0x7f0bf40a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf2d9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0bf41385b0 0x7f0bf40a61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46238/0 (socket says 172.21.15.47:46238) 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf2d9d700 1 -- 172.21.15.47:0/2008366861 learned_addr learned my addr 172.21.15.47:0/2008366861 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): get_auth_request con 0x7f0bf4138170 auth_method 0 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 1 --2- 172.21.15.47:0/2008366861 >> v2:172.21.15.179:3300/0 conn(0x7f0bf4137d90 0x7f0bf40a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): get_auth_request con 0x7f0bf4137d90 auth_method 0 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf159a700 1 -- 172.21.15.47:0/2008366861 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708439747 0 0) 0x7f0bf40751e0 con 0x7f0bf41385b0 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf159a700 10 monclient(hunting): _negotiate 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf159a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf159a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf159a700 10 monclient(hunting): my global_id is 35253 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.339+0000 7f0bf159a700 1 -- 172.21.15.47:0/2008366861 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bdc0035b0 con 0x7f0bf41385b0 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_done global_id 54123 payload 274 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 1 -- 172.21.15.47:0/2008366861 >> v2:172.21.15.179:3300/0 conn(0x7f0bf4137d90 msgr2=0x7f0bf40a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/2008366861 >> v2:172.21.15.179:3300/0 conn(0x7f0bf4137d90 0x7f0bf40a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 1 -- 172.21.15.47:0/2008366861 >> v1:172.21.15.47:6789/0 conn(0x7f0bf41385b0 legacy=0x7f0bf40a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 1 monclient: found mon.noname-d 2024-04-20T02:22:35.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 1 -- 172.21.15.47:0/2008366861 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40a7bc0 con 0x7f0bf4138170 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/2008366861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bf4138170 0x7f0bf40a5c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009840 tx=0x7f0be8009b50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6541ec0a80d94aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 1 -- 172.21.15.47:0/2008366861 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0be8011aa0 con 0x7f0bf4138170 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: dump: 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.354 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: _finish_auth 0 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: _check_auth_tickets 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 1 -- 172.21.15.47:0/2008366861 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8011c00 con 0x7f0bf4138170 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 5 monclient: authenticate success, global_id 54123 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:35.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 1 -- 172.21.15.47:0/2008366861 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0be8011d70 con 0x7f0bf4138170 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 4 set_mon_vals no callback set 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf159a700 10 monclient: dump: 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.356 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 10 monclient: shutdown 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/2008366861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bf4138170 msgr2=0x7f0bf40a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.340+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/2008366861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bf4138170 0x7f0bf40a5c90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009840 tx=0x7f0be8009b50 comp rx=0 tx=0).stop 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/2008366861 shutdown_connections 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/2008366861 >> v2:172.21.15.179:3300/0 conn(0x7f0bf4137d90 0x7f0bf40a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/2008366861 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0bf4138170 0x7f0bf40a5c90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/2008366861 >> 172.21.15.47:0/2008366861 conn(0x7f0bf40aa8b0 msgr2=0x7f0bf40a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/2008366861 shutdown_connections 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/2008366861 wait complete. 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.341+0000 7f0bf8ab3700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient: monmap: 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:35.358 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 librados: starting msgr at 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 librados: starting objecter 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 Processor -- start 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 -- start start 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 librados: setting wanted keys 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 librados: calling monclient init 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient: init 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient: _add_conns ranks=[5,3,1,0,7,4,6,2,8] 2024-04-20T02:22:35.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0bf4137d90 0x7f0bf41cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient(hunting): picked mon.noname-h con 0x7f0bf4137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bf4138170 0x7f0bf4136da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient(hunting): picked mon.noname-g con 0x7f0bf4138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf41385b0 0x7f0bf41372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.342+0000 7f0bf8ab3700 10 monclient(hunting): picked mon.noname-f con 0x7f0bf41385b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4123ec0 con 0x7f0bf41385b0 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf409e9e0 con 0x7f0bf4138170 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf409dc20 con 0x7f0bf4137d90 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:35.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf8ab3700 10 monclient(hunting): authenticate will time out at 2034.744025s 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0bf4137d90 0x7f0bf41cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0bf4137d90 0x7f0bf41cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:41592/0 (socket says 172.21.15.47:41592) 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 -- 172.21.15.47:0/1771494794 learned_addr learned my addr 172.21.15.47:0/1771494794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): get_auth_request con 0x7f0bf4137d90 auth_method 0 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf41385b0 0x7f0bf41372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bf4138170 0x7f0bf4136da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): get_auth_request con 0x7f0bf41385b0 auth_method 0 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): get_auth_request con 0x7f0bf4138170 auth_method 0 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient(hunting): handle_auth_done global_id 44375 payload 824 2024-04-20T02:22:35.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bf4138170 msgr2=0x7f0bf4136da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0bf4138170 0x7f0bf4136da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf41385b0 msgr2=0x7f0bf41372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf41385b0 0x7f0bf41372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient: _finish_hunting 0 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 monclient: found mon.noname-h 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf259c700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80094a0 con 0x7f0bf4137d90 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.343+0000 7f0bf2d9d700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0bf41385b0 0x7f0bf41372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf259c700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0bf4137d90 0x7f0bf41cc0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9b0 tx=0x7f0be400ecc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 client.?.objecter ms_handle_connect 0x7f0bf4137d90 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0be4009710 con 0x7f0bf4137d90 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: dump: 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.364 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 5 monclient: authenticate success, global_id 44375 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be400ee50 con 0x7f0bf4137d90 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 10 monclient: _renew_subs 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf4122390 con 0x7f0bf4137d90 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0be400ccc0 con 0x7f0bf4137d90 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf0d99700 4 set_mon_vals no callback set 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 10 client.44375.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0be2ffd700 10 monclient: dump: 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 10 monclient: _renew_subs 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf4137820 con 0x7f0bf4137d90 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.344+0000 7f0bf8ab3700 1 librados: init done 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0be400c850 con 0x7f0bf4137d90 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf8ab3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 0x7f0bf41208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4068180 con 0x7f0bf406aac0 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 0x7f0bf41208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bd403a350 0x7f0bd403c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient: get_auth_request con 0x7f0bf406aac0 auth_method 0 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(296..296 src has 1..296) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f0be404ac00 con 0x7f0bf4137d90 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 10 client.44375.objecter ms_dispatch 0x7f0bf4001550 osd_map(296..296 src has 1..296) v4 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 3 client.44375.objecter handle_osd_map got epochs [296,296] > 0 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 3 client.44375.objecter handle_osd_map decoding full epoch 296 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf2d9d700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bd403a350 0x7f0bd403c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 20 client.44375.objecter dump_active .. 0 homeless 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf2d9d700 10 monclient: get_auth_request con 0x7f0bd403a350 auth_method 0 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_done global_id 35262 payload 824 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bf409f760 con 0x7f0bf406aac0 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 10 client.44375.objecter ms_handle_connect 0x7f0bf406aac0 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 0x7f0bf41208d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0bf409e9b0 tx=0x7f0be8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.345+0000 7f0be2ffd700 10 client.44375.objecter resend_mon_ops 2024-04-20T02:22:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 10 client.44375.objecter ms_handle_connect 0x7f0bd403a350 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0bf2d9d700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bd403a350 0x7f0bd403c810 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 crypto rx=0x7f0bec001560 tx=0x7f0bec001970 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0be8020070 con 0x7f0bf406aac0 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 10 monclient: dump: 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.369 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0be80044f0 con 0x7f0bf406aac0 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 msgr2=0x7f0bf41208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 0x7f0bf41208d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0bf409e9b0 tx=0x7f0be8004750 comp rx=0 tx=0).stop 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 msgr2=0x7f0bf41208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.346+0000 7f0be2ffd700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf406aac0 0x7f0bf41208d0 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0bf409e9b0 tx=0x7f0be8004750 comp rx=0 tx=0).stop 2024-04-20T02:22:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf8ab3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 0x7f0bf4121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf8ab3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf406a400 con 0x7f0bf4120e10 2024-04-20T02:22:35.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 0x7f0bf4121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient: get_auth_request con 0x7f0bf4120e10 auth_method 0 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 10 monclient(hunting): handle_auth_done global_id 35271 payload 824 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 1 -- 172.21.15.47:0/1771494794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0bf4121760 con 0x7f0bf4120e10 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0be2ffd700 10 client.44375.objecter ms_handle_connect 0x7f0bf4120e10 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0be2ffd700 10 client.44375.objecter resend_mon_ops 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0bf1d9b700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 0x7f0bf4121220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009b90 tx=0x7f0be8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0be8003dc0 con 0x7f0bf4120e10 2024-04-20T02:22:35.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0be2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0be2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.354+0000 7f0be2ffd700 10 monclient: dump: 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7f0be801b430 con 0x7f0bf4120e10 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 msgr2=0x7f0bf4121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 0x7f0bf4121220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009b90 tx=0x7f0be8011000 comp rx=0 tx=0).stop 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 msgr2=0x7f0bf4121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0be2ffd700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf4120e10 0x7f0bf4121220 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009b90 tx=0x7f0be8011000 comp rx=0 tx=0).stop 2024-04-20T02:22:35.375 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:35.376 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.378 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:35.379 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.380 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.382 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:35.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:35.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:35.393 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:35.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:35.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.401 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:35.401 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:35.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 10 librados: watch_flush enter 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 10 librados: watch_flush exit 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 20 client.44375.objecter shutdown clearing up homeless session... 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 10 client.44375.objecter successfully canceled tick 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bd403a350 msgr2=0x7f0bd403c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0bd403a350 0x7f0bd403c810 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 crypto rx=0x7f0bec001560 tx=0x7f0bec001970 comp rx=0 tx=0).stop 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 10 monclient: shutdown 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0bf4137d90 msgr2=0x7f0bf41cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 --2- 172.21.15.47:0/1771494794 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0bf4137d90 0x7f0bf41cc0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9b0 tx=0x7f0be400ecc0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf2d9d700 1 -- 172.21.15.47:0/1771494794 reap_dead start 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf2d9d700 1 -- 172.21.15.47:0/1771494794 reap_dead start 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 shutdown_connections 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 >> 172.21.15.47:0/1771494794 conn(0x7f0bf40aa8b0 msgr2=0x7f0bf41bf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 shutdown_connections 2024-04-20T02:22:35.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 -- 172.21.15.47:0/1771494794 wait complete. 2024-04-20T02:22:35.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.355+0000 7f0bf8ab3700 1 librados: shutdown 2024-04-20T02:22:35.404 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:35.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:22:35.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.545+0000 7fb46f25a700 10 monclient: get_monmap_and_config 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.545+0000 7fb46f25a700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.545+0000 7fb46f25a700 10 monclient: monmap: 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:35.546198+0000 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:35.546198+0000 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:35.545 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:35.546 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.545+0000 7fb46f25a700 1 Processor -- start 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 1 -- start start 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient: init 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient: _add_conns ranks=[1,0,9,5,8,4,3,10,7,2,6] 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb468137d90 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb4680a09e0 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb46813e3a0 0x7fb468140790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient(hunting): picked mon.noname-a con 0x7fb46813e3a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4680751e0 con 0x7fb4680a09e0 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46809dfc0 con 0x7fb468137d90 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46809fa40 con 0x7fb46813e3a0 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:35.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46f25a700 10 monclient(hunting): authenticate will time out at 2034.947552s 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb46813e3a0 0x7fb468140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb467fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb4680a09e0 0x7fb46813cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46254/0 (socket says 172.21.15.47:46254) 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb467fff700 1 -- 172.21.15.47:0/3414597463 learned_addr learned my addr 172.21.15.47:0/3414597463 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): get_auth_request con 0x7fb46813e3a0 auth_method 0 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.546+0000 7fb46d7f7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 10 monclient(hunting): handle_auth_done global_id 35280 payload 274 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 1 -- 172.21.15.47:0/3414597463 >> v1:172.21.15.179:6789/0 conn(0x7fb468137d90 legacy=0x7fb468138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 1 -- 172.21.15.47:0/3414597463 >> v1:172.21.15.47:6789/0 conn(0x7fb4680a09e0 legacy=0x7fb46813cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 10 monclient: _finish_hunting 0 2024-04-20T02:22:35.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 1 monclient: found mon.noname-a 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 1 -- 172.21.15.47:0/3414597463 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468138880 con 0x7fb46813e3a0 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46d7f7700 1 --2- 172.21.15.47:0/3414597463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46813e3a0 0x7fb468140790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb460009d90 tx=0x7fb46000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e410f6b09ebc64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 1 -- 172.21.15.47:0/3414597463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb46000e070 con 0x7fb46813e3a0 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: dump: 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:35.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 1 -- 172.21.15.47:0/3414597463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4600098f0 con 0x7fb46813e3a0 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46f25a700 5 monclient: authenticate success, global_id 35280 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46f25a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46f25a700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 1 -- 172.21.15.47:0/3414597463 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb460012d10 con 0x7fb46813e3a0 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb46f25a700 4 set_mon_vals no callback set 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.547+0000 7fb4677fe700 10 monclient: dump: 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.561 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.562 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 10 monclient: shutdown 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 -- 172.21.15.47:0/3414597463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46813e3a0 msgr2=0x7fb468140790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 --2- 172.21.15.47:0/3414597463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46813e3a0 0x7fb468140790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb460009d90 tx=0x7fb46000b330 comp rx=0 tx=0).stop 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 -- 172.21.15.47:0/3414597463 shutdown_connections 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 --2- 172.21.15.47:0/3414597463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb46813e3a0 0x7fb468140790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 -- 172.21.15.47:0/3414597463 >> 172.21.15.47:0/3414597463 conn(0x7fb4680aa8b0 msgr2=0x7fb4680a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 -- 172.21.15.47:0/3414597463 shutdown_connections 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 1 -- 172.21.15.47:0/3414597463 wait complete. 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.548+0000 7fb46f25a700 10 monclient: monmap: 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:35.563 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 librados: starting msgr at 2024-04-20T02:22:35.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 librados: starting objecter 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 Processor -- start 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 -- start start 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 librados: setting wanted keys 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 librados: calling monclient init 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient: init 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient: _add_conns ranks=[4,1,3,0,5,7,2,6,8] 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 0x7fb4680a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): picked mon.noname-c con 0x7fb4680a09e0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb468137d90 0x7fb4680a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): picked mon.noname-f con 0x7fb468137d90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb46813e3a0 0x7fb468135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): picked mon.noname-g con 0x7fb46813e3a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680a2230 con 0x7fb468137d90 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468137020 con 0x7fb46813e3a0 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb46809e9e0 con 0x7fb4680a09e0 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46f25a700 10 monclient(hunting): authenticate will time out at 2034.950909s 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.549+0000 7fb46cff6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 0x7fb4680a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 0x7fb4680a3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35884/0 (socket says 172.21.15.47:35884) 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 -- 172.21.15.47:0/4095725469 learned_addr learned my addr 172.21.15.47:0/4095725469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): get_auth_request con 0x7fb4680a09e0 auth_method 0 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb468137d90 0x7fb4680a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): get_auth_request con 0x7fb468137d90 auth_method 0 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient(hunting): handle_auth_done global_id 54112 payload 824 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb46813e3a0 msgr2=0x7fb468135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb46813e3a0 0x7fb468135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb468137d90 msgr2=0x7fb4680a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb468137d90 0x7fb4680a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient: _finish_hunting 0 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 monclient: found mon.noname-c 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460009a70 con 0x7fb4680a09e0 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46cff6700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 0x7fb4680a3c00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb45800c490 tx=0x7fb45800c7a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 client.?.objecter ms_handle_connect 0x7fb4680a09e0 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb458005540 con 0x7fb4680a09e0 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:35.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: dump: 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: _finish_auth 0 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: _check_auth_tickets 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458003680 con 0x7fb4680a09e0 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 5 monclient: authenticate success, global_id 54112 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 10 monclient: _renew_subs 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4680a26f0 con 0x7fb4680a09e0 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb458003c60 con 0x7fb4680a09e0 2024-04-20T02:22:35.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb466ffd700 4 set_mon_vals no callback set 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 10 client.54112.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb4657fa700 10 monclient: dump: 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.571 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 10 monclient: _renew_subs 2024-04-20T02:22:35.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb468136800 con 0x7fb4680a09e0 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.550+0000 7fb46f25a700 1 librados: init done 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb46f25a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb46f25a700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 0x7fb46806aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb46f25a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468068180 con 0x7fb4681ccd60 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb4580037f0 con 0x7fb4680a09e0 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 0x7fb46806aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient: get_auth_request con 0x7fb4681ccd60 auth_method 0 2024-04-20T02:22:35.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb4657fa700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb44803a2f0 0x7fb44803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(297..297 src has 1..297) v4 ==== 6615+0+0 (secure 0 0 0) 0x7fb458049b40 con 0x7fb4680a09e0 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb4657fa700 10 client.54112.objecter ms_dispatch 0x7fb468001550 osd_map(297..297 src has 1..297) v4 2024-04-20T02:22:35.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb4657fa700 3 client.54112.objecter handle_osd_map got epochs [297,297] > 0 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb4657fa700 3 client.54112.objecter handle_osd_map decoding full epoch 297 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.551+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 20 client.54112.objecter dump_active .. 0 homeless 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb467fff700 10 monclient(hunting): handle_auth_done global_id 23825 payload 824 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb46d7f7700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb44803a2f0 0x7fb44803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb467fff700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb46809e540 con 0x7fb4681ccd60 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb46d7f7700 10 monclient: get_auth_request con 0x7fb44803a2f0 auth_method 0 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 client.54112.objecter ms_handle_connect 0x7fb4681ccd60 2024-04-20T02:22:35.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb467fff700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 0x7fb46806aac0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb468137360 tx=0x7fb4500018a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 client.54112.objecter resend_mon_ops 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb450017070 con 0x7fb4681ccd60 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 monclient: dump: 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.576 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.577 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.578 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.578 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.578 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 client.54112.objecter ms_handle_connect 0x7fb44803a2f0 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb46d7f7700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb44803a2f0 0x7fb44803c7b0 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7fb460014040 tx=0x7fb460010000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb450012560 con 0x7fb4681ccd60 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:35.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 msgr2=0x7fb46806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 0x7fb46806aac0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb468137360 tx=0x7fb4500018a0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 msgr2=0x7fb46806aac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:35.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.552+0000 7fb4657fa700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681ccd60 0x7fb46806aac0 secure :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb468137360 tx=0x7fb4500018a0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb46f25a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb46f25a700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 0x7fb468135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb46f25a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680a24a0 con 0x7fb4681359b0 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 0x7fb468135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient: get_auth_request con 0x7fb4681359b0 auth_method 0 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 10 monclient(hunting): handle_auth_done global_id 23834 payload 824 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.560+0000 7fb467fff700 1 -- 172.21.15.47:0/4095725469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb4680a7bc0 con 0x7fb4681359b0 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 client.54112.objecter ms_handle_connect 0x7fb4681359b0 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 client.54112.objecter resend_mon_ops 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb467fff700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 0x7fb468135dc0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fb468137360 tx=0x7fb450000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb450023070 con 0x7fb4681359b0 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 monclient: dump: 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.584 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.585 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.586 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.587 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fb450012d70 con 0x7fb4681359b0 2024-04-20T02:22:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:35.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 msgr2=0x7fb468135dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 0x7fb468135dc0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fb468137360 tx=0x7fb450000ed0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 msgr2=0x7fb468135dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:35.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb4657fa700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb4681359b0 0x7fb468135dc0 secure :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fb468137360 tx=0x7fb450000ed0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.590 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:35.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:35.590 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:35.591 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:35.592 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:35.592 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:35.592 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.593 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:22:35.594 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:35.594 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:35.594 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:35.594 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.595 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.595 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.595 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.596 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:35.597 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.598 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.599 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.599 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.599 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.599 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.599 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.600 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.600 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:35.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.605 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:35.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:35.608 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:35.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:35.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:35.612 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.612 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.612 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.612 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:35.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.614 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:35.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:35.618 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.618 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.618 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.620 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:35.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:35.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:35.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:35.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.623 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:35.623 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:35.623 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.624 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:35.625 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb46f25a700 10 librados: watch_flush enter 2024-04-20T02:22:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb46f25a700 10 librados: watch_flush exit 2024-04-20T02:22:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb46f25a700 20 client.54112.objecter shutdown clearing up homeless session... 2024-04-20T02:22:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.561+0000 7fb46f25a700 10 client.54112.objecter successfully canceled tick 2024-04-20T02:22:35.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb44803a2f0 msgr2=0x7fb44803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb44803a2f0 0x7fb44803c7b0 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7fb460014040 tx=0x7fb460010000 comp rx=0 tx=0).stop 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 10 monclient: shutdown 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 msgr2=0x7fb4680a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 0x7fb4680a3c00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb45800c490 tx=0x7fb45800c7a0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46d7f7700 1 -- 172.21.15.47:0/4095725469 reap_dead start 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 shutdown_connections 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 --2- 172.21.15.47:0/4095725469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4680a09e0 0x7fb4680a3c00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 >> 172.21.15.47:0/4095725469 conn(0x7fb4680aa8b0 msgr2=0x7fb468134130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 shutdown_connections 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 -- 172.21.15.47:0/4095725469 wait complete. 2024-04-20T02:22:35.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.562+0000 7fb46f25a700 1 librados: shutdown 2024-04-20T02:22:35.629 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:35.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:22:35.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:22:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.772+0000 7f9611b69700 10 monclient: get_monmap_and_config 2024-04-20T02:22:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.772+0000 7f9611b69700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.772+0000 7f9611b69700 10 monclient: monmap: 2024-04-20T02:22:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.774 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:35.773631+0000 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:35.773631+0000 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:35.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:35.776 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:35.776 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 Processor -- start 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 -- start start 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient: init 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient: _add_conns ranks=[7,4,0,9,2,5,3,1,6,10,8] 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f960c0a3960 0x7f960c071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): picked mon.noname-i con 0x7f960c0a3960 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 0x7f960c137f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): picked mon.noname-c con 0x7f960c137b20 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f960c138460 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960c0751e0 con 0x7f960c138460 2024-04-20T02:22:35.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06e530 con 0x7f960c137b20 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c09ef40 con 0x7f960c0a3960 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.773+0000 7f9611b69700 10 monclient(hunting): authenticate will time out at 2035.174923s 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 0x7f960c137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f960c138460 0x7f960c13ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46270/0 (socket says 172.21.15.47:46270) 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960bfff700 1 -- 172.21.15.47:0/2610671136 learned_addr learned my addr 172.21.15.47:0/2610671136 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 0x7f960c137f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35892/0 (socket says 172.21.15.47:35892) 2024-04-20T02:22:35.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): get_auth_request con 0x7f960c137b20 auth_method 0 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f960c0a3960 0x7f960c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): get_auth_request con 0x7f960c0a3960 auth_method 0 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient(hunting): handle_auth_done global_id 54121 payload 274 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 -- 172.21.15.47:0/2610671136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f960c0a3960 msgr2=0x7f960c071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f960c0a3960 0x7f960c071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 1 -- 172.21.15.47:0/2610671136 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288824174 0 0) 0x7f960c0751e0 con 0x7f960c138460 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 -- 172.21.15.47:0/2610671136 >> v1:172.21.15.47:6789/0 conn(0x7f960c138460 legacy=0x7f960c13ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 monclient: found mon.noname-c 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 -- 172.21.15.47:0/2610671136 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0a3d40 con 0x7f960c137b20 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960affd700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 0x7f960c137f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009870 tx=0x7f95fc009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e18db26235416fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 1 -- 172.21.15.47:0/2610671136 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc011aa0 con 0x7f960c137b20 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: dump: 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.802 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f9611b69700 5 monclient: authenticate success, global_id 54121 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 1 -- 172.21.15.47:0/2610671136 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc011c00 con 0x7f960c137b20 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f9611b69700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f9611b69700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f9611b69700 4 set_mon_vals no callback set 2024-04-20T02:22:35.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 1 -- 172.21.15.47:0/2610671136 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc011d70 con 0x7f960c137b20 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.774+0000 7f960a7fc700 10 monclient: dump: 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.804 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 10 monclient: shutdown 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 -- 172.21.15.47:0/2610671136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 msgr2=0x7f960c137f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 0x7f960c137f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009870 tx=0x7f95fc009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 -- 172.21.15.47:0/2610671136 shutdown_connections 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f960c0a3960 0x7f960c071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 --2- 172.21.15.47:0/2610671136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c137b20 0x7f960c137f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 -- 172.21.15.47:0/2610671136 >> 172.21.15.47:0/2610671136 conn(0x7f960c0a4a50 msgr2=0x7f960c0a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 -- 172.21.15.47:0/2610671136 shutdown_connections 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.775+0000 7f9611b69700 1 -- 172.21.15.47:0/2610671136 wait complete. 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 10 monclient: monmap: 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.806 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:35.807 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 1 librados: starting msgr at 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 1 librados: starting objecter 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 1 Processor -- start 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 1 -- start start 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 1 librados: setting wanted keys 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 1 librados: calling monclient init 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.776+0000 7f9611b69700 10 monclient: init 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient: _add_conns ranks=[6,3,5,8,2,0,7,4,1] 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f960c0a3960 0x7f960c136c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): picked mon.noname-d con 0x7f960c0a3960 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f960c137b20 0x7f960c137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): picked mon.noname-g con 0x7f960c137b20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f960c138460 0x7f960c1cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): picked mon.noname-h con 0x7f960c138460 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c121c70 con 0x7f960c137b20 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0721d0 con 0x7f960c138460 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f960c124030 con 0x7f960c0a3960 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f9611b69700 10 monclient(hunting): authenticate will time out at 2035.178447s 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f960c0a3960 0x7f960c136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f960c0a3960 0x7f960c136c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48146/0 (socket says 172.21.15.47:48146) 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 -- 172.21.15.47:0/2271973079 learned_addr learned my addr 172.21.15.47:0/2271973079 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): get_auth_request con 0x7f960c0a3960 auth_method 0 2024-04-20T02:22:35.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f960c138460 0x7f960c1cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f960c137b20 0x7f960c137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): get_auth_request con 0x7f960c137b20 auth_method 0 2024-04-20T02:22:35.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): get_auth_request con 0x7f960c138460 auth_method 0 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient(hunting): handle_auth_done global_id 54132 payload 824 2024-04-20T02:22:35.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f960c138460 msgr2=0x7f960c1cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f960c138460 0x7f960c1cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f960c137b20 msgr2=0x7f960c137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f960c137b20 0x7f960c137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 monclient: found mon.noname-d 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.777+0000 7f960b7fe700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0094a0 con 0x7f960c0a3960 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f960b7fe700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f960c0a3960 0x7f960c136c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f95f400e870 tx=0x7f95f400eb80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 client.?.objecter ms_handle_connect 0x7f960c0a3960 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4009590 con 0x7f960c0a3960 2024-04-20T02:22:35.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: dump: 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.813 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 5 monclient: authenticate success, global_id 54132 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4013af0 con 0x7f960c0a3960 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 10 monclient: _renew_subs 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c06d9d0 con 0x7f960c0a3960 2024-04-20T02:22:35.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4013c80 con 0x7f960c0a3960 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9609ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 10 client.54132.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9603fff700 10 monclient: dump: 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.815 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.816 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 10 monclient: _renew_subs 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c123d50 con 0x7f960c0a3960 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.778+0000 7f9611b69700 1 librados: init done 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9611b69700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f95f4013680 con 0x7f960c0a3960 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9611b69700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 0x7f960c1cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960c068430 con 0x7f960c06aac0 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 0x7f960c1cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient: get_auth_request con 0x7f960c06aac0 auth_method 0 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f95ec03a2f0 0x7f95ec03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(297..297 src has 1..297) v4 ==== 6615+0+0 (secure 0 0 0) 0x7f95f404a860 con 0x7f960c0a3960 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 10 client.54132.objecter ms_dispatch 0x7f960c001550 osd_map(297..297 src has 1..297) v4 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 3 client.54132.objecter handle_osd_map got epochs [297,297] > 0 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 3 client.54132.objecter handle_osd_map decoding full epoch 297 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 10 monclient(hunting): handle_auth_done global_id 54130 payload 824 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f960c06dbe0 con 0x7f960c06aac0 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 20 client.54132.objecter dump_active .. 0 homeless 2024-04-20T02:22:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960bfff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f95ec03a2f0 0x7f95ec03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960bfff700 10 monclient: get_auth_request con 0x7f95ec03a2f0 auth_method 0 2024-04-20T02:22:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f960affd700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 0x7f960c1cca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f960c067510 tx=0x7f95fc004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 10 client.54132.objecter ms_handle_connect 0x7f960c06aac0 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 10 client.54132.objecter resend_mon_ops 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc020070 con 0x7f960c06aac0 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.779+0000 7f9603fff700 10 monclient: dump: 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.821 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 10 client.54132.objecter ms_handle_connect 0x7f95ec03a2f0 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f960bfff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f95ec03a2f0 0x7f95ec03c7b0 secure :-1 s=READY pgs=3175 cs=0 l=1 rev1=1 crypto rx=0x7f9604001d10 tx=0x7f9604001ca0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f95fc0036a0 con 0x7f960c06aac0 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 msgr2=0x7f960c1cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 0x7f960c1cca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f960c067510 tx=0x7f95fc004750 comp rx=0 tx=0).stop 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 msgr2=0x7f960c1cca40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.780+0000 7f9603fff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c06aac0 0x7f960c1cca40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9611b69700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9611b69700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 0x7f960c09f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9611b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960c06a400 con 0x7f960c1ccf80 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 0x7f960c09f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient: get_auth_request con 0x7f960c1ccf80 auth_method 0 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 10 monclient(hunting): handle_auth_done global_id 54139 payload 824 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 1 -- 172.21.15.47:0/2271973079 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f960c06e250 con 0x7f960c1ccf80 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9603fff700 10 client.54132.objecter ms_handle_connect 0x7f960c1ccf80 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f960affd700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 0x7f960c09f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc009210 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9603fff700 10 client.54132.objecter resend_mon_ops 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95fc02c300 con 0x7f960c1ccf80 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9603fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9603fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.788+0000 7f9603fff700 10 monclient: dump: 2024-04-20T02:22:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f95fc003e90 con 0x7f960c1ccf80 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 msgr2=0x7f960c09f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 0x7f960c09f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc009210 comp rx=0 tx=0).stop 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 msgr2=0x7f960c09f5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9603fff700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f960c1ccf80 0x7f960c09f5a0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc009210 comp rx=0 tx=0).stop 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:35.827 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.828 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.829 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:35.830 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:35.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:35.833 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:35.835 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:35.837 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.839 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.841 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.844 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:35.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:35.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:35.849 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:35.850 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:35.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 10 librados: watch_flush enter 2024-04-20T02:22:35.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 10 librados: watch_flush exit 2024-04-20T02:22:35.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 20 client.54132.objecter shutdown clearing up homeless session... 2024-04-20T02:22:35.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 10 client.54132.objecter successfully canceled tick 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f95ec03a2f0 msgr2=0x7f95ec03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f95ec03a2f0 0x7f95ec03c7b0 secure :-1 s=READY pgs=3175 cs=0 l=1 rev1=1 crypto rx=0x7f9604001d10 tx=0x7f9604001ca0 comp rx=0 tx=0).stop 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 10 monclient: shutdown 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f960c0a3960 msgr2=0x7f960c136c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 --2- 172.21.15.47:0/2271973079 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f960c0a3960 0x7f960c136c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f95f400e870 tx=0x7f95f400eb80 comp rx=0 tx=0).stop 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f960bfff700 1 -- 172.21.15.47:0/2271973079 reap_dead start 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f960bfff700 1 -- 172.21.15.47:0/2271973079 reap_dead start 2024-04-20T02:22:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 shutdown_connections 2024-04-20T02:22:35.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 >> 172.21.15.47:0/2271973079 conn(0x7f960c0a4a50 msgr2=0x7f960c070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:35.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 shutdown_connections 2024-04-20T02:22:35.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 -- 172.21.15.47:0/2271973079 wait complete. 2024-04-20T02:22:35.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.789+0000 7f9611b69700 1 librados: shutdown 2024-04-20T02:22:35.853 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:35.853 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:22:35.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:22:35.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.994+0000 7f4dc2126700 10 monclient: get_monmap_and_config 2024-04-20T02:22:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.994+0000 7f4dc2126700 10 monclient: build_initial_monmap 2024-04-20T02:22:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.994+0000 7f4dc2126700 10 monclient: monmap: 2024-04-20T02:22:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:35.998 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:35.996081+0000 2024-04-20T02:22:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:35.996081+0000 2024-04-20T02:22:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:35.999 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.000 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:36.001 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:36.001 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.995+0000 7f4dc2126700 1 Processor -- start 2024-04-20T02:22:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.995+0000 7f4dc2126700 1 -- start start 2024-04-20T02:22:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.995+0000 7f4dc2126700 10 monclient: init 2024-04-20T02:22:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient: _add_conns ranks=[2,4,6,0,1,10,7,5,8,3,9] 2024-04-20T02:22:36.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4dbc066610 0x7f4dbc0a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): picked mon.noname-b con 0x7f4dbc066610 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc0a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): picked mon.noname-c con 0x7f4dbc0a2450 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc13c6b0 0x7f4dbc13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): picked mon.noname-d con 0x7f4dbc13c6b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:36.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc06cd70 con 0x7f4dbc066610 2024-04-20T02:22:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc0669f0 con 0x7f4dbc0a2450 2024-04-20T02:22:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc09c420 con 0x7f4dbc13c6b0 2024-04-20T02:22:36.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dc2126700 10 monclient(hunting): authenticate will time out at 2035.397443s 2024-04-20T02:22:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc0a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc0a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35920/0 (socket says 172.21.15.47:35920) 2024-04-20T02:22:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 1 -- 172.21.15.47:0/2671988390 learned_addr learned my addr 172.21.15.47:0/2671988390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): get_auth_request con 0x7f4dbc0a2450 auth_method 0 2024-04-20T02:22:36.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4dbc066610 0x7f4dbc0a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc13c6b0 0x7f4dbc13eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): get_auth_request con 0x7f4dbc066610 auth_method 0 2024-04-20T02:22:36.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): get_auth_request con 0x7f4dbc13c6b0 auth_method 0 2024-04-20T02:22:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_done global_id 23843 payload 274 2024-04-20T02:22:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc13c6b0 msgr2=0x7f4dbc13eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:36.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc13c6b0 0x7f4dbc13eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 msgr2=0x7f4dbc0a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:36.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc0a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 monclient: found mon.noname-b 2024-04-20T02:22:36.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/2671988390 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc0751e0 con 0x7f4dbc066610 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.996+0000 7f4dbaffd700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc0a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4dbc066610 0x7f4dbc0a1f10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009870 tx=0x7f4da4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b441779d9b24f21d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 1 -- 172.21.15.47:0/2671988390 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4da4011aa0 con 0x7f4dbc066610 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: dump: 2024-04-20T02:22:36.038 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.039 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dc2126700 5 monclient: authenticate success, global_id 23843 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dc2126700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 1 -- 172.21.15.47:0/2671988390 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da4011c00 con 0x7f4dbc066610 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dc2126700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 1 -- 172.21.15.47:0/2671988390 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4da4011d70 con 0x7f4dbc066610 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dc2126700 4 set_mon_vals no callback set 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.997+0000 7f4dba7fc700 10 monclient: dump: 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.041 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 10 monclient: shutdown 2024-04-20T02:22:36.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 -- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4dbc066610 msgr2=0x7f4dbc0a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4dbc066610 0x7f4dbc0a1f10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009870 tx=0x7f4da4009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 -- 172.21.15.47:0/2671988390 shutdown_connections 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc13c6b0 0x7f4dbc13eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4dbc066610 0x7f4dbc0a1f10 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 --2- 172.21.15.47:0/2671988390 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc0a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 -- 172.21.15.47:0/2671988390 >> 172.21.15.47:0/2671988390 conn(0x7f4dbc0a3000 msgr2=0x7f4dbc0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 -- 172.21.15.47:0/2671988390 shutdown_connections 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 1 -- 172.21.15.47:0/2671988390 wait complete. 2024-04-20T02:22:36.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.998+0000 7f4dc2126700 10 monclient: monmap: 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:36.044 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 1 librados: starting msgr at 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 1 librados: starting objecter 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 1 Processor -- start 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 1 -- start start 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 1 librados: setting wanted keys 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 1 librados: calling monclient init 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 10 monclient: init 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:35.999+0000 7f4dc2126700 10 monclient: _add_conns ranks=[0,4,1,6,7,5,2,8,3] 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dbc066610 0x7f4dbc0902d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): picked mon.noname-a con 0x7f4dbc066610 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:36.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc090810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): picked mon.noname-c con 0x7f4dbc0a2450 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc13c6b0 0x7f4dbc090d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): picked mon.noname-f con 0x7f4dbc13c6b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc09ad70 con 0x7f4dbc066610 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc09a890 con 0x7f4dbc13c6b0 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc0927e0 con 0x7f4dbc0a2450 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 10 monclient(hunting): authenticate will time out at 2035.401088s 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dbc066610 0x7f4dbc0902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc090810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dbc066610 0x7f4dbc0902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60886/0 (socket says 172.21.15.47:60886) 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc090810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35932/0 (socket says 172.21.15.47:35932) 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/165893523 learned_addr learned my addr 172.21.15.47:0/165893523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): get_auth_request con 0x7f4dbc066610 auth_method 0 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): get_auth_request con 0x7f4dbc0a2450 auth_method 0 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc13c6b0 0x7f4dbc090d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): get_auth_request con 0x7f4dbc13c6b0 auth_method 0 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient(hunting): handle_auth_done global_id 35307 payload 824 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 msgr2=0x7f4dbc090810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc090810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc13c6b0 msgr2=0x7f4dbc090d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4dbc13c6b0 0x7f4dbc090d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 monclient: found mon.noname-a 2024-04-20T02:22:36.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009550 con 0x7f4dbc066610 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbaffd700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4dbc0a2450 0x7f4dbc090810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dbb7fe700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dbc066610 0x7f4dbc0902d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4da40090b0 tx=0x7f4da4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 client.?.objecter ms_handle_connect 0x7f4dbc066610 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4da40036a0 con 0x7f4dbc066610 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 monclient: dump: 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.051 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4dc2126700 5 monclient: authenticate success, global_id 35307 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da401f070 con 0x7f4dbc066610 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.000+0000 7f4db3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4da4003c90 con 0x7f4dbc066610 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 monclient: _renew_subs 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da40094a0 con 0x7f4dbc066610 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4db3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4db9ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4db3fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4db3fff700 10 monclient: dump: 2024-04-20T02:22:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.053 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.054 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 client.35307.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 monclient: _renew_subs 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc092220 con 0x7f4dbc066610 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 1 librados: init done 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 0x7f4dbc1cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc068230 con 0x7f4dbc06aac0 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.001+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4da4021030 con 0x7f4dbc066610 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 0x7f4dbc1cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient: get_auth_request con 0x7f4dbc06aac0 auth_method 0 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d9c03a2f0 0x7f4d9c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(297..297 src has 1..297) v4 ==== 6615+0+0 (secure 0 0 0) 0x7f4da404ba90 con 0x7f4dbc066610 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 client.35307.objecter ms_dispatch 0x7f4dbc001550 osd_map(297..297 src has 1..297) v4 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 3 client.35307.objecter handle_osd_map got epochs [297,297] > 0 2024-04-20T02:22:36.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 3 client.35307.objecter handle_osd_map decoding full epoch 297 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_done global_id 54150 payload 824 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4dbc0a04d0 con 0x7f4dbc06aac0 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbbfff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d9c03a2f0 0x7f4d9c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 20 client.35307.objecter dump_active .. 0 homeless 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbbfff700 10 monclient: get_auth_request con 0x7f4d9c03a2f0 auth_method 0 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbaffd700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 0x7f4dbc1cd020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc072140 tx=0x7f4dac00f520 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 client.35307.objecter ms_handle_connect 0x7f4dbc06aac0 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 client.35307.objecter resend_mon_ops 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4dac00fdb0 con 0x7f4dbc06aac0 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 monclient: dump: 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.057 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.058 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4db3fff700 10 client.35307.objecter ms_handle_connect 0x7f4d9c03a2f0 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.002+0000 7f4dbbfff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d9c03a2f0 0x7f4d9c03c7b0 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc06c030 tx=0x7f4db400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4dac014440 con 0x7f4dbc06aac0 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 msgr2=0x7f4dbc1cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 0x7f4dbc1cd020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc072140 tx=0x7f4dac00f520 comp rx=0 tx=0).stop 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 msgr2=0x7f4dbc1cd020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.003+0000 7f4db3fff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc06aac0 0x7f4dbc1cd020 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc072140 tx=0x7f4dac00f520 comp rx=0 tx=0).stop 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.010+0000 7f4dc2126700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.010+0000 7f4dc2126700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 0x7f4dbc02bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.010+0000 7f4dc2126700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.010+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc06a960 con 0x7f4dbc02bbc0 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.010+0000 7f4dbaffd700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 0x7f4dbc02bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient: get_auth_request con 0x7f4dbc02bbc0 auth_method 0 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 10 monclient(hunting): handle_auth_done global_id 54159 payload 824 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 1 -- 172.21.15.47:0/165893523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4dbc09b960 con 0x7f4dbc02bbc0 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 client.35307.objecter ms_handle_connect 0x7f4dbc02bbc0 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 client.35307.objecter resend_mon_ops 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4dbaffd700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 0x7f4dbc02bfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc04ed60 tx=0x7f4dac004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4dac022c00 con 0x7f4dbc02bbc0 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 monclient: dump: 2024-04-20T02:22:36.061 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.062 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f4dac022d60 con 0x7f4dbc02bbc0 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 msgr2=0x7f4dbc02bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 0x7f4dbc02bfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc04ed60 tx=0x7f4dac004750 comp rx=0 tx=0).stop 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 msgr2=0x7f4dbc02bfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.011+0000 7f4db3fff700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4dbc02bbc0 0x7f4dbc02bfd0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc04ed60 tx=0x7f4dac004750 comp rx=0 tx=0).stop 2024-04-20T02:22:36.063 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:36.064 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.065 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.066 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:36.067 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.068 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:36.070 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:36.073 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.076 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.079 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.081 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.082 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.083 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.084 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:36.085 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:36.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 10 librados: watch_flush enter 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 10 librados: watch_flush exit 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 20 client.35307.objecter shutdown clearing up homeless session... 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 10 client.35307.objecter successfully canceled tick 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d9c03a2f0 msgr2=0x7f4d9c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4d9c03a2f0 0x7f4d9c03c7b0 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc06c030 tx=0x7f4db400b040 comp rx=0 tx=0).stop 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 10 monclient: shutdown 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dbc066610 msgr2=0x7f4dbc0902d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 --2- 172.21.15.47:0/165893523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4dbc066610 0x7f4dbc0902d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4da40090b0 tx=0x7f4da4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dbbfff700 1 -- 172.21.15.47:0/165893523 reap_dead start 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dbbfff700 1 -- 172.21.15.47:0/165893523 reap_dead start 2024-04-20T02:22:36.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 shutdown_connections 2024-04-20T02:22:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 >> 172.21.15.47:0/165893523 conn(0x7f4dbc0a3000 msgr2=0x7f4dbc13bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 shutdown_connections 2024-04-20T02:22:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 -- 172.21.15.47:0/165893523 wait complete. 2024-04-20T02:22:36.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.012+0000 7f4dc2126700 1 librados: shutdown 2024-04-20T02:22:36.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:36.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:22:36.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:22:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.229+0000 7f93639bc700 10 monclient: get_monmap_and_config 2024-04-20T02:22:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.229+0000 7f93639bc700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.229+0000 7f93639bc700 10 monclient: monmap: 2024-04-20T02:22:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.229 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:36.230524+0000 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:36.230524+0000 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:36.230 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:36.231 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.229+0000 7f93639bc700 1 Processor -- start 2024-04-20T02:22:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 1 -- start start 2024-04-20T02:22:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient: init 2024-04-20T02:22:36.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient: _add_conns ranks=[3,0,2,7,6,5,9,4,10,8,1] 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f935c09fa80 0x7f935c135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): picked mon.noname-g con 0x7f935c09fa80 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f935c09fe60 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c0a0240 0x7f935c0a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): picked mon.noname-b con 0x7f935c0a0240 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0751e0 con 0x7f935c09fe60 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c09df90 con 0x7f935c0a0240 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c09dcb0 con 0x7f935c09fa80 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f93639bc700 10 monclient(hunting): authenticate will time out at 2035.631897s 2024-04-20T02:22:36.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.230+0000 7f9360f57700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f935c09fe60 0x7f935c0a1f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46282/0 (socket says 172.21.15.47:46282) 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c0a0240 0x7f935c0a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9360f57700 1 -- 172.21.15.47:0/1980959278 learned_addr learned my addr 172.21.15.47:0/1980959278 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): get_auth_request con 0x7f935c0a0240 auth_method 0 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361758700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f935c09fa80 0x7f935c135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361758700 10 monclient(hunting): get_auth_request con 0x7f935c09fa80 auth_method 0 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361758700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361758700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361758700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 1 -- 172.21.15.47:0/1980959278 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961495372 0 0) 0x7f935c0751e0 con 0x7f935c09fe60 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient(hunting): _negotiate 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient(hunting): my global_id is 35316 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 1 -- 172.21.15.47:0/1980959278 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93480035b0 con 0x7f935c09fe60 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient(hunting): handle_auth_done global_id 23852 payload 274 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 -- 172.21.15.47:0/1980959278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f935c09fa80 msgr2=0x7f935c135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f935c09fa80 0x7f935c135df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 -- 172.21.15.47:0/1980959278 >> v1:172.21.15.47:6789/0 conn(0x7f935c09fe60 legacy=0x7f935c0a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 monclient: found mon.noname-b 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 -- 172.21.15.47:0/1980959278 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c09e270 con 0x7f935c0a0240 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361758700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f935c09fa80 0x7f935c135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9361f59700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c0a0240 0x7f935c0a2620 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9354009dc0 tx=0x7f935400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d577f13e1521e160 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 1 -- 172.21.15.47:0/1980959278 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935400e070 con 0x7f935c0a0240 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: dump: 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.247 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f93639bc700 5 monclient: authenticate success, global_id 23852 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 1 -- 172.21.15.47:0/1980959278 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93540098f0 con 0x7f935c0a0240 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f93639bc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:36.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f93639bc700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 1 -- 172.21.15.47:0/1980959278 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93540128a0 con 0x7f935c0a0240 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f93639bc700 4 set_mon_vals no callback set 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.231+0000 7f9353fff700 10 monclient: dump: 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.249 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 10 monclient: shutdown 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 -- 172.21.15.47:0/1980959278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c0a0240 msgr2=0x7f935c0a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c0a0240 0x7f935c0a2620 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9354009dc0 tx=0x7f935400b330 comp rx=0 tx=0).stop 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 -- 172.21.15.47:0/1980959278 shutdown_connections 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c0a0240 0x7f935c0a2620 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 --2- 172.21.15.47:0/1980959278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f935c09fa80 0x7f935c135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 -- 172.21.15.47:0/1980959278 >> 172.21.15.47:0/1980959278 conn(0x7f935c0a3000 msgr2=0x7f935c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 -- 172.21.15.47:0/1980959278 shutdown_connections 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.232+0000 7f93639bc700 1 -- 172.21.15.47:0/1980959278 wait complete. 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 10 monclient: monmap: 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.251 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 1 librados: starting msgr at 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 1 librados: starting objecter 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 1 Processor -- start 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 1 -- start start 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 1 librados: setting wanted keys 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 1 librados: calling monclient init 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 10 monclient: init 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.233+0000 7f93639bc700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient: _add_conns ranks=[6,2,4,8,1,3,0,5,7] 2024-04-20T02:22:36.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f935c09fa80 0x7f935c06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): picked mon.noname-d con 0x7f935c09fa80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c09fe60 0x7f935c06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): picked mon.noname-b con 0x7f935c09fe60 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f935c0a0240 0x7f935c070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): picked mon.noname-c con 0x7f935c0a0240 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c09d010 con 0x7f935c09fe60 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f935c09cb50 con 0x7f935c0a0240 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f935c071570 con 0x7f935c09fa80 2024-04-20T02:22:36.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f93639bc700 10 monclient(hunting): authenticate will time out at 2035.635080s 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c09fe60 0x7f935c06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f935c0a0240 0x7f935c070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c09fe60 0x7f935c06fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60840/0 (socket says 172.21.15.47:60840) 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 1 -- 172.21.15.47:0/4192698063 learned_addr learned my addr 172.21.15.47:0/4192698063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): get_auth_request con 0x7f935c09fe60 auth_method 0 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f935c09fa80 0x7f935c06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): get_auth_request con 0x7f935c0a0240 auth_method 0 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): get_auth_request con 0x7f935c09fa80 auth_method 0 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient(hunting): handle_auth_done global_id 54168 payload 824 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f935c0a0240 msgr2=0x7f935c070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f935c0a0240 0x7f935c070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c09fe60 msgr2=0x7f935c06fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c09fe60 0x7f935c06fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 monclient: found mon.noname-d 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354009a70 con 0x7f935c09fa80 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9360f57700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f935c09fe60 0x7f935c06fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9361758700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f935c09fa80 0x7f935c06f8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f935800e7e0 tx=0x7f935800eaf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9351ffb700 10 client.?.objecter ms_handle_connect 0x7f935c09fa80 2024-04-20T02:22:36.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9351ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935800c070 con 0x7f935c09fa80 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9351ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9351ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.234+0000 7f9351ffb700 10 monclient: dump: 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.257 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 5 monclient: authenticate success, global_id 54168 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9358013880 con 0x7f935c09fa80 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 10 monclient: _renew_subs 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c0719c0 con 0x7f935c09fa80 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9358013e60 con 0x7f935c09fa80 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 10 monclient: dump: 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.259 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93537fe700 4 set_mon_vals no callback set 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 10 client.54168.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 10 monclient: _renew_subs 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c071e00 con 0x7f935c09fa80 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f93639bc700 1 librados: init done 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.235+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f93580139f0 con 0x7f935c09fa80 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f93639bc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f93639bc700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 0x7f935c12f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f935c068180 con 0x7f935c06aac0 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(297..297 src has 1..297) v4 ==== 6615+0+0 (secure 0 0 0) 0x7f935804a4c0 con 0x7f935c09fa80 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 client.54168.objecter ms_dispatch 0x7f935c001550 osd_map(297..297 src has 1..297) v4 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 3 client.54168.objecter handle_osd_map got epochs [297,297] > 0 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 3 client.54168.objecter handle_osd_map decoding full epoch 297 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 0x7f935c12f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 20 client.54168.objecter dump_active .. 0 homeless 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient: get_auth_request con 0x7f935c06aac0 auth_method 0 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9360f57700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9360f57700 10 monclient: get_auth_request con 0x7f934003a2f0 auth_method 0 2024-04-20T02:22:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 10 monclient(hunting): handle_auth_done global_id 35018 payload 824 2024-04-20T02:22:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f935c0a06c0 con 0x7f935c06aac0 2024-04-20T02:22:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9361f59700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 0x7f935c12f9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f935c0a0bc0 tx=0x7f9354004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 client.54168.objecter ms_handle_connect 0x7f935c06aac0 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 client.54168.objecter resend_mon_ops 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93540047d0 con 0x7f935c06aac0 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 monclient: dump: 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.263 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9351ffb700 10 client.54168.objecter ms_handle_connect 0x7f934003a2f0 2024-04-20T02:22:36.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.236+0000 7f9360f57700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7f935c070cb0 tx=0x7f934c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9354004930 con 0x7f935c06aac0 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 msgr2=0x7f935c12f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 0x7f935c12f9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f935c0a0bc0 tx=0x7f9354004ad0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 msgr2=0x7f935c12f9d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.237+0000 7f9351ffb700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c06aac0 0x7f935c12f9d0 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f935c0a0bc0 tx=0x7f9354004ad0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f93639bc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f93639bc700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 0x7f935c130320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f93639bc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f935c072d60 con 0x7f935c12ff10 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 0x7f935c130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient: get_auth_request con 0x7f935c12ff10 auth_method 0 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 10 monclient(hunting): handle_auth_done global_id 35027 payload 824 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 1 -- 172.21.15.47:0/4192698063 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f935c0668e0 con 0x7f935c12ff10 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9361f59700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 0x7f935c130320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f935400b370 tx=0x7f9354012000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9351ffb700 10 client.54168.objecter ms_handle_connect 0x7f935c12ff10 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9351ffb700 10 client.54168.objecter resend_mon_ops 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935402b320 con 0x7f935c12ff10 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9351ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9351ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.245+0000 7f9351ffb700 10 monclient: dump: 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.268 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f9354012640 con 0x7f935c12ff10 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 msgr2=0x7f935c130320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 0x7f935c130320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f935400b370 tx=0x7f9354012000 comp rx=0 tx=0).stop 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 msgr2=0x7f935c130320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9351ffb700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f935c12ff10 0x7f935c130320 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f935400b370 tx=0x7f9354012000 comp rx=0 tx=0).stop 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:36.270 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:22:36.271 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.272 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:36.274 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.275 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:36.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:36.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.279 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:36.280 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.281 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.282 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.284 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.285 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.286 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:36.287 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.288 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.291 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:36.292 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 10 librados: watch_flush enter 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 10 librados: watch_flush exit 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 20 client.54168.objecter shutdown clearing up homeless session... 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 10 client.54168.objecter successfully canceled tick 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 msgr2=0x7f934003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7f935c070cb0 tx=0x7f934c009040 comp rx=0 tx=0).stop 2024-04-20T02:22:36.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 10 monclient: shutdown 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f935c09fa80 msgr2=0x7f935c06f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f9361f59700 1 -- 172.21.15.47:0/4192698063 reap_dead start 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f935c09fa80 0x7f935c06f8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f935800e7e0 tx=0x7f935800eaf0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 shutdown_connections 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 --2- 172.21.15.47:0/4192698063 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f935c09fa80 0x7f935c06f8a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 >> 172.21.15.47:0/4192698063 conn(0x7f935c0a3000 msgr2=0x7f935c134cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 shutdown_connections 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 -- 172.21.15.47:0/4192698063 wait complete. 2024-04-20T02:22:36.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.246+0000 7f93639bc700 1 librados: shutdown 2024-04-20T02:22:36.295 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:36.296 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:22:36.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:22:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.441+0000 7fb706e58700 10 monclient: get_monmap_and_config 2024-04-20T02:22:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.441+0000 7fb706e58700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.441+0000 7fb706e58700 10 monclient: monmap: 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:36.442659+0000 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:36.442659+0000 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:36.443 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 1 Processor -- start 2024-04-20T02:22:36.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 1 -- start start 2024-04-20T02:22:36.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 10 monclient: init 2024-04-20T02:22:36.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 10 monclient: _add_conns ranks=[9,6,8,5,10,7,3,1,0,2,4] 2024-04-20T02:22:36.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 0x7fb7000a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 10 monclient(hunting): picked mon.noname-a con 0x7fb700137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb700138170 0x7fb7000a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 10 monclient(hunting): picked mon.noname-d con 0x7fb700138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.442+0000 7fb706e58700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb7001385b0 0x7fb7000a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient(hunting): picked mon.noname-e con 0x7fb7001385b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70009fa40 con 0x7fb700137d90 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb70009dfc0 con 0x7fb700138170 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb7000a7bc0 con 0x7fb7001385b0 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient(hunting): authenticate will time out at 2035.844055s 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 0x7fb7000a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb700138170 0x7fb7000a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb7001385b0 0x7fb7000a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 0x7fb7000a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60894/0 (socket says 172.21.15.47:60894) 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 1 -- 172.21.15.47:0/1722952178 learned_addr learned my addr 172.21.15.47:0/1722952178 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): get_auth_request con 0x7fb700138170 auth_method 0 2024-04-20T02:22:36.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): get_auth_request con 0x7fb7001385b0 auth_method 0 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): get_auth_request con 0x7fb700137d90 auth_method 0 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient(hunting): handle_auth_done global_id 54177 payload 274 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 -- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb7001385b0 msgr2=0x7fb7000a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 --2- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb7001385b0 0x7fb7000a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 -- 172.21.15.47:0/1722952178 >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 msgr2=0x7fb7000a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 --2- 172.21.15.47:0/1722952178 >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 0x7fb7000a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 monclient: found mon.noname-d 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 -- 172.21.15.47:0/1722952178 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000751e0 con 0x7fb700138170 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb7053f5700 1 --2- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb7001385b0 0x7fb7000a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb704bf4700 1 --2- 172.21.15.47:0/1722952178 >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 0x7fb7000a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6fffff700 1 --2- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb700138170 0x7fb7000a5c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7645c79f6554142b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 1 -- 172.21.15.47:0/1722952178 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6ec011aa0 con 0x7fb700138170 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: dump: 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.465 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 5 monclient: authenticate success, global_id 54177 2024-04-20T02:22:36.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 1 -- 172.21.15.47:0/1722952178 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec011c00 con 0x7fb700138170 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb706e58700 4 set_mon_vals no callback set 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 1 -- 172.21.15.47:0/1722952178 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6ec011d70 con 0x7fb700138170 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.443+0000 7fb6ff7fe700 10 monclient: dump: 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.467 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 10 monclient: shutdown 2024-04-20T02:22:36.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 -- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb700138170 msgr2=0x7fb7000a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 --2- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb700138170 0x7fb7000a5c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 -- 172.21.15.47:0/1722952178 shutdown_connections 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 --2- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb7001385b0 0x7fb7000a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 --2- 172.21.15.47:0/1722952178 >> v2:172.21.15.47:3300/0 conn(0x7fb700137d90 0x7fb7000a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 --2- 172.21.15.47:0/1722952178 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb700138170 0x7fb7000a5c90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 -- 172.21.15.47:0/1722952178 >> 172.21.15.47:0/1722952178 conn(0x7fb7000aa8b0 msgr2=0x7fb7000a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 -- 172.21.15.47:0/1722952178 shutdown_connections 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.444+0000 7fb706e58700 1 -- 172.21.15.47:0/1722952178 wait complete. 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 10 monclient: monmap: 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:36.469 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 1 librados: starting msgr at 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 1 librados: starting objecter 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 1 Processor -- start 2024-04-20T02:22:36.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 1 -- start start 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 1 librados: setting wanted keys 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 1 librados: calling monclient init 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.445+0000 7fb706e58700 10 monclient: init 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient: _add_conns ranks=[1,3,5,4,0,2,7,6,8] 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700137d90 0x7fb7001cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): picked mon.noname-f con 0x7fb700137d90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb700138170 0x7fb700136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): picked mon.noname-g con 0x7fb700138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb7001385b0 0x7fb7001373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): picked mon.noname-h con 0x7fb7001385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001242b0 con 0x7fb700137d90 2024-04-20T02:22:36.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb700137930 con 0x7fb700138170 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7001ccfe0 con 0x7fb7001385b0 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb706e58700 10 monclient(hunting): authenticate will time out at 2035.847400s 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700137d90 0x7fb7001cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb7001385b0 0x7fb7001373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb700138170 0x7fb700136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700137d90 0x7fb7001cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:51176/0 (socket says 172.21.15.47:51176) 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): get_auth_request con 0x7fb700138170 auth_method 0 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 1 -- 172.21.15.47:0/2781178131 learned_addr learned my addr 172.21.15.47:0/2781178131 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): get_auth_request con 0x7fb7001385b0 auth_method 0 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): get_auth_request con 0x7fb700137d90 auth_method 0 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb6fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb7053f5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.446+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 10 monclient(hunting): handle_auth_done global_id 44393 payload 824 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb700138170 msgr2=0x7fb700136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb700138170 0x7fb700136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700137d90 msgr2=0x7fb7001cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700137d90 0x7fb7001cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 monclient: found mon.noname-h 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f4009550 con 0x7fb7001385b0 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb704bf4700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700137d90 0x7fb7001cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fffff700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb700138170 0x7fb700136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb7053f5700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb7001385b0 0x7fb7001373f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800ef90 tx=0x7fb6f800c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 client.?.objecter ms_handle_connect 0x7fb7001385b0 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f8009e70 con 0x7fb7001385b0 2024-04-20T02:22:36.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: dump: 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.476 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 5 monclient: authenticate success, global_id 44393 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f800f040 con 0x7fb7001385b0 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f80148f0 con 0x7fb7001385b0 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6feffd700 4 set_mon_vals no callback set 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb6fd7fa700 10 monclient: dump: 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.478 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 10 monclient: _renew_subs 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f8009a70 con 0x7fb7001385b0 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 10 client.44393.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 10 monclient: _renew_subs 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec0094a0 con 0x7fb7001385b0 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.447+0000 7fb706e58700 1 librados: init done 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb706e58700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb706e58700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 0x7fb700120910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb700068180 con 0x7fb70006aac0 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb6f8014480 con 0x7fb7001385b0 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 0x7fb700120910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient: get_auth_request con 0x7fb70006aac0 auth_method 0 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.448+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb6e403a2f0 0x7fb6e403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb704bf4700 10 monclient(hunting): handle_auth_done global_id 54154 payload 824 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb704bf4700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb70009f760 con 0x7fb70006aac0 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(298..298 src has 1..298) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fb6f804aeb0 con 0x7fb7001385b0 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 client.44393.objecter ms_dispatch 0x7fb700001550 osd_map(298..298 src has 1..298) v4 2024-04-20T02:22:36.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 3 client.44393.objecter handle_osd_map got epochs [298,298] > 0 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 3 client.44393.objecter handle_osd_map decoding full epoch 298 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 20 client.44393.objecter dump_active .. 0 homeless 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fffff700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb6e403a2f0 0x7fb6e403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fffff700 10 monclient: get_auth_request con 0x7fb6e403a2f0 auth_method 0 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 client.44393.objecter ms_handle_connect 0x7fb70006aac0 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 client.44393.objecter resend_mon_ops 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb704bf4700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 0x7fb700120910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400f470 tx=0x7fb6f400f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f4009ea0 con 0x7fb70006aac0 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 monclient: dump: 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 client.44393.objecter ms_handle_connect 0x7fb6e403a2f0 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fffff700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb6e403a2f0 0x7fb6e403c7b0 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb6f4014440 con 0x7fb70006aac0 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 msgr2=0x7fb700120910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 0x7fb700120910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400f470 tx=0x7fb6f400f830 comp rx=0 tx=0).stop 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 msgr2=0x7fb700120910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.449+0000 7fb6fd7fa700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70006aac0 0x7fb700120910 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400f470 tx=0x7fb6f400f830 comp rx=0 tx=0).stop 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.457+0000 7fb706e58700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.457+0000 7fb706e58700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 0x7fb700121260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.457+0000 7fb706e58700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.457+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70006a400 con 0x7fb700120e50 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 0x7fb700121260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient: get_auth_request con 0x7fb700120e50 auth_method 0 2024-04-20T02:22:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 10 monclient(hunting): handle_auth_done global_id 54163 payload 824 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 1 -- 172.21.15.47:0/2781178131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb7001245f0 con 0x7fb700120e50 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb704bf4700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 0x7fb700121260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400f470 tx=0x7fb6f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb6fd7fa700 10 client.44393.objecter ms_handle_connect 0x7fb700120e50 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb6fd7fa700 10 client.44393.objecter resend_mon_ops 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f402b070 con 0x7fb700120e50 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb6fd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb6fd7fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.458+0000 7fb6fd7fa700 10 monclient: dump: 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.486 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fb6f4023040 con 0x7fb700120e50 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 msgr2=0x7fb700121260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 0x7fb700121260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400f470 tx=0x7fb6f4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 msgr2=0x7fb700121260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb6fd7fa700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb700120e50 0x7fb700121260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.487 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:36.488 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.489 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.490 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:36.491 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.492 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:36.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:36.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.499 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:36.500 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.502 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:36.504 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.505 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.507 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.510 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:36.511 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 10 librados: watch_flush enter 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 10 librados: watch_flush exit 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 20 client.44393.objecter shutdown clearing up homeless session... 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 10 client.44393.objecter successfully canceled tick 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb6e403a2f0 msgr2=0x7fb6e403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb6e403a2f0 0x7fb6e403c7b0 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec008030 comp rx=0 tx=0).stop 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 10 monclient: shutdown 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb7001385b0 msgr2=0x7fb7001373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 --2- 172.21.15.47:0/2781178131 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb7001385b0 0x7fb7001373f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800ef90 tx=0x7fb6f800c3a0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb7053f5700 1 -- 172.21.15.47:0/2781178131 reap_dead start 2024-04-20T02:22:36.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb7053f5700 1 -- 172.21.15.47:0/2781178131 reap_dead start 2024-04-20T02:22:36.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 shutdown_connections 2024-04-20T02:22:36.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 >> 172.21.15.47:0/2781178131 conn(0x7fb7000aa8b0 msgr2=0x7fb7000a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 shutdown_connections 2024-04-20T02:22:36.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 -- 172.21.15.47:0/2781178131 wait complete. 2024-04-20T02:22:36.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.459+0000 7fb706e58700 1 librados: shutdown 2024-04-20T02:22:36.515 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:36.515 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:22:36.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:22:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.653+0000 7f11a869f700 10 monclient: get_monmap_and_config 2024-04-20T02:22:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.653+0000 7f11a869f700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.653+0000 7f11a869f700 10 monclient: monmap: 2024-04-20T02:22:36.654 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:36.655055+0000 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:36.655055+0000 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.655 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.654+0000 7f11a869f700 1 Processor -- start 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.654+0000 7f11a869f700 1 -- start start 2024-04-20T02:22:36.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.654+0000 7f11a869f700 10 monclient: init 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient: _add_conns ranks=[0,6,2,7,4,8,9,1,3,10,5] 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11a0071610 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 0x7f11a0139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): picked mon.noname-d con 0x7f11a0137930 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11a013a260 0x7f11a013c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): picked mon.noname-b con 0x7f11a013a260 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a00751e0 con 0x7f11a0071610 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00a0880 con 0x7f11a013a260 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009e8f0 con 0x7f11a0137930 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a869f700 10 monclient(hunting): authenticate will time out at 2036.056562s 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 0x7f11a0139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 0x7f11a0139d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48184/0 (socket says 172.21.15.47:48184) 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a643b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f11a0071610 0x7f11a00719f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46292/0 (socket says 172.21.15.47:46292) 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 -- 172.21.15.47:0/515537936 learned_addr learned my addr 172.21.15.47:0/515537936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): get_auth_request con 0x7f11a0137930 auth_method 0 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11a013a260 0x7f11a013c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): get_auth_request con 0x7f11a013a260 auth_method 0 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_done global_id 54186 payload 274 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 -- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11a013a260 msgr2=0x7f11a013c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11a013a260 0x7f11a013c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 -- 172.21.15.47:0/515537936 >> v1:172.21.15.47:6789/0 conn(0x7f11a0071610 legacy=0x7f11a00719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 monclient: found mon.noname-d 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5c3a700 1 -- 172.21.15.47:0/515537936 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a009ed00 con 0x7f11a0137930 2024-04-20T02:22:36.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a6c3c700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11a013a260 0x7f11a013c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.655+0000 7f11a5439700 1 -- 172.21.15.47:0/515537936 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976081963 0 0) 0x7f11a00751e0 con 0x7f11a0071610 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5c3a700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 0x7f11a0139d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1190009920 tx=0x7f1190009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=18ba3fb7f471ee34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 1 -- 172.21.15.47:0/515537936 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1190011aa0 con 0x7f11a0137930 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: dump: 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.669 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 5 monclient: authenticate success, global_id 54186 2024-04-20T02:22:36.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 1 -- 172.21.15.47:0/515537936 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190011c00 con 0x7f11a0137930 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 4 set_mon_vals no callback set 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 1 -- 172.21.15.47:0/515537936 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1190011d70 con 0x7f11a0137930 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a5439700 10 monclient: dump: 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.671 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 10 monclient: shutdown 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 1 -- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 msgr2=0x7f11a0139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 0x7f11a0139d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1190009920 tx=0x7f1190009c30 comp rx=0 tx=0).stop 2024-04-20T02:22:36.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 1 -- 172.21.15.47:0/515537936 shutdown_connections 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11a013a260 0x7f11a013c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 1 --2- 172.21.15.47:0/515537936 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11a0137930 0x7f11a0139d20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.656+0000 7f11a869f700 1 -- 172.21.15.47:0/515537936 >> 172.21.15.47:0/515537936 conn(0x7f11a00a7570 msgr2=0x7f11a00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 1 -- 172.21.15.47:0/515537936 shutdown_connections 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 1 -- 172.21.15.47:0/515537936 wait complete. 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 10 monclient: monmap: 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.673 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 1 librados: starting msgr at 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 1 librados: starting objecter 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.657+0000 7f11a869f700 1 Processor -- start 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 -- start start 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 librados: setting wanted keys 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 librados: calling monclient init 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient: init 2024-04-20T02:22:36.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient: _add_conns ranks=[3,8,7,6,4,0,1,2,5] 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a0071610 0x7f11a009a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): picked mon.noname-g con 0x7f11a0071610 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 0x7f11a009ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): picked mon.noname-e con 0x7f11a0137930 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11a013a260 0x7f11a009b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): picked mon.noname-i con 0x7f11a013a260 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009f260 con 0x7f11a0071610 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00723f0 con 0x7f11a013a260 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009d450 con 0x7f11a0137930 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a869f700 10 monclient(hunting): authenticate will time out at 2036.059581s 2024-04-20T02:22:36.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 0x7f11a009ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 0x7f11a009ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58252/0 (socket says 172.21.15.47:58252) 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 1 -- 172.21.15.47:0/933284634 learned_addr learned my addr 172.21.15.47:0/933284634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): get_auth_request con 0x7f11a0137930 auth_method 0 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a0071610 0x7f11a009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11a013a260 0x7f11a009b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): get_auth_request con 0x7f11a0071610 auth_method 0 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): get_auth_request con 0x7f11a013a260 auth_method 0 2024-04-20T02:22:36.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.658+0000 7f11a6c3c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 10 monclient(hunting): handle_auth_done global_id 35045 payload 824 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11a013a260 msgr2=0x7f11a009b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11a013a260 0x7f11a009b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a0071610 msgr2=0x7f11a009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a0071610 0x7f11a009a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 monclient: found mon.noname-e 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1198009a70 con 0x7f11a0137930 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a5c3a700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 0x7f11a009ae70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1190009840 tx=0x7f1190004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 client.?.objecter ms_handle_connect 0x7f11a0137930 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1190004480 con 0x7f11a0137930 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: dump: 2024-04-20T02:22:36.678 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190003b10 con 0x7f11a0137930 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 5 monclient: authenticate success, global_id 35045 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 10 monclient: _renew_subs 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1190009550 con 0x7f11a0137930 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1190003c80 con 0x7f11a0137930 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 10 client.35045.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f1196ffd700 10 monclient: dump: 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.680 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 10 monclient: _renew_subs 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a009d730 con 0x7f11a0137930 2024-04-20T02:22:36.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a869f700 1 librados: init done 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.659+0000 7f11a4c38700 4 set_mon_vals no callback set 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a869f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a869f700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 0x7f11a0066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0068310 con 0x7f11a006aac0 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 0x7f11a0066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient: get_auth_request con 0x7f11a006aac0 auth_method 0 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 10 monclient(hunting): handle_auth_done global_id 21963 payload 824 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.660+0000 7f11a643b700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11a009d5c0 con 0x7f11a006aac0 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1190023070 con 0x7f11a0137930 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f11a643b700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 0x7f11a0066ea0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f119c000f60 tx=0x7f119c0018f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f118403a2f0 0x7f118403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 client.35045.objecter ms_handle_connect 0x7f11a006aac0 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 client.35045.objecter resend_mon_ops 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(298..298 src has 1..298) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f119004ddc0 con 0x7f11a0137930 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 client.35045.objecter ms_dispatch 0x7f11a0001550 osd_map(298..298 src has 1..298) v4 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 3 client.35045.objecter handle_osd_map got epochs [298,298] > 0 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 3 client.35045.objecter handle_osd_map decoding full epoch 298 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f11a6c3c700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f118403a2f0 0x7f118403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 20 client.35045.objecter dump_active .. 0 homeless 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f119c017070 con 0x7f11a006aac0 2024-04-20T02:22:36.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 monclient: dump: 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.684 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f11a6c3c700 10 monclient: get_auth_request con 0x7f118403a2f0 auth_method 0 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f119c012560 con 0x7f11a006aac0 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 msgr2=0x7f11a0066ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 0x7f11a0066ea0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f119c000f60 tx=0x7f119c0018f0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 msgr2=0x7f11a0066ea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.661+0000 7f1196ffd700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a006aac0 0x7f11a0066ea0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.662+0000 7f1196ffd700 10 client.35045.objecter ms_handle_connect 0x7f118403a2f0 2024-04-20T02:22:36.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.662+0000 7f11a6c3c700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f118403a2f0 0x7f118403c7b0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7f11a0139990 tx=0x7f119800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a869f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a869f700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 0x7f11a00677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a869f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a009d8f0 con 0x7f11a00673e0 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a643b700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 0x7f11a00677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a643b700 10 monclient: get_auth_request con 0x7f11a00673e0 auth_method 0 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a643b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a643b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.669+0000 7f11a643b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f11a643b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f11a643b700 10 monclient(hunting): handle_auth_done global_id 21972 payload 824 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f11a643b700 1 -- 172.21.15.47:0/933284634 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f11a011d810 con 0x7f11a00673e0 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f11a643b700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 0x7f11a00677f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f119c000fb0 tx=0x7f119c000ed0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f1196ffd700 10 client.35045.objecter ms_handle_connect 0x7f11a00673e0 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f1196ffd700 10 client.35045.objecter resend_mon_ops 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f119c023070 con 0x7f11a00673e0 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f1196ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f1196ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.670+0000 7f1196ffd700 10 monclient: dump: 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.688 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f119c012e50 con 0x7f11a00673e0 2024-04-20T02:22:36.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 msgr2=0x7f11a00677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 0x7f11a00677f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f119c000fb0 tx=0x7f119c000ed0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 msgr2=0x7f11a00677f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f1196ffd700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11a00673e0 0x7f11a00677f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:36.690 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:36.691 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.692 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:36.693 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:36.694 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:36.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.698 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.701 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:36.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:36.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.709 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:36.711 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 10 librados: watch_flush enter 2024-04-20T02:22:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 10 librados: watch_flush exit 2024-04-20T02:22:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 20 client.35045.objecter shutdown clearing up homeless session... 2024-04-20T02:22:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 10 client.35045.objecter successfully canceled tick 2024-04-20T02:22:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f118403a2f0 msgr2=0x7f118403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f118403a2f0 0x7f118403c7b0 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7f11a0139990 tx=0x7f119800b040 comp rx=0 tx=0).stop 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 10 monclient: shutdown 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 msgr2=0x7f11a009ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 0x7f11a009ae70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1190009840 tx=0x7f1190004750 comp rx=0 tx=0).stop 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a6c3c700 1 -- 172.21.15.47:0/933284634 reap_dead start 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 shutdown_connections 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 --2- 172.21.15.47:0/933284634 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f11a0137930 0x7f11a009ae70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 >> 172.21.15.47:0/933284634 conn(0x7f11a00a7570 msgr2=0x7f11a0134bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 shutdown_connections 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 -- 172.21.15.47:0/933284634 wait complete. 2024-04-20T02:22:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.671+0000 7f11a869f700 1 librados: shutdown 2024-04-20T02:22:36.715 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:36.715 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:22:36.715 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:22:36.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.852+0000 7f68ef9a8700 10 monclient: get_monmap_and_config 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.852+0000 7f68ef9a8700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.852+0000 7f68ef9a8700 10 monclient: monmap: 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:36.853267+0000 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:36.853267+0000 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.853 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.854 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.852+0000 7f68ef9a8700 1 Processor -- start 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 1 -- start start 2024-04-20T02:22:36.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient: init 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient: _add_conns ranks=[1,6,0,5,10,3,4,7,9,8,2] 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f68e809ce50 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f68e809c3b0 0x7f68e80a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient(hunting): picked mon.noname-d con 0x7f68e809c3b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f68e813d6c0 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80751e0 con 0x7f68e813d6c0 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e809c7d0 con 0x7f68e809ce50 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f68e809bc40 con 0x7f68e809c3b0 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ef9a8700 10 monclient(hunting): authenticate will time out at 2036.254649s 2024-04-20T02:22:36.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68edf45700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f68e813d6c0 0x7f68e813fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46308/0 (socket says 172.21.15.47:46308) 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68edf45700 1 -- 172.21.15.47:0/3676134112 learned_addr learned my addr 172.21.15.47:0/3676134112 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 1 --2- 172.21.15.47:0/3676134112 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f68e809c3b0 0x7f68e80a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): get_auth_request con 0x7f68e809c3b0 auth_method 0 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.853+0000 7f68ecf43700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 1 -- 172.21.15.47:0/3676134112 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552874052 0 0) 0x7f68e80751e0 con 0x7f68e813d6c0 2024-04-20T02:22:36.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient(hunting): _negotiate 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient(hunting): my global_id is 35343 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 1 -- 172.21.15.47:0/3676134112 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d40035b0 con 0x7f68e813d6c0 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 10 monclient(hunting): handle_auth_done global_id 54195 payload 274 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 1 -- 172.21.15.47:0/3676134112 >> v1:172.21.15.179:6789/0 conn(0x7f68e809ce50 legacy=0x7f68e80a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 1 -- 172.21.15.47:0/3676134112 >> v1:172.21.15.47:6789/0 conn(0x7f68e813d6c0 legacy=0x7f68e813fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 1 monclient: found mon.noname-d 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 1 -- 172.21.15.47:0/3676134112 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e809ab10 con 0x7f68e809c3b0 2024-04-20T02:22:36.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ecf43700 1 --2- 172.21.15.47:0/3676134112 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f68e809c3b0 0x7f68e80a2620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d8009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2fe0540aa1da5ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 1 -- 172.21.15.47:0/3676134112 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68d8011aa0 con 0x7f68e809c3b0 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: dump: 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.870 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 1 -- 172.21.15.47:0/3676134112 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d8011c00 con 0x7f68e809c3b0 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ef9a8700 5 monclient: authenticate success, global_id 54195 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ef9a8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ef9a8700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68ef9a8700 4 set_mon_vals no callback set 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 1 -- 172.21.15.47:0/3676134112 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68d8011d70 con 0x7f68e809c3b0 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.854+0000 7f68dffff700 10 monclient: dump: 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.872 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 10 monclient: shutdown 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3676134112 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f68e809c3b0 msgr2=0x7f68e80a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 --2- 172.21.15.47:0/3676134112 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f68e809c3b0 0x7f68e80a2620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d8009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3676134112 shutdown_connections 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 --2- 172.21.15.47:0/3676134112 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f68e809c3b0 0x7f68e80a2620 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3676134112 >> 172.21.15.47:0/3676134112 conn(0x7f68e80a3000 msgr2=0x7f68e80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3676134112 shutdown_connections 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.855+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3676134112 wait complete. 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 10 monclient: build_initial_monmap 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 10 monclient: monmap: 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:36.874 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 1 librados: starting msgr at 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 1 librados: starting objecter 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 1 Processor -- start 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 1 -- start start 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 1 librados: setting wanted keys 2024-04-20T02:22:36.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 1 librados: calling monclient init 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.856+0000 7f68ef9a8700 10 monclient: init 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient: _add_conns ranks=[8,4,0,6,5,3,2,1,7] 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68e809c3b0 0x7f68e81ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): picked mon.noname-e con 0x7f68e809c3b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f68e809ce50 0x7f68e81cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): picked mon.noname-c con 0x7f68e809ce50 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e813d6c0 0x7f68e806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): picked mon.noname-a con 0x7f68e813d6c0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e809ad50 con 0x7f68e813d6c0 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68e809a900 con 0x7f68e809ce50 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8071ff0 con 0x7f68e809c3b0 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ef9a8700 10 monclient(hunting): authenticate will time out at 2036.258492s 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e813d6c0 0x7f68e806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68e809c3b0 0x7f68e81ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e813d6c0 0x7f68e806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60908/0 (socket says 172.21.15.47:60908) 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 1 -- 172.21.15.47:0/3379836776 learned_addr learned my addr 172.21.15.47:0/3379836776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): get_auth_request con 0x7f68e813d6c0 auth_method 0 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68e809c3b0 0x7f68e81ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58268/0 (socket says 172.21.15.47:58268) 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): get_auth_request con 0x7f68e809c3b0 auth_method 0 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68edf45700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ecf43700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f68e809ce50 0x7f68e81cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ecf43700 10 monclient(hunting): get_auth_request con 0x7f68e809ce50 auth_method 0 2024-04-20T02:22:36.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ecf43700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ecf43700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.857+0000 7f68ecf43700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 10 monclient(hunting): handle_auth_done global_id 35352 payload 824 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68e809c3b0 msgr2=0x7f68e81ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68e809c3b0 0x7f68e81ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f68e809ce50 msgr2=0x7f68e81cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f68e809ce50 0x7f68e81cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 10 monclient: _finish_hunting 0 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 monclient: found mon.noname-a 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:36.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d80094a0 con 0x7f68e813d6c0 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ecf43700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f68e809ce50 0x7f68e81cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ed744700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68e809c3b0 0x7f68e81ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68edf45700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e813d6c0 0x7f68e806ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f68e000caa0 tx=0x7f68e000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 client.?.objecter ms_handle_connect 0x7f68e813d6c0 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e000ac50 con 0x7f68e813d6c0 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: dump: 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.880 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.881 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e000adb0 con 0x7f68e813d6c0 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 5 monclient: authenticate success, global_id 35352 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 10 monclient: _renew_subs 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e80722d0 con 0x7f68e813d6c0 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e0011810 con 0x7f68e813d6c0 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:36.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ddffb700 10 monclient: dump: 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.883 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 10 client.35352.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 10 monclient: _renew_subs 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68df7fe700 4 set_mon_vals no callback set 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e81cd680 con 0x7f68e813d6c0 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.858+0000 7f68ef9a8700 1 librados: init done 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f68e00113a0 con 0x7f68e813d6c0 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ef9a8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ef9a8700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 0x7f68e80a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ef9a8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8071270 con 0x7f68e806aac0 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68cc03a2f0 0x7f68cc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(298..298 src has 1..298) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f68e004a0b0 con 0x7f68e813d6c0 2024-04-20T02:22:36.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 10 client.35352.objecter ms_dispatch 0x7f68e8001550 osd_map(298..298 src has 1..298) v4 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 3 client.35352.objecter handle_osd_map got epochs [298,298] > 0 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 3 client.35352.objecter handle_osd_map decoding full epoch 298 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ddffb700 20 client.35352.objecter dump_active .. 0 homeless 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ed744700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 0x7f68e80a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ed744700 10 monclient: get_auth_request con 0x7f68e806aac0 auth_method 0 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ed744700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ed744700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.859+0000 7f68ed744700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ecf43700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68cc03a2f0 0x7f68cc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ecf43700 10 monclient: get_auth_request con 0x7f68cc03a2f0 auth_method 0 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ed744700 10 monclient(hunting): handle_auth_done global_id 44402 payload 824 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ed744700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68e806bd80 con 0x7f68e806aac0 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ed744700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 0x7f68e80a4400 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68e4007fd0 tx=0x7f68e400fe00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 10 client.35352.objecter ms_handle_connect 0x7f68e806aac0 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 10 client.35352.objecter resend_mon_ops 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 10 client.35352.objecter ms_handle_connect 0x7f68cc03a2f0 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ecf43700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68cc03a2f0 0x7f68cc03c7b0 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d8008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e4001820 con 0x7f68e806aac0 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.860+0000 7f68ddffb700 10 monclient: dump: 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.886 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f68e40052e0 con 0x7f68e806aac0 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:36.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 msgr2=0x7f68e80a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 0x7f68e80a4400 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68e4007fd0 tx=0x7f68e400fe00 comp rx=0 tx=0).stop 2024-04-20T02:22:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 msgr2=0x7f68e80a4400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.861+0000 7f68ddffb700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e806aac0 0x7f68e80a4400 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68e4007fd0 tx=0x7f68e400fe00 comp rx=0 tx=0).stop 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.868+0000 7f68ef9a8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.868+0000 7f68ef9a8700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 0x7f68e80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.868+0000 7f68ef9a8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.868+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8068180 con 0x7f68e80a4940 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 0x7f68e80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient: get_auth_request con 0x7f68e80a4940 auth_method 0 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:36.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 10 monclient(hunting): handle_auth_done global_id 44411 payload 824 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 1 -- 172.21.15.47:0/3379836776 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f68e809b960 con 0x7f68e80a4940 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ddffb700 10 client.35352.objecter ms_handle_connect 0x7f68e80a4940 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ddffb700 10 client.35352.objecter resend_mon_ops 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ed744700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 0x7f68e80a4d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68e4017000 tx=0x7f68e4001120 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e4023070 con 0x7f68e80a4940 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ddffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:36.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.869+0000 7f68ddffb700 10 monclient: dump: 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:36.892 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f68e4003c00 con 0x7f68e80a4940 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 msgr2=0x7f68e80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 0x7f68e80a4d50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68e4017000 tx=0x7f68e4001120 comp rx=0 tx=0).stop 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 msgr2=0x7f68e80a4d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ddffb700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68e80a4940 0x7f68e80a4d50 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68e4017000 tx=0x7f68e4001120 comp rx=0 tx=0).stop 2024-04-20T02:22:36.893 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:36.894 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.895 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.896 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:36.897 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:36.898 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:36.899 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:36.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.903 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:36.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.909 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.910 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.911 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:36.913 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:36.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:36.915 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 10 librados: watch_flush enter 2024-04-20T02:22:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 10 librados: watch_flush exit 2024-04-20T02:22:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 20 client.35352.objecter shutdown clearing up homeless session... 2024-04-20T02:22:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 10 client.35352.objecter successfully canceled tick 2024-04-20T02:22:36.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68cc03a2f0 msgr2=0x7f68cc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68cc03a2f0 0x7f68cc03c7b0 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009870 tx=0x7f68d8008030 comp rx=0 tx=0).stop 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 10 monclient: shutdown 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e813d6c0 msgr2=0x7f68e806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 --2- 172.21.15.47:0/3379836776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68e813d6c0 0x7f68e806ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f68e000caa0 tx=0x7f68e000cdb0 comp rx=0 tx=0).stop 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68edf45700 1 -- 172.21.15.47:0/3379836776 reap_dead start 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68edf45700 1 -- 172.21.15.47:0/3379836776 reap_dead start 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 shutdown_connections 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 >> 172.21.15.47:0/3379836776 conn(0x7f68e80a3000 msgr2=0x7f68e8137db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 shutdown_connections 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 -- 172.21.15.47:0/3379836776 wait complete. 2024-04-20T02:22:36.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:36.870+0000 7f68ef9a8700 1 librados: shutdown 2024-04-20T02:22:36.919 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:36.919 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:22:36.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:22:37.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.058+0000 7fcb894ed700 10 monclient: get_monmap_and_config 2024-04-20T02:22:37.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.058+0000 7fcb894ed700 10 monclient: build_initial_monmap 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.058+0000 7fcb894ed700 10 monclient: monmap: 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:37.059867+0000 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:37.059867+0000 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:37.059 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:37.060 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:37.060 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 1 Processor -- start 2024-04-20T02:22:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 1 -- start start 2024-04-20T02:22:37.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 10 monclient: init 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 10 monclient: _add_conns ranks=[9,8,1,4,7,0,5,10,3,6,2] 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 0x7fcb840a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 10 monclient(hunting): picked mon.noname-a con 0x7fcb84137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.059+0000 7fcb894ed700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb84138170 0x7fcb840a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 10 monclient(hunting): picked mon.noname-e con 0x7fcb84138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcb841385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb840751e0 con 0x7fcb841385b0 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8409fa40 con 0x7fcb84137d90 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8409dfc0 con 0x7fcb84138170 2024-04-20T02:22:37.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb894ed700 10 monclient(hunting): authenticate will time out at 2036.461096s 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 0x7fcb840a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 0x7fcb840a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60914/0 (socket says 172.21.15.47:60914) 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 1 -- 172.21.15.47:0/2261083864 learned_addr learned my addr 172.21.15.47:0/2261083864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): get_auth_request con 0x7fcb84137d90 auth_method 0 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb837fe700 1 --1- 172.21.15.47:0/2261083864 >> v1:172.21.15.179:6789/0 conn(0x7fcb841385b0 0x7fcb840a61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:47706/0 (socket says 172.21.15.47:47706) 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 --2- 172.21.15.47:0/2261083864 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb84138170 0x7fcb840a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): get_auth_request con 0x7fcb84138170 auth_method 0 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb827fc700 1 -- 172.21.15.47:0/2261083864 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558059688 0 0) 0x7fcb840751e0 con 0x7fcb841385b0 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb827fc700 10 monclient(hunting): _negotiate 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb827fc700 10 monclient(hunting): my global_id is 54181 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb827fc700 1 -- 172.21.15.47:0/2261083864 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb700035b0 con 0x7fcb841385b0 2024-04-20T02:22:37.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_done global_id 35063 payload 274 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2261083864 >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 msgr2=0x7fcb840a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 --2- 172.21.15.47:0/2261083864 >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 0x7fcb840a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2261083864 >> v1:172.21.15.179:6789/0 conn(0x7fcb841385b0 legacy=0x7fcb840a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 monclient: found mon.noname-e 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2261083864 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840a7bc0 con 0x7fcb84138170 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb82ffd700 1 --2- 172.21.15.47:0/2261083864 >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 0x7fcb840a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.060+0000 7fcb7a7fc700 1 --2- 172.21.15.47:0/2261083864 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb84138170 0x7fcb840a5c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009840 tx=0x7fcb6c009b50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ea09b977a0bb853e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 1 -- 172.21.15.47:0/2261083864 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb6c011aa0 con 0x7fcb84138170 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: dump: 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.077 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:37.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 1 -- 172.21.15.47:0/2261083864 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c011c00 con 0x7fcb84138170 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 5 monclient: authenticate success, global_id 35063 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 1 -- 172.21.15.47:0/2261083864 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb6c011d70 con 0x7fcb84138170 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 4 set_mon_vals no callback set 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb827fc700 10 monclient: dump: 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 10 monclient: shutdown 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 -- 172.21.15.47:0/2261083864 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb84138170 msgr2=0x7fcb840a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2261083864 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb84138170 0x7fcb840a5c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009840 tx=0x7fcb6c009b50 comp rx=0 tx=0).stop 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 -- 172.21.15.47:0/2261083864 shutdown_connections 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2261083864 >> v2:172.21.15.47:3300/0 conn(0x7fcb84137d90 0x7fcb840a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2261083864 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb84138170 0x7fcb840a5c90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 -- 172.21.15.47:0/2261083864 >> 172.21.15.47:0/2261083864 conn(0x7fcb840aa8b0 msgr2=0x7fcb840a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 -- 172.21.15.47:0/2261083864 shutdown_connections 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.061+0000 7fcb894ed700 1 -- 172.21.15.47:0/2261083864 wait complete. 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.062+0000 7fcb894ed700 10 monclient: build_initial_monmap 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.062+0000 7fcb894ed700 10 monclient: monmap: 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.062+0000 7fcb894ed700 1 librados: starting msgr at 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.062+0000 7fcb894ed700 1 librados: starting objecter 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 Processor -- start 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 -- start start 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 librados: setting wanted keys 2024-04-20T02:22:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 librados: calling monclient init 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient: init 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient: _add_conns ranks=[3,6,2,0,5,7,1,8,4] 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb84137d90 0x7fcb841cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): picked mon.noname-g con 0x7fcb84137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb84138170 0x7fcb84134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): picked mon.noname-d con 0x7fcb84138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb841385b0 0x7fcb84135300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): picked mon.noname-b con 0x7fcb841385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84122270 con 0x7fcb841385b0 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840a3a10 con 0x7fcb84137d90 2024-04-20T02:22:37.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8409dc20 con 0x7fcb84138170 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb894ed700 10 monclient(hunting): authenticate will time out at 2036.464860s 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb7a7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb84138170 0x7fcb84134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.063+0000 7fcb837fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb841385b0 0x7fcb84135300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb84138170 0x7fcb84134dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48208/0 (socket says 172.21.15.47:48208) 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2172517885 learned_addr learned my addr 172.21.15.47:0/2172517885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): get_auth_request con 0x7fcb841385b0 auth_method 0 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb84137d90 0x7fcb841cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): get_auth_request con 0x7fcb84138170 auth_method 0 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): get_auth_request con 0x7fcb84137d90 auth_method 0 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient(hunting): handle_auth_done global_id 54204 payload 824 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb84137d90 msgr2=0x7fcb841cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb84137d90 0x7fcb841cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb841385b0 msgr2=0x7fcb84135300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb841385b0 0x7fcb84135300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 monclient: found mon.noname-d 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74009550 con 0x7fcb84138170 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb837fe700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcb841385b0 0x7fcb84135300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7a7fc700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb84138170 0x7fcb84134dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009b90 tx=0x7fcb6c004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 client.?.objecter ms_handle_connect 0x7fcb84138170 2024-04-20T02:22:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb6c0036a0 con 0x7fcb84138170 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: dump: 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.087 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: _finish_auth 0 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:37.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 5 monclient: authenticate success, global_id 54204 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c01f070 con 0x7fcb84138170 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb6c003c90 con 0x7fcb84138170 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb81ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb7bfff700 10 monclient: dump: 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.089 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 10 monclient: _renew_subs 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c0094a0 con 0x7fcb84138170 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 10 client.54204.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 10 monclient: _renew_subs 2024-04-20T02:22:37.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb84121f90 con 0x7fcb84138170 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.064+0000 7fcb894ed700 1 librados: init done 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.065+0000 7fcb894ed700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.065+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 0x7fcb840a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.065+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.065+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84068180 con 0x7fcb8406aac0 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.065+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcb6c021030 con 0x7fcb84138170 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.065+0000 7fcb82ffd700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 0x7fcb840a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient: get_auth_request con 0x7fcb8406aac0 auth_method 0 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb6403a2f0 0x7fcb6403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(298..298 src has 1..298) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fcb6c04b9b0 con 0x7fcb84138170 2024-04-20T02:22:37.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 client.54204.objecter ms_dispatch 0x7fcb84001550 osd_map(298..298 src has 1..298) v4 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 3 client.54204.objecter handle_osd_map got epochs [298,298] > 0 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 3 client.54204.objecter handle_osd_map decoding full epoch 298 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb837fe700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb6403a2f0 0x7fcb6403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 20 client.54204.objecter dump_active .. 0 homeless 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb837fe700 10 monclient: get_auth_request con 0x7fcb6403a2f0 auth_method 0 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_done global_id 54124 payload 824 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb8409f760 con 0x7fcb8406aac0 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 client.54204.objecter ms_handle_connect 0x7fcb8406aac0 2024-04-20T02:22:37.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb82ffd700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 0x7fcb840a2510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb74011bd0 tx=0x7fcb74011f90 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 client.54204.objecter resend_mon_ops 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 client.54204.objecter ms_handle_connect 0x7fcb6403a2f0 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb837fe700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb6403a2f0 0x7fcb6403c7b0 secure :-1 s=READY pgs=3181 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c002230 tx=0x7fcb7c001a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb7400fe50 con 0x7fcb8406aac0 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.066+0000 7fcb7bfff700 10 monclient: dump: 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.093 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcb7400f9e0 con 0x7fcb8406aac0 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:37.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 msgr2=0x7fcb840a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 0x7fcb840a2510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb74011bd0 tx=0x7fcb74011f90 comp rx=0 tx=0).stop 2024-04-20T02:22:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 msgr2=0x7fcb840a2510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:37.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.067+0000 7fcb7bfff700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb8406aac0 0x7fcb840a2510 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb74011bd0 tx=0x7fcb74011f90 comp rx=0 tx=0).stop 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb894ed700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 0x7fcb840a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb894ed700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8406a400 con 0x7fcb840a2a50 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 0x7fcb840a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient: get_auth_request con 0x7fcb840a2a50 auth_method 0 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 10 monclient(hunting): handle_auth_done global_id 54133 payload 824 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 1 -- 172.21.15.47:0/2172517885 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcb840a33a0 con 0x7fcb840a2a50 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.075+0000 7fcb82ffd700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 0x7fcb840a2e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb740097f0 tx=0x7fcb74004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 client.54204.objecter ms_handle_connect 0x7fcb840a2a50 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 client.54204.objecter resend_mon_ops 2024-04-20T02:22:37.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb740148b0 con 0x7fcb840a2a50 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 monclient: dump: 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.099 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fcb74014440 con 0x7fcb840a2a50 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 msgr2=0x7fcb840a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 0x7fcb840a2e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb740097f0 tx=0x7fcb74004750 comp rx=0 tx=0).stop 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 msgr2=0x7fcb840a2e60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:37.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb7bfff700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb840a2a50 0x7fcb840a2e60 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcb740097f0 tx=0x7fcb74004750 comp rx=0 tx=0).stop 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:37.101 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:22:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:37.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:37.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:37.105 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:37.105 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:37.105 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:37.105 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:37.105 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:37.105 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.106 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:37.106 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:37.106 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:37.106 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:37.106 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:37.107 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:37.107 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:37.107 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:37.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:37.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:37.108 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:37.108 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:37.108 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:37.108 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:37.108 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:37.108 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:37.109 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.111 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:37.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:37.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.115 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:37.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.118 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.119 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:37.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:37.119 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:37.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:37.121 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.121 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.121 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:37.122 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:37.122 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.122 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.122 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:37.124 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.126 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:37.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:37.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:37.129 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 10 librados: watch_flush enter 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 10 librados: watch_flush exit 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 20 client.54204.objecter shutdown clearing up homeless session... 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 10 client.54204.objecter successfully canceled tick 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb6403a2f0 msgr2=0x7fcb6403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb6403a2f0 0x7fcb6403c7b0 secure :-1 s=READY pgs=3181 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c002230 tx=0x7fcb7c001a30 comp rx=0 tx=0).stop 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 10 monclient: shutdown 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb84138170 msgr2=0x7fcb84134dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 1 --2- 172.21.15.47:0/2172517885 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb84138170 0x7fcb84134dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009b90 tx=0x7fcb6c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:37.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb837fe700 1 -- 172.21.15.47:0/2172517885 reap_dead start 2024-04-20T02:22:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb837fe700 1 -- 172.21.15.47:0/2172517885 reap_dead start 2024-04-20T02:22:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 shutdown_connections 2024-04-20T02:22:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.076+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 >> 172.21.15.47:0/2172517885 conn(0x7fcb840aa8b0 msgr2=0x7fcb840a5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.077+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 shutdown_connections 2024-04-20T02:22:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.077+0000 7fcb894ed700 1 -- 172.21.15.47:0/2172517885 wait complete. 2024-04-20T02:22:37.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.077+0000 7fcb894ed700 1 librados: shutdown 2024-04-20T02:22:37.133 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:37.133 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:22:37.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:22:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.272+0000 7f5a13667700 10 monclient: get_monmap_and_config 2024-04-20T02:22:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.272+0000 7f5a13667700 10 monclient: build_initial_monmap 2024-04-20T02:22:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.272+0000 7f5a13667700 10 monclient: monmap: 2024-04-20T02:22:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:37.272 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:37.273789+0000 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:37.273789+0000 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:37.274 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 1 Processor -- start 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 1 -- start start 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 10 monclient: init 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 10 monclient: _add_conns ranks=[6,0,7,8,3,4,5,1,10,2,9] 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a0c0a2950 0x7f5a0c0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.273+0000 7f5a13667700 10 monclient(hunting): picked mon.noname-d con 0x7f5a0c0a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5a0c0a3270 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a0c138210 0x7f5a0c13ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 10 monclient(hunting): picked mon.noname-i con 0x7f5a0c138210 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c0751e0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c09ef20 con 0x7f5a0c0a2950 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c09e900 con 0x7f5a0c138210 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a13667700 10 monclient(hunting): authenticate will time out at 2036.675072s 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a10c02700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5a0c0a3270 0x7f5a0c137b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46312/0 (socket says 172.21.15.47:46312) 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a0c0a2950 0x7f5a0c0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a10c02700 1 -- 172.21.15.47:0/1175980368 learned_addr learned my addr 172.21.15.47:0/1175980368 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): get_auth_request con 0x7f5a0c0a2950 auth_method 0 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 1 --2- 172.21.15.47:0/1175980368 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a0c138210 0x7f5a0c13ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): get_auth_request con 0x7f5a0c138210 auth_method 0 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11c04700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a03fff700 1 -- 172.21.15.47:0/1175980368 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424430014 0 0) 0x7f5a0c0751e0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a03fff700 10 monclient(hunting): _negotiate 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a03fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a03fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a03fff700 10 monclient(hunting): my global_id is 35370 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a03fff700 1 -- 172.21.15.47:0/1175980368 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f80035b0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient(hunting): handle_auth_done global_id 54213 payload 274 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 -- 172.21.15.47:0/1175980368 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a0c138210 msgr2=0x7f5a0c13ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 --2- 172.21.15.47:0/1175980368 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a0c138210 0x7f5a0c13ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 -- 172.21.15.47:0/1175980368 >> v1:172.21.15.47:6789/0 conn(0x7f5a0c0a3270 legacy=0x7f5a0c137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient: _finish_hunting 0 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 monclient: found mon.noname-d 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 -- 172.21.15.47:0/1175980368 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c067390 con 0x7f5a0c0a2950 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.274+0000 7f5a11403700 1 --2- 172.21.15.47:0/1175980368 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a0c0a2950 0x7f5a0c0a2d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a080098a0 tx=0x7f5a08009bb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=10200d4f9013d9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 1 -- 172.21.15.47:0/1175980368 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a08011aa0 con 0x7f5a0c0a2950 2024-04-20T02:22:37.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: dump: 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.288 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.289 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:37.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 1 -- 172.21.15.47:0/1175980368 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08011c00 con 0x7f5a0c0a2950 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 5 monclient: authenticate success, global_id 54213 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 1 -- 172.21.15.47:0/1175980368 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a08011d70 con 0x7f5a0c0a2950 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 4 set_mon_vals no callback set 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a03fff700 10 monclient: dump: 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.291 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 10 monclient: shutdown 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 1 -- 172.21.15.47:0/1175980368 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a0c0a2950 msgr2=0x7f5a0c0a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 1 --2- 172.21.15.47:0/1175980368 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a0c0a2950 0x7f5a0c0a2d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5a080098a0 tx=0x7f5a08009bb0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 1 -- 172.21.15.47:0/1175980368 shutdown_connections 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 1 --2- 172.21.15.47:0/1175980368 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a0c138210 0x7f5a0c13ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 1 --2- 172.21.15.47:0/1175980368 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5a0c0a2950 0x7f5a0c0a2d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.275+0000 7f5a13667700 1 -- 172.21.15.47:0/1175980368 >> 172.21.15.47:0/1175980368 conn(0x7f5a0c0a7570 msgr2=0x7f5a0c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.276+0000 7f5a13667700 1 -- 172.21.15.47:0/1175980368 shutdown_connections 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.276+0000 7f5a13667700 1 -- 172.21.15.47:0/1175980368 wait complete. 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.276+0000 7f5a13667700 10 monclient: build_initial_monmap 2024-04-20T02:22:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.276+0000 7f5a13667700 10 monclient: monmap: 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:37.293 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.276+0000 7f5a13667700 1 librados: starting msgr at 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.276+0000 7f5a13667700 1 librados: starting objecter 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 Processor -- start 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 -- start start 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 librados: setting wanted keys 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 librados: calling monclient init 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient: init 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient: _add_conns ranks=[3,8,1,2,5,7,0,6,4] 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c0a2950 0x7f5a0c136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): picked mon.noname-g con 0x7f5a0c0a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 0x7f5a0c1373b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): picked mon.noname-e con 0x7f5a0c0a3270 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c138210 0x7f5a0c1cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): picked mon.noname-f con 0x7f5a0c138210 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c123c80 con 0x7f5a0c138210 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c1378f0 con 0x7f5a0c0a2950 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c123f90 con 0x7f5a0c0a3270 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a13667700 10 monclient(hunting): authenticate will time out at 2036.678623s 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 0x7f5a0c1373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 0x7f5a0c1373b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58294/0 (socket says 172.21.15.47:58294) 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 1 -- 172.21.15.47:0/3368191711 learned_addr learned my addr 172.21.15.47:0/3368191711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): get_auth_request con 0x7f5a0c0a3270 auth_method 0 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11403700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c0a2950 0x7f5a0c136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a10c02700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11c04700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c138210 0x7f5a0c1cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11403700 10 monclient(hunting): get_auth_request con 0x7f5a0c0a2950 auth_method 0 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11403700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11403700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11403700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11c04700 10 monclient(hunting): get_auth_request con 0x7f5a0c138210 auth_method 0 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11c04700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11c04700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.277+0000 7f5a11c04700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a11403700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a11403700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a11403700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a11c04700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a11c04700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a11c04700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 10 monclient(hunting): handle_auth_done global_id 35072 payload 824 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 -- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c0a2950 msgr2=0x7f5a0c136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c0a2950 0x7f5a0c136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 -- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c138210 msgr2=0x7f5a0c1cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c138210 0x7f5a0c1cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 10 monclient: _finish_hunting 0 2024-04-20T02:22:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 monclient: found mon.noname-e 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 -- 172.21.15.47:0/3368191711 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080094a0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a10c02700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 0x7f5a0c1373b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59fc001eb0 tx=0x7f59fc00c7b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 client.?.objecter ms_handle_connect 0x7f5a0c0a3270 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f59fc0070d0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: dump: 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.299 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc007270 con 0x7f5a0c0a3270 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 5 monclient: authenticate success, global_id 35072 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 10 monclient: _renew_subs 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f59fc010300 con 0x7f5a0c0a3270 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c124270 con 0x7f5a0c0a3270 2024-04-20T02:22:37.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a037fe700 4 set_mon_vals no callback set 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a01ffb700 10 monclient: dump: 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 10 client.35072.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 10 monclient: _renew_subs 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c09cb20 con 0x7f5a0c0a3270 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.278+0000 7f5a13667700 1 librados: init done 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.279+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f59fc005b70 con 0x7f5a0c0a3270 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f59f003a2f0 0x7f59f003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a13667700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:37.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a13667700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a13667700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c0008d0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(298..298 src has 1..298) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f59fc0497c0 con 0x7f5a0c0a3270 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 10 client.35072.objecter ms_dispatch 0x7f5a0c001550 osd_map(298..298 src has 1..298) v4 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 3 client.35072.objecter handle_osd_map got epochs [298,298] > 0 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 3 client.35072.objecter handle_osd_map decoding full epoch 298 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 20 client.35072.objecter dump_active .. 0 homeless 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a11403700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f59f003a2f0 0x7f59f003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a11403700 10 monclient: get_auth_request con 0x7f59f003a2f0 auth_method 0 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a01ffb700 10 client.35072.objecter ms_handle_connect 0x7f59f003a2f0 2024-04-20T02:22:37.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.280+0000 7f5a11403700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f59f003a2f0 0x7f59f003c7b0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7f5a080090b0 tx=0x7f5a0801c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:37.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.285+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f59fc017020 con 0x7f5a0c0a3270 2024-04-20T02:22:37.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.285+0000 7f5a01ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:37.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.285+0000 7f5a01ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.597+0000 7f5a13667700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:22:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.597+0000 7f5a13667700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:22:37.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.597+0000 7f5a13667700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:37.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.597+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f5a0c123030 con 0x7f5a0c0a3270 2024-04-20T02:22:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.599+0000 7f5a01ffb700 1 -- 172.21.15.47:0/3368191711 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f59fc018860 con 0x7f5a0c0a3270 2024-04-20T02:22:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.599+0000 7f5a01ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:22:37.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.599+0000 7f5a01ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:37.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=12,config=2,health=10,kv=11,logm=65} crc {auth=793961017,config=692619927,health=1216691399,kv=1530195301,logm=1629627046}) 2024-04-20T02:22:37.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=1991903100,config=692619927,health=2941319762,kv=2083578540,logm=4161246457}) 2024-04-20T02:22:37.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=12,config=2,health=8,kv=8,logm=70} crc {auth=793961017,config=692619927,health=303483429,kv=3050069044,logm=3380037135}) 2024-04-20T02:22:37.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=12,config=2,health=10,kv=11,logm=65} crc {auth=793961017,config=692619927,health=867538651,kv=1530195301,logm=1413154969}) 2024-04-20T02:22:37.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=1991903100,config=692619927,health=2941319762,kv=1530195301,logm=1022729595}) 2024-04-20T02:22:37.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=11,config=2,health=11,kv=11,logm=65} crc {auth=1200783306,config=692619927,health=872527168,kv=1530195301,logm=546085276}) 2024-04-20T02:22:37.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=3930766642,config=692619927,health=2941319762,kv=1530195301,logm=3670877987}) 2024-04-20T02:22:37.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=793961017,config=692619927,health=2941319762,kv=1530195301,logm=3557896310}) 2024-04-20T02:22:37.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.599+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=12,config=2,health=11,kv=11,logm=64} crc {auth=793961017,config=692619927,health=262107873,kv=2525026046,logm=558863716}) 2024-04-20T02:22:37.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1229182935}) 2024-04-20T02:22:37.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=101608721}) 2024-04-20T02:22:37.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=94882930}) 2024-04-20T02:22:37.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=214485667}) 2024-04-20T02:22:37.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.602+0000 7f5a13667700 10 librados: watch_flush enter 2024-04-20T02:22:37.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2722719835}) 2024-04-20T02:22:37.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.602+0000 7f5a13667700 10 librados: watch_flush exit 2024-04-20T02:22:37.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.602+0000 7f5a13667700 20 client.35072.objecter shutdown clearing up homeless session... 2024-04-20T02:22:37.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.602+0000 7f5a13667700 10 client.35072.objecter successfully canceled tick 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=499621628}) 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1264207746}) 2024-04-20T02:22:37.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.602+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f59f003a2f0 msgr2=0x7f59f003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.602+0000 7f5a13667700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f59f003a2f0 0x7f59f003c7b0 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 crypto rx=0x7f5a080090b0 tx=0x7f5a0801c040 comp rx=0 tx=0).stop 2024-04-20T02:22:37.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 10 monclient: shutdown 2024-04-20T02:22:37.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.602+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3859951226}) 2024-04-20T02:22:37.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.603+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2820227788}) 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 msgr2=0x7f5a0c1373b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 0x7f5a0c1373b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59fc001eb0 tx=0x7f59fc00c7b0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 shutdown_connections 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f59f003a2f0 0x7f59f003c7b0 unknown :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a0c138210 0x7f5a0c1cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a0c0a2950 0x7f5a0c136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 --2- 172.21.15.47:0/3368191711 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5a0c0a3270 0x7f5a0c1373b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 >> 172.21.15.47:0/3368191711 conn(0x7f5a0c0a7570 msgr2=0x7f5a0c0a5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.603+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 shutdown_connections 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.604+0000 7f5a13667700 1 -- 172.21.15.47:0/3368191711 wait complete. 2024-04-20T02:22:37.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:37.604+0000 7f5a13667700 1 librados: shutdown 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545099024}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1047043609}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2205771156}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3675278950}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3676590557}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2502500928}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=88636073}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1482827776}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.605+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1633464650}) 2024-04-20T02:22:37.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2896690425}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3436696616}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=203483607}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=232976310}) 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3504611854}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3787234382}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1156130254}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1936148088}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.607+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=8872425}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=25077101}) 2024-04-20T02:22:37.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=918185924}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=761690518}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2118160746}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3510511200}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096508272}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=471808653}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222334109}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.609+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=519602035}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.610+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.610+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.610+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3223285699}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.610+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.610+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.610+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3979110353}) 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2469210657}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2692630865}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2423161082}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3780747534}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=495549835}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3289225587}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.611+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2034964602}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3408099172}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2967529942}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1226009450}) 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.612+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=470044756}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1193923942}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4098401268}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=488082211}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3212138848}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.613+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=506828085}) 2024-04-20T02:22:37.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738427358}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1677308015}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3427024331}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=443684590}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953121945}) 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.614+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2863408879}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1825643131}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1090987568}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.615+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3422697235}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=294880647}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3355494498}) 2024-04-20T02:22:37.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=299592605}) 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=634541865}) 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2944817366}) 2024-04-20T02:22:37.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=713442316}) 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.616+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.617+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.617+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=899144459}) 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.617+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.617+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1614134518}) 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.617+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=228943428}) 2024-04-20T02:22:37.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944168038}) 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4014655854}) 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=194178604}) 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=354903681}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2996046128}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665819542}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3463039018}) 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3794591475}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.619+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180650282}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2859312214}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3682509034}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2746857124}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1786622845}) 2024-04-20T02:22:37.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3016332413}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4240637811}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2786059658}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3208033436}) 2024-04-20T02:22:37.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.621+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4106661849}) 2024-04-20T02:22:37.656 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:22:37.752 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4137038290}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4074878186}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2333466257}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3932191618}) 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.753 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2118815091}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2857416019}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=105106876}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49409172}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.623+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=913588551}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.754 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2409737548}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=966890449}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3895751314}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4288134215}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.755 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2655525379}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2686301641}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3289883722}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2454886296}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2310496896}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=313874442}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.756 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1722784228}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=112147281}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3697053471}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2416895953}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1591675728}) 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.757 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1490896011}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170298461}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3753045844}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1082223444}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3086712299}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1682549891}) 2024-04-20T02:22:37.758 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3191351176}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3445404387}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1590686366}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4012001654}) 2024-04-20T02:22:37.759 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908550957}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.629+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2419558270}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2540487151}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=522902073}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4048530200}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1661131579}) 2024-04-20T02:22:37.760 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1284193424}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=870949693}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3363369458}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2234169052}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.632+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2761085314}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.761 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=669772059}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1424876870}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3928543590}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2375783301}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3857310181}) 2024-04-20T02:22:37.762 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135761070}) 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4208384371}) 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656385630}) 2024-04-20T02:22:37.763 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2498711278}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1746727132}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1841374151}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2251124165}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3158038988}) 2024-04-20T02:22:37.764 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3443203857}) 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1473828252}) 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=991136085}) 2024-04-20T02:22:37.765 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696218849}) 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2078287555}) 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1279426374}) 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2431514241}) 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.766 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=304676685}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1654524057}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1567536993}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4110529315}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.639+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1959497946}) 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.639+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.767 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.639+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995896504}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.639+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2440544233}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2949158629}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=577937798}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4005114999}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=41724311}) 2024-04-20T02:22:37.768 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=228946502}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1102913767}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2721090370}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494455599}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3918619432}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.769 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1435723877}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2626804963}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1078849501}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1167673090}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=524625548}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.770 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1477108892}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=484581659}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1236610459}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=935408821}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725062001}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3312346911}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.771 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=333534010}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2340523944}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1888811920}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=719147473}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1976809887}) 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.772 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3652744132}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2119430607}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4187631795}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2471413102}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1038301080}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3786348611}) 2024-04-20T02:22:37.773 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1752054356}) 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088940513}) 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=698654132}) 2024-04-20T02:22:37.774 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2435668482}) 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1293978261}) 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328847496}) 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.775 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3460790183}) 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=333150186}) 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3330155155}) 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.776 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2492598790}) 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4199684887}) 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4098803164}) 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.777 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1537904111}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.648+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=468690149}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=602239625}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=119377493}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1306572005}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.778 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2353390626}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3570161456}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3602007787}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2445585135}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3828455383}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629942011}) 2024-04-20T02:22:37.779 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3904604103}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1625298805}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2274827315}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1988615089}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.780 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=393010976}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=861402271}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3165671532}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814748606}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.652+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2375449868}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.653+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172587891}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.781 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2249314935}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1940394921}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2412415549}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4060729901}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3759537684}) 2024-04-20T02:22:37.782 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2947211834}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3776000228}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.654+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3870210649}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=948164579}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4002527387}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820537122}) 2024-04-20T02:22:37.783 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1565665247}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3404653348}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3529531635}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1544083931}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3691132225}) 2024-04-20T02:22:37.784 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.655+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1305603972}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3290073709}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2782332401}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2363702237}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1447241199}) 2024-04-20T02:22:37.785 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2950200202}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3064067483}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1661883031}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558028772}) 2024-04-20T02:22:37.786 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.657+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4095497972}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2403751762}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415261548}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=658495654}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.787 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1653985447}) 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3951462828}) 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1388060939}) 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.788 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3062759724}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272217027}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.659+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2246215375}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662846627}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1412925727}) 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.789 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2929899980}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=376858471}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3804856477}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4098852287}) 2024-04-20T02:22:37.790 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=339599500}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2729456460}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1222842053}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2685662383}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4272155193}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.791 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2500092778}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2556205780}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3692021567}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3738326549}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2538744528}) 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.792 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=575434384}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.663+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3335779063}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=191787709}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3929774180}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2773268753}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3595254128}) 2024-04-20T02:22:37.793 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=748771044}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4172961576}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1111247591}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1824894959}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=171179102}) 2024-04-20T02:22:37.794 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.795 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.795 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=862183698}) 2024-04-20T02:22:37.795 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.795 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.795 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3199258538}) 2024-04-20T02:22:37.795 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.796 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.796 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2377757021}) 2024-04-20T02:22:37.796 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.796 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.796 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1350569243}) 2024-04-20T02:22:37.797 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.797 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.797 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2887718536}) 2024-04-20T02:22:37.797 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.797 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.798 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2897673497}) 2024-04-20T02:22:37.798 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.798 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.798 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1121998654}) 2024-04-20T02:22:37.798 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.799 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333294620}) 2024-04-20T02:22:37.799 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.668+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1293541353}) 2024-04-20T02:22:37.799 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.799 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.799 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=785150578}) 2024-04-20T02:22:37.800 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.800 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.800 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=863460940}) 2024-04-20T02:22:37.800 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.800 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.800 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3355542940}) 2024-04-20T02:22:37.801 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.801 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.801 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3539864891}) 2024-04-20T02:22:37.801 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.801 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1892895235}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=553797019}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3059152092}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4273591778}) 2024-04-20T02:22:37.802 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2209960824}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2179651096}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4019549471}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.671+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=548037801}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=345719420}) 2024-04-20T02:22:37.803 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=490502188}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1094590930}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4255796243}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4054849710}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3226722486}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.804 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188870774}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2846722247}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3122859243}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2955970127}) 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.805 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3641295889}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4210062884}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1889265229}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619007554}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.674+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3808665983}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.806 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1064557973}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1057505920}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=636731188}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=836540073}) 2024-04-20T02:22:37.807 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3029347567}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2207806817}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.676+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.676+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.676+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3954431131}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.676+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.676+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1306981058}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.676+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=210934839}) 2024-04-20T02:22:37.808 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.809 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.809 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4084259090}) 2024-04-20T02:22:37.809 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.809 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.809 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=659833129}) 2024-04-20T02:22:37.809 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.810 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.810 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4196503440}) 2024-04-20T02:22:37.810 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.810 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.810 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3288436764}) 2024-04-20T02:22:37.810 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.811 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.811 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1911441877}) 2024-04-20T02:22:37.811 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.811 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.811 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4049929073}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1143349018}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532956465}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3070761369}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2441073}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.812 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2057507367}) 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4176932132}) 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1270395926}) 2024-04-20T02:22:37.813 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=335811761}) 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2804541765}) 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3849976405}) 2024-04-20T02:22:37.814 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441808273}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.679+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3316069308}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1255449113}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3305252608}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3630481761}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3403086413}) 2024-04-20T02:22:37.815 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038209158}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1181213466}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2204878334}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=571204305}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.682+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=466235334}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.816 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2591688345}) 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2992315874}) 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2708511048}) 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.817 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3364105978}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2783833942}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3614111720}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2808445792}) 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.818 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=873025932}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2095204839}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469956712}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112662940}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2099198322}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2760960996}) 2024-04-20T02:22:37.819 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3442267669}) 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=637359033}) 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2053523453}) 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.820 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=441217070}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2253845622}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19} crc {logm=1458035807,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2726072816,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071527,osd_snap=563383936}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20} crc {logm=2494804183,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2945615395,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071527,osd_snap=2800195436}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=34,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=23} crc {logm=948324912,mdsmap=3585761405,mgr=4024091195,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4099217683,mon_config_key=2120219358,monmap=2370797809,osd_metadata=3566517105,osd_pg_creating=2084071527,osd_snap=791831361}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=36,mdsmap=2,mgr=5,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=9,monmap=2,osd_metadata=6,osd_snap=25} crc {logm=158917121,mdsmap=3140514009,mgr=3430823933,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3705215958,mon_config_key=3172738919,monmap=3140514009,osd_metadata=3566517105,osd_snap=2914951293}) 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.821 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=32,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=21} crc {logm=2088644411,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2470188379,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517105,osd_pg_creating=2084071527,osd_snap=96873661}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=31,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=23} crc {logm=3005193612,mdsmap=3585761405,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2470188379,mon_config_key=2868358642,monmap=3385036551,osd_metadata=1493347452,osd_pg_creating=2084071527,osd_snap=902973910}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=9,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=24} crc {logm=2222150466,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1250505715,mon_config_key=3277655795,monmap=3385036551,osd_metadata=557344636,osd_pg_creating=2084071527,osd_snap=1339776253}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22} crc {logm=3378707054,mdsmap=3585761405,mgr=57141823,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=816312026,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071528,osd_snap=1376527550}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18} crc {logm=3323211586,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2470188379,mon_config_key=1169884507,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2084071527,osd_snap=2909382330}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.822 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=3729222748}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=464019627}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4268961122}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=264642236}) 2024-04-20T02:22:37.823 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=2744691278}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=3081300467}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=3997332273}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=4049590494}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=5,osdmap=95} crc {osd_snap=3932281927,osdmap=2126928411}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.824 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3874710336}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2988414226}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3810121394}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1245291265}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3390613844}) 2024-04-20T02:22:37.825 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3101321418}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=895153097}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=167607191}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1035868239}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3212214439}) 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.826 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1376539164}) 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4154796677}) 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2049897105}) 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.827 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2110233666}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3894004006}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3026298317}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1780804429}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=159359040}) 2024-04-20T02:22:37.828 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3128907002}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2941866723}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2955165866}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1848997212}) 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.829 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.697+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=717189729}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1059093966}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=736862694}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1209491180}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=478349842}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=46} crc {osdmap=1966670427}) 2024-04-20T02:22:37.830 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=48} crc {osdmap=1734516686}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=46} crc {osdmap=4234383898}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=43} crc {osdmap=346850237}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=41} crc {osdmap=3197352944}) 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.831 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=47} crc {osdmap=2177803975}) 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=48} crc {osdmap=2915959936}) 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=46} crc {osdmap=1016373016}) 2024-04-20T02:22:37.832 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:22:37.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=48} crc {osdmap=320898843}) 2024-04-20T02:22:38.657 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:22:38.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.800+0000 7fb68d68c700 10 monclient: get_monmap_and_config 2024-04-20T02:22:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.800+0000 7fb68d68c700 10 monclient: build_initial_monmap 2024-04-20T02:22:38.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.800+0000 7fb68d68c700 10 monclient: monmap: 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:38.801530+0000 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:38.801530+0000 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:38.817 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.800+0000 7fb68d68c700 1 Processor -- start 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 1 -- start start 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient: init 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient: _add_conns ranks=[10,7,0,6,2,8,9,3,4,5,1] 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6880a4c80 0x7fb6880a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): picked mon.noname-f con 0x7fb6880a4c80 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6880a0d20 0x7fb6880a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): picked mon.noname-i con 0x7fb6880a0d20 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6880a55a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880751e0 con 0x7fb6880a55a0 2024-04-20T02:22:38.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6880a0000 con 0x7fb6880a4c80 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880a6a10 con 0x7fb6880a0d20 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb68d68c700 10 monclient(hunting): authenticate will time out at 2038.202626s 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb6877fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb6880a55a0 0x7fb688137a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46326/0 (socket says 172.21.15.47:46326) 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb6877fe700 1 -- 172.21.15.47:0/1556079751 learned_addr learned my addr 172.21.15.47:0/1556079751 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb6867fc700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6880a0d20 0x7fb6880a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.801+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196810063 0 0) 0x7fb6880751e0 con 0x7fb6880a55a0 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): my global_id is 35397 2024-04-20T02:22:38.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6700035b0 con 0x7fb6880a55a0 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): get_auth_request con 0x7fb6880a0d20 auth_method 0 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 1 --2- 172.21.15.47:0/1556079751 >> v2:172.21.15.179:3300/0 conn(0x7fb6880a4c80 0x7fb6880a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): get_auth_request con 0x7fb6880a4c80 auth_method 0 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 69082016 0 0) 0x7fb6700035b0 con 0x7fb6880a55a0 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6880a0d20 msgr2=0x7fb6880a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6880a0d20 0x7fb6880a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 >> v2:172.21.15.179:3300/0 conn(0x7fb6880a4c80 msgr2=0x7fb6880a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 --2- 172.21.15.47:0/1556079751 >> v2:172.21.15.179:3300/0 conn(0x7fb6880a4c80 0x7fb6880a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: _finish_hunting 0 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68809e440 con 0x7fb6880a55a0 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb68d68c700 5 monclient: authenticate success, global_id 35397 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb68d68c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:38.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7fb680001c60 con 0x7fb6880a55a0 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: dump: 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:38.822 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 >> v1:172.21.15.47:6789/0 conn(0x7fb6880a55a0 legacy=0x7fb688137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:38.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient: _add_conns ranks=[5,4,1,8,2,3,6,0,7] 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb670004250 0x7fb670004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): picked mon.h con 0x7fb670004250 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb670004d90 0x7fb670005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): picked mon.c con 0x7fb670004d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb670009b00 0x7fb67000bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): picked mon.f con 0x7fb670009b00 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880a0000 con 0x7fb670009b00 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880a6a10 con 0x7fb670004d90 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb67000c8a0 con 0x7fb670004250 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:38.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb685ffb700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb68d68c700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb670004d90 0x7fb670005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): get_auth_request con 0x7fb670004d90 auth_method 0 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb670004250 0x7fb670004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6877fe700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb670009b00 0x7fb67000bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): get_auth_request con 0x7fb670004250 auth_method 0 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb686ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6877fe700 10 monclient(hunting): get_auth_request con 0x7fb670009b00 auth_method 0 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6877fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6877fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6877fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient(hunting): handle_auth_done global_id 35397 payload 293 2024-04-20T02:22:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 -- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb670004250 msgr2=0x7fb670004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb670004250 0x7fb670004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 -- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb670009b00 msgr2=0x7fb67000bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb670009b00 0x7fb67000bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient: _finish_hunting 0 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 monclient: found mon.c 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6877fe700 1 -- 172.21.15.47:0/1556079751 reap_dead start 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 1 -- 172.21.15.47:0/1556079751 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700035b0 con 0x7fb670004d90 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.802+0000 7fb6867fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb6867fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb6867fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:38.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb6867fc700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb670004d90 0x7fb670005210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6880ab1a0 tx=0x7fb670005880 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c688f9b57b636b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb67c00eb30 con 0x7fb670004d90 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: dump: 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67c00ecf0 con 0x7fb670004d90 2024-04-20T02:22:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 1 -- 172.21.15.47:0/1556079751 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb67c00ee60 con 0x7fb670004d90 2024-04-20T02:22:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb685ffb700 10 monclient: dump: 2024-04-20T02:22:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 4 set_mon_vals no callback set 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 10 monclient: shutdown 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 1 -- 172.21.15.47:0/1556079751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb670004d90 msgr2=0x7fb670005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb670004d90 0x7fb670005210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6880ab1a0 tx=0x7fb670005880 comp rx=0 tx=0).stop 2024-04-20T02:22:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 1 -- 172.21.15.47:0/1556079751 shutdown_connections 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 1 --2- 172.21.15.47:0/1556079751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb670004d90 0x7fb670005210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.803+0000 7fb68d68c700 1 -- 172.21.15.47:0/1556079751 >> 172.21.15.47:0/1556079751 conn(0x7fb6880aa280 msgr2=0x7fb6880aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.804+0000 7fb68d68c700 1 -- 172.21.15.47:0/1556079751 shutdown_connections 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.804+0000 7fb68d68c700 1 -- 172.21.15.47:0/1556079751 wait complete. 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.804+0000 7fb68d68c700 10 monclient: build_initial_monmap 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.804+0000 7fb68d68c700 10 monclient: monmap: 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.804+0000 7fb68d68c700 1 librados: starting msgr at 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.804+0000 7fb68d68c700 1 librados: starting objecter 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 Processor -- start 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 -- start start 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 librados: setting wanted keys 2024-04-20T02:22:38.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 librados: calling monclient init 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient: init 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient: _add_conns ranks=[0,7,1,6,2,8,4,3,5] 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 0x7fb6881cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): picked mon.noname-a con 0x7fb670004d90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6881bc880 0x7fb6881bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): picked mon.noname-i con 0x7fb6881bc880 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6881bd1e0 0x7fb6881a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): picked mon.noname-f con 0x7fb6881bd1e0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68809ea00 con 0x7fb670004d90 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68809dd20 con 0x7fb6881bd1e0 2024-04-20T02:22:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881aa240 con 0x7fb6881bc880 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb68d68c700 10 monclient(hunting): authenticate will time out at 2038.206857s 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 0x7fb6881cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 0x7fb6881cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60924/0 (socket says 172.21.15.47:60924) 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 1 -- 172.21.15.47:0/748840680 learned_addr learned my addr 172.21.15.47:0/748840680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 10 monclient(hunting): get_auth_request con 0x7fb670004d90 auth_method 0 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.805+0000 7fb686ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6881bd1e0 0x7fb6881a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): get_auth_request con 0x7fb6881bd1e0 auth_method 0 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6881bc880 0x7fb6881bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): get_auth_request con 0x7fb6881bc880 auth_method 0 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6877fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6867fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 10 monclient(hunting): handle_auth_done global_id 35406 payload 824 2024-04-20T02:22:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 -- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6881bc880 msgr2=0x7fb6881bcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6881bc880 0x7fb6881bcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 -- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6881bd1e0 msgr2=0x7fb6881a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6881bd1e0 0x7fb6881a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 monclient: found mon.noname-a 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 -- 172.21.15.47:0/748840680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6881a87c0 con 0x7fb670004d90 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb686ffd700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 0x7fb6881cc400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb678010890 tx=0x7fb678010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 client.?.objecter ms_handle_connect 0x7fb670004d90 2024-04-20T02:22:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6780044a0 con 0x7fb670004d90 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: dump: 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67800c050 con 0x7fb670004d90 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 5 monclient: authenticate success, global_id 35406 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 10 monclient: _renew_subs 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6881a99a0 con 0x7fb670004d90 2024-04-20T02:22:38.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb678003b10 con 0x7fb670004d90 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb66f7fe700 10 monclient: dump: 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 10 client.35406.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb6857fa700 4 set_mon_vals no callback set 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 10 monclient: _renew_subs 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6881a6820 con 0x7fb670004d90 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.806+0000 7fb68d68c700 1 librados: init done 2024-04-20T02:22:38.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb678003c90 con 0x7fb670004d90 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb68d68c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb68d68c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb68d68c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6881a9ea0 con 0x7fb670004d90 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb66f7fe700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb67403a2f0 0x7fb67403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(300..300 src has 1..300) v4 ==== 6536+0+0 (secure 0 0 0) 0x7fb67804ae50 con 0x7fb670004d90 2024-04-20T02:22:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb66f7fe700 10 client.35406.objecter ms_dispatch 0x7fb688001550 osd_map(300..300 src has 1..300) v4 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb66f7fe700 3 client.35406.objecter handle_osd_map got epochs [300,300] > 0 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.807+0000 7fb66f7fe700 3 client.35406.objecter handle_osd_map decoding full epoch 300 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.808+0000 7fb66f7fe700 20 client.35406.objecter dump_active .. 0 homeless 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.808+0000 7fb6867fc700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb67403a2f0 0x7fb67403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.808+0000 7fb6867fc700 10 monclient: get_auth_request con 0x7fb67403a2f0 auth_method 0 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.808+0000 7fb66f7fe700 10 client.35406.objecter ms_handle_connect 0x7fb67403a2f0 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.808+0000 7fb6867fc700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb67403a2f0 0x7fb67403c7b0 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000c00 tx=0x7fb67c01f040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.812+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fb6881a9ea0 con 0x7fb670004d90 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.812+0000 7fb66f7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:38.812+0000 7fb66f7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb68d68c700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb68d68c700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb68d68c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb6881a9ea0 con 0x7fb670004d90 2024-04-20T02:22:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb66f7fe700 1 -- 172.21.15.47:0/748840680 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fb6881a9ea0 con 0x7fb670004d90 2024-04-20T02:22:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb66f7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.144+0000 7fb66f7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:39.145 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":5,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.146+0000 7fb68d68c700 10 librados: watch_flush enter 2024-04-20T02:22:39.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 10 librados: watch_flush exit 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 20 client.35406.objecter shutdown clearing up homeless session... 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 10 client.35406.objecter successfully canceled tick 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb67403a2f0 msgr2=0x7fb67403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb67403a2f0 0x7fb67403c7b0 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000c00 tx=0x7fb67c01f040 comp rx=0 tx=0).stop 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 10 monclient: shutdown 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 msgr2=0x7fb6881cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 0x7fb6881cc400 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb678010890 tx=0x7fb678010ba0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 shutdown_connections 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb67403a2f0 0x7fb67403c7b0 unknown :-1 s=CLOSED pgs=3184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6881bd1e0 0x7fb6881a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb670004d90 0x7fb6881cc400 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 --2- 172.21.15.47:0/748840680 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb6881bc880 0x7fb6881bcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 >> 172.21.15.47:0/748840680 conn(0x7fb6880aa280 msgr2=0x7fb6881bfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 shutdown_connections 2024-04-20T02:22:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 -- 172.21.15.47:0/748840680 wait complete. 2024-04-20T02:22:39.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.147+0000 7fb68d68c700 1 librados: shutdown 2024-04-20T02:22:39.161 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:22:39.161 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:22:39.161 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:22:39.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:22:39.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.299+0000 7fbf817dc700 10 monclient: get_monmap_and_config 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.299+0000 7fbf817dc700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.299+0000 7fbf817dc700 10 monclient: monmap: 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:39.300919+0000 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:39.300919+0000 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:39.301 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 1 Processor -- start 2024-04-20T02:22:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 1 -- start start 2024-04-20T02:22:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 10 monclient: init 2024-04-20T02:22:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 10 monclient: _add_conns ranks=[10,2,8,1,5,4,7,6,3,0,9] 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbf7c137d90 0x7fbf7c0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 10 monclient(hunting): picked mon.noname-f con 0x7fbf7c137d90 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c138170 0x7fbf7c0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 10 monclient(hunting): picked mon.noname-b con 0x7fbf7c138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 0x7fbf7c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.300+0000 7fbf817dc700 10 monclient(hunting): picked mon.noname-e con 0x7fbf7c1385b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c09fa40 con 0x7fbf7c137d90 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c09dfc0 con 0x7fbf7c138170 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0a7bc0 con 0x7fbf7c1385b0 2024-04-20T02:22:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf817dc700 10 monclient(hunting): authenticate will time out at 2038.702044s 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 0x7fbf7c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c138170 0x7fbf7c0a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 0x7fbf7c0a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58326/0 (socket says 172.21.15.47:58326) 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 1 -- 172.21.15.47:0/2180943951 learned_addr learned my addr 172.21.15.47:0/2180943951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): get_auth_request con 0x7fbf7c1385b0 auth_method 0 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): get_auth_request con 0x7fbf7c138170 auth_method 0 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 1 --2- 172.21.15.47:0/2180943951 >> v2:172.21.15.179:3300/0 conn(0x7fbf7c137d90 0x7fbf7c0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): get_auth_request con 0x7fbf7c137d90 auth_method 0 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_done global_id 23888 payload 274 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 msgr2=0x7fbf7c0a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 --2- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 0x7fbf7c0a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/2180943951 >> v2:172.21.15.179:3300/0 conn(0x7fbf7c137d90 msgr2=0x7fbf7c0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 --2- 172.21.15.47:0/2180943951 >> v2:172.21.15.179:3300/0 conn(0x7fbf7c137d90 0x7fbf7c0a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 monclient: found mon.noname-b 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/2180943951 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0751e0 con 0x7fbf7c138170 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7b7fe700 1 --2- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 0x7fbf7c0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf7a7fc700 1 --2- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c138170 0x7fbf7c0a5c90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009870 tx=0x7fbf70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a342ed4473527f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf79ffb700 1 -- 172.21.15.47:0/2180943951 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf70011aa0 con 0x7fbf7c138170 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf79ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf79ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.301+0000 7fbf79ffb700 10 monclient: dump: 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.318 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 1 -- 172.21.15.47:0/2180943951 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf70011c00 con 0x7fbf7c138170 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 5 monclient: authenticate success, global_id 23888 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 1 -- 172.21.15.47:0/2180943951 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf70011d70 con 0x7fbf7c138170 2024-04-20T02:22:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 4 set_mon_vals no callback set 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf79ffb700 10 monclient: dump: 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.320 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 10 monclient: shutdown 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 -- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c138170 msgr2=0x7fbf7c0a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 --2- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c138170 0x7fbf7c0a5c90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009870 tx=0x7fbf70009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 -- 172.21.15.47:0/2180943951 shutdown_connections 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 --2- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fbf7c1385b0 0x7fbf7c0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 --2- 172.21.15.47:0/2180943951 >> v2:172.21.15.179:3300/0 conn(0x7fbf7c137d90 0x7fbf7c0a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 --2- 172.21.15.47:0/2180943951 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c138170 0x7fbf7c0a5c90 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 -- 172.21.15.47:0/2180943951 >> 172.21.15.47:0/2180943951 conn(0x7fbf7c0aa8b0 msgr2=0x7fbf7c0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 -- 172.21.15.47:0/2180943951 shutdown_connections 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.302+0000 7fbf817dc700 1 -- 172.21.15.47:0/2180943951 wait complete. 2024-04-20T02:22:39.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 10 monclient: monmap: 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.322 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 1 librados: starting msgr at 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 1 librados: starting objecter 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 1 Processor -- start 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 1 -- start start 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 1 librados: setting wanted keys 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 1 librados: calling monclient init 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.303+0000 7fbf817dc700 10 monclient: init 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient: _add_conns ranks=[3,1,5,8,2,7,6,4,0] 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbf7c137d90 0x7fbf7c1cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): picked mon.noname-g con 0x7fbf7c137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c138170 0x7fbf7c136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): picked mon.noname-f con 0x7fbf7c138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbf7c1385b0 0x7fbf7c137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): picked mon.noname-h con 0x7fbf7c1385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c123e60 con 0x7fbf7c138170 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c09e9e0 con 0x7fbf7c137d90 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c09dc20 con 0x7fbf7c1385b0 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf817dc700 10 monclient(hunting): authenticate will time out at 2038.705539s 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c138170 0x7fbf7c136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbf7c137d90 0x7fbf7c1cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c138170 0x7fbf7c136d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:51256/0 (socket says 172.21.15.47:51256) 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbf7c1385b0 0x7fbf7c137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/1856739583 learned_addr learned my addr 172.21.15.47:0/1856739583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): get_auth_request con 0x7fbf7c138170 auth_method 0 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7affd700 10 monclient(hunting): get_auth_request con 0x7fbf7c137d90 auth_method 0 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 10 monclient(hunting): get_auth_request con 0x7fbf7c1385b0 auth_method 0 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.304+0000 7fbf7b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 10 monclient(hunting): handle_auth_done global_id 54244 payload 824 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbf7c1385b0 msgr2=0x7fbf7c137280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbf7c1385b0 0x7fbf7c137280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbf7c137d90 msgr2=0x7fbf7c1cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbf7c137d90 0x7fbf7c1cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 monclient: found mon.noname-f 2024-04-20T02:22:39.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf74009b30 con 0x7fbf7c138170 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf7a7fc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c138170 0x7fbf7c136d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbf70013000 tx=0x7fbf70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 client.?.objecter ms_handle_connect 0x7fbf7c138170 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf70004410 con 0x7fbf7c138170 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: dump: 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.327 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7001f070 con 0x7fbf7c138170 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 5 monclient: authenticate success, global_id 54244 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 10 monclient: _renew_subs 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf700094a0 con 0x7fbf7c138170 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf700045b0 con 0x7fbf7c138170 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf797fa700 4 set_mon_vals no callback set 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 10 client.54244.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf637fe700 10 monclient: dump: 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.329 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 10 monclient: _renew_subs 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:22:39.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c122c80 con 0x7fbf7c138170 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.305+0000 7fbf817dc700 1 librados: init done 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbf700036a0 con 0x7fbf7c138170 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf817dc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf817dc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 0x7fbf7c1208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c068180 con 0x7fbf7c06aac0 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 0x7fbf7c1208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient: get_auth_request con 0x7fbf7c06aac0 auth_method 0 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbf6803a2f0 0x7fbf6803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(301..301 src has 1..301) v4 ==== 6645+0+0 (secure 0 0 0) 0x7fbf70021070 con 0x7fbf7c138170 2024-04-20T02:22:39.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 10 client.54244.objecter ms_dispatch 0x7fbf7c001550 osd_map(301..301 src has 1..301) v4 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 3 client.54244.objecter handle_osd_map got epochs [301,301] > 0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 3 client.54244.objecter handle_osd_map decoding full epoch 301 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7b7fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbf6803a2f0 0x7fbf6803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf7b7fe700 10 monclient: get_auth_request con 0x7fbf6803a2f0 auth_method 0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.306+0000 7fbf637fe700 20 client.54244.objecter dump_active .. 0 homeless 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf7affd700 10 monclient(hunting): handle_auth_done global_id 35415 payload 824 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf7affd700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf7c09f760 con 0x7fbf7c06aac0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 client.54244.objecter ms_handle_connect 0x7fbf7c06aac0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf7affd700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 0x7fbf7c1208d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0119d0 tx=0x7fbf6c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 client.54244.objecter resend_mon_ops 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf7b7fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbf6803a2f0 0x7fbf6803c7b0 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 crypto rx=0x7fbf740072e0 tx=0x7fbf7400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 client.54244.objecter ms_handle_connect 0x7fbf6803a2f0 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf6c0036a0 con 0x7fbf7c06aac0 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 monclient: dump: 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.333 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbf6c004240 con 0x7fbf7c06aac0 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 msgr2=0x7fbf7c1208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 0x7fbf7c1208d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0119d0 tx=0x7fbf6c011d90 comp rx=0 tx=0).stop 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 msgr2=0x7fbf7c1208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.307+0000 7fbf637fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c06aac0 0x7fbf7c1208d0 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0119d0 tx=0x7fbf6c011d90 comp rx=0 tx=0).stop 2024-04-20T02:22:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf817dc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:39.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf817dc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 0x7fbf7c121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf817dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c06a400 con 0x7fbf7c120e10 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 0x7fbf7c121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient: get_auth_request con 0x7fbf7c120e10 auth_method 0 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 10 monclient(hunting): handle_auth_done global_id 35424 payload 824 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.315+0000 7fbf7affd700 1 -- 172.21.15.47:0/1856739583 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbf7c121760 con 0x7fbf7c120e10 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf7affd700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 0x7fbf7c121220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c006b20 tx=0x7fbf6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 client.54244.objecter ms_handle_connect 0x7fbf7c120e10 2024-04-20T02:22:39.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 client.54244.objecter resend_mon_ops 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf6c00fc60 con 0x7fbf7c120e10 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 monclient: dump: 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.338 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7fbf6c00fdc0 con 0x7fbf7c120e10 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 msgr2=0x7fbf7c121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 0x7fbf7c121220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c006b20 tx=0x7fbf6c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 msgr2=0x7fbf7c121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf637fe700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c120e10 0x7fbf7c121220 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c006b20 tx=0x7fbf6c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.339 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:39.340 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.341 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.342 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:39.343 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.344 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:39.345 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.347 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:39.350 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.352 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:39.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.355 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:39.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.358 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.359 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.360 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.361 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:39.361 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 10 librados: watch_flush enter 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 10 librados: watch_flush exit 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 20 client.54244.objecter shutdown clearing up homeless session... 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 10 client.54244.objecter successfully canceled tick 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbf6803a2f0 msgr2=0x7fbf6803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbf6803a2f0 0x7fbf6803c7b0 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 crypto rx=0x7fbf740072e0 tx=0x7fbf7400b040 comp rx=0 tx=0).stop 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 10 monclient: shutdown 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c138170 msgr2=0x7fbf7c136d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.316+0000 7fbf817dc700 1 --2- 172.21.15.47:0/1856739583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf7c138170 0x7fbf7c136d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbf70013000 tx=0x7fbf70004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf7b7fe700 1 -- 172.21.15.47:0/1856739583 reap_dead start 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf7b7fe700 1 -- 172.21.15.47:0/1856739583 reap_dead start 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 shutdown_connections 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 >> 172.21.15.47:0/1856739583 conn(0x7fbf7c0aa8b0 msgr2=0x7fbf7c0a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 shutdown_connections 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf817dc700 1 -- 172.21.15.47:0/1856739583 wait complete. 2024-04-20T02:22:39.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.317+0000 7fbf817dc700 1 librados: shutdown 2024-04-20T02:22:39.364 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:39.364 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:22:39.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:22:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.505+0000 7f0228a0e700 10 monclient: get_monmap_and_config 2024-04-20T02:22:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.505+0000 7f0228a0e700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.505+0000 7f0228a0e700 10 monclient: monmap: 2024-04-20T02:22:39.505 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:39.506928+0000 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:39.506928+0000 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.506 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:39.507 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:39.507 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.506+0000 7f0228a0e700 1 Processor -- start 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.506+0000 7f0228a0e700 1 -- start start 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.506+0000 7f0228a0e700 10 monclient: init 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient: _add_conns ranks=[0,8,7,3,10,4,9,2,6,1,5] 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f02240a19d0 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f02240a1db0 0x7f022413ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): picked mon.noname-e con 0x7f02240a1db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f022413b150 0x7f022413d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): picked mon.noname-i con 0x7f022413b150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02240751e0 con 0x7f02240a19d0 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0224136610 con 0x7f022413b150 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f02240a2230 con 0x7f02240a1db0 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0228a0e700 10 monclient(hunting): authenticate will time out at 2038.908228s 2024-04-20T02:22:39.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f022259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f02240a19d0 0x7f0224135f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46338/0 (socket says 172.21.15.47:46338) 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f02240a1db0 0x7f022413ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f022259c700 1 -- 172.21.15.47:0/1124493054 learned_addr learned my addr 172.21.15.47:0/1124493054 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): get_auth_request con 0x7f02240a1db0 auth_method 0 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0222d9d700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f022413b150 0x7f022413d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0222d9d700 10 monclient(hunting): get_auth_request con 0x7f022413b150 auth_method 0 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0222d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0222d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0222d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient(hunting): handle_auth_done global_id 35090 payload 274 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f022159a700 1 -- 172.21.15.47:0/1124493054 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521915894 0 0) 0x7f02240751e0 con 0x7f02240a19d0 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 -- 172.21.15.47:0/1124493054 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f022413b150 msgr2=0x7f022413d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f022413b150 0x7f022413d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 -- 172.21.15.47:0/1124493054 >> v1:172.21.15.47:6789/0 conn(0x7f02240a19d0 legacy=0x7f0224135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 monclient: found mon.noname-e 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 -- 172.21.15.47:0/1124493054 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02240a4040 con 0x7f02240a1db0 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f022159a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0222d9d700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f022413b150 0x7f022413d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.507+0000 7f0221d9b700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f02240a1db0 0x7f022413ac10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f020c009870 tx=0x7f020c009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d642b0da0fca1948 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 1 -- 172.21.15.47:0/1124493054 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f020c011aa0 con 0x7f02240a1db0 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: dump: 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 5 monclient: authenticate success, global_id 35090 2024-04-20T02:22:39.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 1 -- 172.21.15.47:0/1124493054 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c011c00 con 0x7f02240a1db0 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 4 set_mon_vals no callback set 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 1 -- 172.21.15.47:0/1124493054 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f020c011d70 con 0x7f02240a1db0 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f022159a700 10 monclient: dump: 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.523 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 10 monclient: shutdown 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 -- 172.21.15.47:0/1124493054 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f02240a1db0 msgr2=0x7f022413ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f02240a1db0 0x7f022413ac10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f020c009870 tx=0x7f020c009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 -- 172.21.15.47:0/1124493054 shutdown_connections 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f022413b150 0x7f022413d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 --2- 172.21.15.47:0/1124493054 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f02240a1db0 0x7f022413ac10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 -- 172.21.15.47:0/1124493054 >> 172.21.15.47:0/1124493054 conn(0x7f02240a4a50 msgr2=0x7f02240a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 -- 172.21.15.47:0/1124493054 shutdown_connections 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.508+0000 7f0228a0e700 1 -- 172.21.15.47:0/1124493054 wait complete. 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.509+0000 7f0228a0e700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.509+0000 7f0228a0e700 10 monclient: monmap: 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:39.525 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.509+0000 7f0228a0e700 1 librados: starting msgr at 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.509+0000 7f0228a0e700 1 librados: starting objecter 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 Processor -- start 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 -- start start 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 librados: setting wanted keys 2024-04-20T02:22:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 librados: calling monclient init 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient: init 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient: _add_conns ranks=[7,1,3,2,8,5,0,6,4] 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f02240a19d0 0x7f022409db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): picked mon.noname-i con 0x7f02240a19d0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02240a1db0 0x7f0224134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): picked mon.noname-f con 0x7f02240a1db0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f022413b150 0x7f0224135210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): picked mon.noname-g con 0x7f022413b150 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0224071680 con 0x7f02240a1db0 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02241cb5c0 con 0x7f022413b150 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f022409e820 con 0x7f02240a19d0 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0228a0e700 10 monclient(hunting): authenticate will time out at 2038.911611s 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f022259c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f02240a19d0 0x7f022409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0221d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02240a1db0 0x7f0224134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.510+0000 7f0222d9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f022413b150 0x7f0224135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f02240a19d0 0x7f022409db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:47672/0 (socket says 172.21.15.47:47672) 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02240a1db0 0x7f0224134cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:51258/0 (socket says 172.21.15.47:51258) 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 1 -- 172.21.15.47:0/3855318095 learned_addr learned my addr 172.21.15.47:0/3855318095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): get_auth_request con 0x7f022413b150 auth_method 0 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): get_auth_request con 0x7f02240a19d0 auth_method 0 2024-04-20T02:22:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): get_auth_request con 0x7f02240a1db0 auth_method 0 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient(hunting): handle_auth_done global_id 22017 payload 824 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f02240a19d0 msgr2=0x7f022409db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f02240a19d0 0x7f022409db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02240a1db0 msgr2=0x7f0224134cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02240a1db0 0x7f0224134cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 monclient: found mon.noname-g 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0094a0 con 0x7f022413b150 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f022259c700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f02240a19d0 0x7f022409db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0221d9b700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02240a1db0 0x7f0224134cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0222d9d700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f022413b150 0x7f0224135210 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f02241369b0 tx=0x7f021c00d0d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 client.?.objecter ms_handle_connect 0x7f022413b150 2024-04-20T02:22:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f021c00ddd0 con 0x7f022413b150 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: dump: 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c00f040 con 0x7f022413b150 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 5 monclient: authenticate success, global_id 22017 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 10 monclient: _renew_subs 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0224071230 con 0x7f022413b150 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f021c014810 con 0x7f022413b150 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0220d99700 4 set_mon_vals no callback set 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f021affd700 10 monclient: dump: 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.533 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 10 client.22017.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 10 monclient: _renew_subs 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02241210c0 con 0x7f022413b150 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.511+0000 7f0228a0e700 1 librados: init done 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f021c0143a0 con 0x7f022413b150 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f0228a0e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f0228a0e700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 0x7f022411e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0224121250 con 0x7f0224120230 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f021affd700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f020403a2f0 0x7f020403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(301..301 src has 1..301) v4 ==== 6645+0+0 (secure 0 0 0) 0x7f021c04a4d0 con 0x7f022413b150 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f021affd700 10 client.22017.objecter ms_dispatch 0x7f0224001550 osd_map(301..301 src has 1..301) v4 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f021affd700 3 client.22017.objecter handle_osd_map got epochs [301,301] > 0 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.512+0000 7f021affd700 3 client.22017.objecter handle_osd_map decoding full epoch 301 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 0x7f022411e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient: get_auth_request con 0x7f0224120230 auth_method 0 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f0221d9b700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f020403a2f0 0x7f020403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f0221d9b700 10 monclient: get_auth_request con 0x7f020403a2f0 auth_method 0 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 20 client.22017.objecter dump_active .. 0 homeless 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 10 monclient(hunting): handle_auth_done global_id 23897 payload 824 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0224072300 con 0x7f0224120230 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f022259c700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 0x7f022411e5a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0214007300 tx=0x7f02140076c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 10 client.22017.objecter ms_handle_connect 0x7f0224120230 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 10 client.22017.objecter resend_mon_ops 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 10 client.22017.objecter ms_handle_connect 0x7f020403a2f0 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f0221d9b700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f020403a2f0 0x7f020403c7b0 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 crypto rx=0x7f020c009870 tx=0x7f020c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f021400fca0 con 0x7f0224120230 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.513+0000 7f021affd700 10 monclient: dump: 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f021400fe00 con 0x7f0224120230 2024-04-20T02:22:39.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 msgr2=0x7f022411e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 0x7f022411e5a0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0214007300 tx=0x7f02140076c0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 msgr2=0x7f022411e5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.514+0000 7f021affd700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0224120230 0x7f022411e5a0 secure :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0214007300 tx=0x7f02140076c0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.521+0000 7f0228a0e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.521+0000 7f0228a0e700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 0x7f022411eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.521+0000 7f0228a0e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.521+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0224068430 con 0x7f022406aac0 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 0x7f022411eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient: get_auth_request con 0x7f022406aac0 auth_method 0 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 10 monclient(hunting): handle_auth_done global_id 23906 payload 824 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 1 -- 172.21.15.47:0/3855318095 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f02240a4480 con 0x7f022406aac0 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f022259c700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 0x7f022411eae0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0214007300 tx=0x7f0214000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 client.22017.objecter ms_handle_connect 0x7f022406aac0 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 client.22017.objecter resend_mon_ops 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0214022070 con 0x7f022406aac0 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 monclient: dump: 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.541 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f02140136c0 con 0x7f022406aac0 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:39.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 msgr2=0x7f022411eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 0x7f022411eae0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0214007300 tx=0x7f0214000ed0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 msgr2=0x7f022411eae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.522+0000 7f021affd700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f022406aac0 0x7f022411eae0 secure :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0214007300 tx=0x7f0214000ed0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:39.543 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.544 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.545 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:39.546 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.549 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.551 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:39.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.555 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:39.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.558 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.561 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.562 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:39.563 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:39.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 10 librados: watch_flush enter 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 10 librados: watch_flush exit 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 20 client.22017.objecter shutdown clearing up homeless session... 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 10 client.22017.objecter successfully canceled tick 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f020403a2f0 msgr2=0x7f020403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f020403a2f0 0x7f020403c7b0 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 crypto rx=0x7f020c009870 tx=0x7f020c008030 comp rx=0 tx=0).stop 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 10 monclient: shutdown 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f022413b150 msgr2=0x7f0224135210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0222d9d700 1 -- 172.21.15.47:0/3855318095 reap_dead start 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f022413b150 0x7f0224135210 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f02241369b0 tx=0x7f021c00d0d0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 shutdown_connections 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 --2- 172.21.15.47:0/3855318095 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f022413b150 0x7f0224135210 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 >> 172.21.15.47:0/3855318095 conn(0x7f02240a4a50 msgr2=0x7f022409f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 shutdown_connections 2024-04-20T02:22:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 -- 172.21.15.47:0/3855318095 wait complete. 2024-04-20T02:22:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.523+0000 7f0228a0e700 1 librados: shutdown 2024-04-20T02:22:39.567 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:39.567 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:22:39.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:22:39.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.705+0000 7f584db8d700 10 monclient: get_monmap_and_config 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.705+0000 7f584db8d700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.705+0000 7f584db8d700 10 monclient: monmap: 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:39.706905+0000 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:39.706905+0000 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.706 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:39.707 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:39.707 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.706+0000 7f584db8d700 1 Processor -- start 2024-04-20T02:22:39.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.706+0000 7f584db8d700 1 -- start start 2024-04-20T02:22:39.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.706+0000 7f584db8d700 10 monclient: init 2024-04-20T02:22:39.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient: _add_conns ranks=[4,7,6,1,5,3,10,8,9,2,0] 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 0x7f58480a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): picked mon.noname-c con 0x7f58480a07f0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58480a0bd0 0x7f58480a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): picked mon.noname-i con 0x7f58480a0bd0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 0x7f58480a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): picked mon.noname-d con 0x7f58480a1050 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5848067800 con 0x7f58480a07f0 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f58480672f0 con 0x7f58480a1050 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f584809ed90 con 0x7f58480a0bd0 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f584db8d700 10 monclient(hunting): authenticate will time out at 2039.108177s 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 0x7f58480a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 0x7f58480a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 0x7f58480a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35954/0 (socket says 172.21.15.47:35954) 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 1 -- 172.21.15.47:0/2305640430 learned_addr learned my addr 172.21.15.47:0/2305640430 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): get_auth_request con 0x7f58480a07f0 auth_method 0 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 0x7f58480a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48224/0 (socket says 172.21.15.47:48224) 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): get_auth_request con 0x7f58480a1050 auth_method 0 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58480a0bd0 0x7f58480a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): get_auth_request con 0x7f58480a0bd0 auth_method 0 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5846ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient(hunting): handle_auth_done global_id 54222 payload 274 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 -- 172.21.15.47:0/2305640430 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58480a0bd0 msgr2=0x7f58480a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58480a0bd0 0x7f58480a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 -- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 msgr2=0x7f58480a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 0x7f58480a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 monclient: found mon.noname-d 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 -- 172.21.15.47:0/2305640430 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480751e0 con 0x7f58480a1050 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f58477fe700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 0x7f58480a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.707+0000 7f5847fff700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 0x7f58480a2db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5840009dc0 tx=0x7f584000b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=612f0d772b7f7554 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 1 -- 172.21.15.47:0/2305640430 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f584000e070 con 0x7f58480a1050 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: dump: 2024-04-20T02:22:39.722 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 1 -- 172.21.15.47:0/2305640430 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5840012430 con 0x7f58480a1050 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 5 monclient: authenticate success, global_id 54222 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 1 -- 172.21.15.47:0/2305640430 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5840012a10 con 0x7f58480a1050 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 4 set_mon_vals no callback set 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f58467fc700 10 monclient: dump: 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.724 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 10 monclient: shutdown 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 -- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 msgr2=0x7f58480a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 0x7f58480a2db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5840009dc0 tx=0x7f584000b330 comp rx=0 tx=0).stop 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 -- 172.21.15.47:0/2305640430 shutdown_connections 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f58480a1050 0x7f58480a2db0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58480a07f0 0x7f58480a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 --2- 172.21.15.47:0/2305640430 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58480a0bd0 0x7f58480a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 -- 172.21.15.47:0/2305640430 >> 172.21.15.47:0/2305640430 conn(0x7f58480a3f90 msgr2=0x7f584806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 -- 172.21.15.47:0/2305640430 shutdown_connections 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.708+0000 7f584db8d700 1 -- 172.21.15.47:0/2305640430 wait complete. 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.709+0000 7f584db8d700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.709+0000 7f584db8d700 10 monclient: monmap: 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:39.726 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.709+0000 7f584db8d700 1 librados: starting msgr at 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.709+0000 7f584db8d700 1 librados: starting objecter 2024-04-20T02:22:39.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.709+0000 7f584db8d700 1 Processor -- start 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 -- start start 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 librados: setting wanted keys 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 librados: calling monclient init 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient: init 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient: _add_conns ranks=[1,0,2,4,8,7,6,5,3] 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480a07f0 0x7f58481cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): picked mon.noname-f con 0x7f58480a07f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 0x7f58481bba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): picked mon.noname-a con 0x7f58480a0bd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f58480a1050 0x7f58481bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): picked mon.noname-b con 0x7f58480a1050 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58481aacb0 con 0x7f58480a0bd0 2024-04-20T02:22:39.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58481ab1e0 con 0x7f58480a07f0 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58481ab450 con 0x7f58480a1050 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f584db8d700 10 monclient(hunting): authenticate will time out at 2039.111668s 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 0x7f58481bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f58480a1050 0x7f58481bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 0x7f58481bba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60952/0 (socket says 172.21.15.47:60952) 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f58480a1050 0x7f58481bbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60872/0 (socket says 172.21.15.47:60872) 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 1 -- 172.21.15.47:0/3776573809 learned_addr learned my addr 172.21.15.47:0/3776573809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): get_auth_request con 0x7f58480a1050 auth_method 0 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): get_auth_request con 0x7f58480a0bd0 auth_method 0 2024-04-20T02:22:39.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480a07f0 0x7f58481cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 10 monclient(hunting): get_auth_request con 0x7f58480a07f0 auth_method 0 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5846ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f5847fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.710+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 10 monclient(hunting): handle_auth_done global_id 35442 payload 824 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f58480a1050 msgr2=0x7f58481bbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f58480a1050 0x7f58481bbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480a07f0 msgr2=0x7f58481cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480a07f0 0x7f58481cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 monclient: found mon.noname-a 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009550 con 0x7f58480a0bd0 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5847fff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f58480a1050 0x7f58481bbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f58477fe700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58480a07f0 0x7f58481cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5846ffd700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 0x7f58481bba50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f583c00e9b0 tx=0x7f583c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 client.?.objecter ms_handle_connect 0x7f58480a0bd0 2024-04-20T02:22:39.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583c009710 con 0x7f58480a0bd0 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: dump: 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.732 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 5 monclient: authenticate success, global_id 35442 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00ee50 con 0x7f58480a0bd0 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 10 monclient: _renew_subs 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583c00ccc0 con 0x7f58480a0bd0 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5840009a70 con 0x7f58480a0bd0 2024-04-20T02:22:39.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f5845ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f582ffff700 10 monclient: dump: 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.734 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 10 client.35442.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 10 monclient: _renew_subs 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584809e360 con 0x7f58480a0bd0 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.711+0000 7f584db8d700 1 librados: init done 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f584db8d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f584db8d700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 0x7f58481a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58481ab5e0 con 0x7f584806aac0 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 0x7f58481a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient: get_auth_request con 0x7f584806aac0 auth_method 0 2024-04-20T02:22:39.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f583c00c850 con 0x7f58480a0bd0 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 10 monclient(hunting): handle_auth_done global_id 54193 payload 824 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f58477fe700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f584809f350 con 0x7f584806aac0 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.712+0000 7f582ffff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a320 0x7f583403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(301..301 src has 1..301) v4 ==== 6645+0+0 (secure 0 0 0) 0x7f583c04a8b0 con 0x7f58480a0bd0 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 client.35442.objecter ms_dispatch 0x7f5848001550 osd_map(301..301 src has 1..301) v4 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 3 client.35442.objecter handle_osd_map got epochs [301,301] > 0 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 3 client.35442.objecter handle_osd_map decoding full epoch 301 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f58477fe700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 0x7f58481a7520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5838011f00 tx=0x7f583800f520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 20 client.35442.objecter dump_active .. 0 homeless 2024-04-20T02:22:39.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 client.35442.objecter ms_handle_connect 0x7f584806aac0 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 client.35442.objecter resend_mon_ops 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f5847fff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a320 0x7f583403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f583800fde0 con 0x7f584806aac0 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 monclient: dump: 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f5847fff700 10 monclient: get_auth_request con 0x7f583403a320 auth_method 0 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5838014440 con 0x7f584806aac0 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 msgr2=0x7f58481a7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 0x7f58481a7520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5838011f00 tx=0x7f583800f520 comp rx=0 tx=0).stop 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 msgr2=0x7f58481a7520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f584806aac0 0x7f58481a7520 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5838011f00 tx=0x7f583800f520 comp rx=0 tx=0).stop 2024-04-20T02:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f5847fff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a320 0x7f583403c7e0 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f5840014040 tx=0x7f5840011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.713+0000 7f582ffff700 10 client.35442.objecter ms_handle_connect 0x7f583403a320 2024-04-20T02:22:39.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f584db8d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:39.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f584db8d700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 0x7f58481a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f584db8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5848068280 con 0x7f58481a7a60 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 0x7f58481a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient: get_auth_request con 0x7f58481a7a60 auth_method 0 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 10 monclient(hunting): handle_auth_done global_id 54202 payload 824 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 1 -- 172.21.15.47:0/3776573809 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f584806d160 con 0x7f58481a7a60 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f582ffff700 10 client.35442.objecter ms_handle_connect 0x7f58481a7a60 2024-04-20T02:22:39.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f582ffff700 10 client.35442.objecter resend_mon_ops 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f58477fe700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 0x7f58481a7e70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5838000c00 tx=0x7f5838004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5838004490 con 0x7f58481a7a60 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f582ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f582ffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.721+0000 7f582ffff700 10 monclient: dump: 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.742 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f58380227f0 con 0x7f58481a7a60 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 msgr2=0x7f58481a7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 0x7f58481a7e70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5838000c00 tx=0x7f5838004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 msgr2=0x7f58481a7e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f582ffff700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f58481a7a60 0x7f58481a7e70 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5838000c00 tx=0x7f5838004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:39.744 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.745 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.746 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:39.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.748 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.750 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.753 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:39.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.762 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.763 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:39.764 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 10 librados: watch_flush enter 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 10 librados: watch_flush exit 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 20 client.35442.objecter shutdown clearing up homeless session... 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 10 client.35442.objecter successfully canceled tick 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a320 msgr2=0x7f583403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f583403a320 0x7f583403c7e0 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 crypto rx=0x7f5840014040 tx=0x7f5840011000 comp rx=0 tx=0).stop 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 10 monclient: shutdown 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 msgr2=0x7f58481bba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 0x7f58481bba50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f583c00e9b0 tx=0x7f583c00ecc0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f5847fff700 1 -- 172.21.15.47:0/3776573809 reap_dead start 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 shutdown_connections 2024-04-20T02:22:39.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 --2- 172.21.15.47:0/3776573809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58480a0bd0 0x7f58481bba50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 >> 172.21.15.47:0/3776573809 conn(0x7f58480a3f90 msgr2=0x7f58481be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 shutdown_connections 2024-04-20T02:22:39.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 -- 172.21.15.47:0/3776573809 wait complete. 2024-04-20T02:22:39.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.722+0000 7f584db8d700 1 librados: shutdown 2024-04-20T02:22:39.767 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:39.767 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:22:39.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:22:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.904+0000 7fb5bc676700 10 monclient: get_monmap_and_config 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.904+0000 7fb5bc676700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.904+0000 7fb5bc676700 10 monclient: monmap: 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:39.905919+0000 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:39.905919+0000 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.906 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.906 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.906 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:39.906 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:39.906 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.905+0000 7fb5bc676700 1 Processor -- start 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.905+0000 7fb5bc676700 1 -- start start 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.905+0000 7fb5bc676700 10 monclient: init 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient: _add_conns ranks=[5,6,9,0,3,8,7,2,1,10,4] 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb5b4135ad0 0x7fb5b409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): picked mon.noname-h con 0x7fb5b4135ad0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b4135eb0 0x7fb5b41362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): picked mon.noname-d con 0x7fb5b4135eb0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5b40a2950 0x7fb5b40a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): picked mon.noname-a con 0x7fb5b40a2950 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b409bc20 con 0x7fb5b40a2950 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4067db0 con 0x7fb5b4135ad0 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40a0ae0 con 0x7fb5b4135eb0 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bc676700 10 monclient(hunting): authenticate will time out at 2039.307214s 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5b40a2950 0x7fb5b40a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b4135eb0 0x7fb5b41362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb5b40a2950 0x7fb5b40a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60964/0 (socket says 172.21.15.47:60964) 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 -- 172.21.15.47:0/2084649671 learned_addr learned my addr 172.21.15.47:0/2084649671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): get_auth_request con 0x7fb5b40a2950 auth_method 0 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): get_auth_request con 0x7fb5b4135eb0 auth_method 0 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5ba412700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb5b4135ad0 0x7fb5b409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5ba412700 10 monclient(hunting): get_auth_request con 0x7fb5b4135ad0 auth_method 0 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5ba412700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5ba412700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5ba412700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient(hunting): handle_auth_done global_id 35451 payload 274 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 -- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b4135eb0 msgr2=0x7fb5b41362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b4135eb0 0x7fb5b41362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 -- 172.21.15.47:0/2084649671 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb5b4135ad0 msgr2=0x7fb5b409b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb5b4135ad0 0x7fb5b409b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 monclient: found mon.noname-a 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 -- 172.21.15.47:0/2084649671 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40751e0 con 0x7fb5b40a2950 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5b9c11700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b4135eb0 0x7fb5b41362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.906+0000 7fb5bac13700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b40a2dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009e80 tx=0x7fb5ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b466edcfd174f45d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 1 -- 172.21.15.47:0/2084649671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5ac00e070 con 0x7fb5b40a2950 2024-04-20T02:22:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: dump: 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 1 -- 172.21.15.47:0/2084649671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac00bd40 con 0x7fb5b40a2950 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 5 monclient: authenticate success, global_id 35451 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 1 -- 172.21.15.47:0/2084649671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5ac012860 con 0x7fb5b40a2950 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 4 set_mon_vals no callback set 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5b9410700 10 monclient: dump: 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 10 monclient: shutdown 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 -- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 msgr2=0x7fb5b40a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b40a2dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009e80 tx=0x7fb5ac00b330 comp rx=0 tx=0).stop 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 -- 172.21.15.47:0/2084649671 shutdown_connections 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b40a2dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb5b4135ad0 0x7fb5b409b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 --2- 172.21.15.47:0/2084649671 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b4135eb0 0x7fb5b41362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 -- 172.21.15.47:0/2084649671 >> 172.21.15.47:0/2084649671 conn(0x7fb5b40a7570 msgr2=0x7fb5b40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 -- 172.21.15.47:0/2084649671 shutdown_connections 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.907+0000 7fb5bc676700 1 -- 172.21.15.47:0/2084649671 wait complete. 2024-04-20T02:22:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.908+0000 7fb5bc676700 10 monclient: build_initial_monmap 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.908+0000 7fb5bc676700 10 monclient: monmap: 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 librados: starting msgr at 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 librados: starting objecter 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 Processor -- start 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 -- start start 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 librados: setting wanted keys 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 librados: calling monclient init 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 10 monclient: init 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 10 monclient: _add_conns ranks=[0,3,1,2,4,7,8,5,6] 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b41ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.909+0000 7fb5bc676700 10 monclient(hunting): picked mon.noname-a con 0x7fb5b40a2950 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5b4135ad0 0x7fb5b411f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): picked mon.noname-g con 0x7fb5b4135ad0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5b4135eb0 0x7fb5b411f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): picked mon.noname-f con 0x7fb5b4135eb0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4121710 con 0x7fb5b40a2950 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4122600 con 0x7fb5b4135eb0 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4066ea0 con 0x7fb5b4135ad0 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient(hunting): authenticate will time out at 2039.311074s 2024-04-20T02:22:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b41ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b41ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60968/0 (socket says 172.21.15.47:60968) 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 -- 172.21.15.47:0/65471978 learned_addr learned my addr 172.21.15.47:0/65471978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): get_auth_request con 0x7fb5b40a2950 auth_method 0 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bac13700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5b4135eb0 0x7fb5b411f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5b9c11700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5b4135ad0 0x7fb5b411f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bac13700 10 monclient(hunting): get_auth_request con 0x7fb5b4135eb0 auth_method 0 2024-04-20T02:22:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bac13700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bac13700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bac13700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient(hunting): handle_auth_done global_id 35460 payload 824 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5b4135ad0 msgr2=0x7fb5b411f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb5b4135ad0 0x7fb5b411f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5b4135eb0 msgr2=0x7fb5b411f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5b4135eb0 0x7fb5b411f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient: _finish_hunting 0 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 monclient: found mon.noname-a 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a40094a0 con 0x7fb5b40a2950 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5ba412700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b41ca6a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000e840 tx=0x7fb5b000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 client.?.objecter ms_handle_connect 0x7fb5b40a2950 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5b0013880 con 0x7fb5b40a2950 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: dump: 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 5 monclient: authenticate success, global_id 35460 2024-04-20T02:22:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b0013a00 con 0x7fb5b40a2950 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient: _renew_subs 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac009b30 con 0x7fb5b40a2950 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5b0012370 con 0x7fb5b40a2950 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 client.35460.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5aaffd700 10 monclient: dump: 2024-04-20T02:22:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.935 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5b8c0f700 4 set_mon_vals no callback set 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient: _renew_subs 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b4122ea0 con 0x7fb5b40a2950 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.910+0000 7fb5bc676700 1 librados: init done 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5bc676700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5bc676700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 0x7fb5b411fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4068310 con 0x7fb5b41237f0 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb5b0013410 con 0x7fb5b40a2950 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 0x7fb5b411fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient: get_auth_request con 0x7fb5b41237f0 auth_method 0 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.911+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb59803a2f0 0x7fb59803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_done global_id 54240 payload 824 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5b9c11700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5b4067a40 con 0x7fb5b41237f0 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(301..301 src has 1..301) v4 ==== 6645+0+0 (secure 0 0 0) 0x7fb5b004a0d0 con 0x7fb5b40a2950 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 client.35460.objecter ms_dispatch 0x7fb5b4001550 osd_map(301..301 src has 1..301) v4 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 3 client.35460.objecter handle_osd_map got epochs [301,301] > 0 2024-04-20T02:22:39.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 3 client.35460.objecter handle_osd_map decoding full epoch 301 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5bac13700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb59803a2f0 0x7fb59803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5bac13700 10 monclient: get_auth_request con 0x7fb59803a2f0 auth_method 0 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5b9c11700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 0x7fb5b411fab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a0880 tx=0x7fb5a4010b90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 20 client.35460.objecter dump_active .. 0 homeless 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 client.35460.objecter ms_handle_connect 0x7fb5b41237f0 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 client.35460.objecter resend_mon_ops 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5a4014a70 con 0x7fb5b41237f0 2024-04-20T02:22:39.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 monclient: dump: 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.939 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 client.35460.objecter ms_handle_connect 0x7fb59803a2f0 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5bac13700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb59803a2f0 0x7fb59803c7b0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009e80 tx=0x7fb5ac009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb5a4014bd0 con 0x7fb5b41237f0 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 msgr2=0x7fb5b411fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 0x7fb5b411fab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a0880 tx=0x7fb5a4010b90 comp rx=0 tx=0).stop 2024-04-20T02:22:39.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 msgr2=0x7fb5b411fab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.912+0000 7fb5aaffd700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b41237f0 0x7fb5b411fab0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a0880 tx=0x7fb5a4010b90 comp rx=0 tx=0).stop 2024-04-20T02:22:39.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5bc676700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5bc676700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 0x7fb5b411fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5bc676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b406a400 con 0x7fb5b406aac0 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 0x7fb5b411fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient: get_auth_request con 0x7fb5b406aac0 auth_method 0 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:39.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.920+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5b9c11700 10 monclient(hunting): handle_auth_done global_id 54249 payload 824 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5b9c11700 1 -- 172.21.15.47:0/65471978 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5b409a9d0 con 0x7fb5b406aac0 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5b9c11700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 0x7fb5b411fff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a0880 tx=0x7fb5a4004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 client.35460.objecter ms_handle_connect 0x7fb5b406aac0 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 client.35460.objecter resend_mon_ops 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5a4021c80 con 0x7fb5b406aac0 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 monclient: dump: 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:39.944 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fb5a4021de0 con 0x7fb5b406aac0 2024-04-20T02:22:39.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 msgr2=0x7fb5b411fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 0x7fb5b411fff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a0880 tx=0x7fb5a4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 msgr2=0x7fb5b411fff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5aaffd700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb5b406aac0 0x7fb5b411fff0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40a0880 tx=0x7fb5a4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:39.946 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:39.947 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.948 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:39.950 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:39.951 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:39.953 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:39.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.962 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.963 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:39.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:39.965 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.966 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:39.967 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5bc676700 10 librados: watch_flush enter 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5bc676700 10 librados: watch_flush exit 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.921+0000 7fb5bc676700 20 client.35460.objecter shutdown clearing up homeless session... 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 10 client.35460.objecter successfully canceled tick 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb59803a2f0 msgr2=0x7fb59803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb59803a2f0 0x7fb59803c7b0 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009e80 tx=0x7fb5ac009000 comp rx=0 tx=0).stop 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 10 monclient: shutdown 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 msgr2=0x7fb5b41ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 --2- 172.21.15.47:0/65471978 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5b40a2950 0x7fb5b41ca6a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000e840 tx=0x7fb5b000ec00 comp rx=0 tx=0).stop 2024-04-20T02:22:39.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bac13700 1 -- 172.21.15.47:0/65471978 reap_dead start 2024-04-20T02:22:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bac13700 1 -- 172.21.15.47:0/65471978 reap_dead start 2024-04-20T02:22:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 shutdown_connections 2024-04-20T02:22:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 >> 172.21.15.47:0/65471978 conn(0x7fb5b40a7570 msgr2=0x7fb5b40a5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 shutdown_connections 2024-04-20T02:22:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 -- 172.21.15.47:0/65471978 wait complete. 2024-04-20T02:22:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:39.922+0000 7fb5bc676700 1 librados: shutdown 2024-04-20T02:22:39.971 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:39.971 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:22:39.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:22:40.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.116+0000 7f8e7e0d1700 10 monclient: get_monmap_and_config 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.116+0000 7f8e7e0d1700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.116+0000 7f8e7e0d1700 10 monclient: monmap: 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:40.117995+0000 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:40.117995+0000 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.117 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:40.118 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:40.118 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.117+0000 7f8e7e0d1700 1 Processor -- start 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.117+0000 7f8e7e0d1700 1 -- start start 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.117+0000 7f8e7e0d1700 10 monclient: init 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient: _add_conns ranks=[7,4,1,8,5,9,0,6,10,3,2] 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8e781341a0 0x7f8e780a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): picked mon.noname-i con 0x7f8e781341a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 0x7f8e78130900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): picked mon.noname-c con 0x7f8e78134580 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8e78134a00 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e780751e0 con 0x7f8e78134a00 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e780a0000 con 0x7f8e78134580 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7809f620 con 0x7f8e781341a0 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e7e0d1700 10 monclient(hunting): authenticate will time out at 2039.519234s 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 0x7f8e78130900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 0x7f8e78130900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35992/0 (socket says 172.21.15.47:35992) 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 -- 172.21.15.47:0/946988076 learned_addr learned my addr 172.21.15.47:0/946988076 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): get_auth_request con 0x7f8e78134580 auth_method 0 2024-04-20T02:22:40.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e777fe700 1 --2- 172.21.15.47:0/946988076 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8e781341a0 0x7f8e780a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e777fe700 10 monclient(hunting): get_auth_request con 0x7f8e781341a0 auth_method 0 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e777fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e777fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e777fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_done global_id 54211 payload 274 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 -- 172.21.15.47:0/946988076 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8e781341a0 msgr2=0x7f8e780a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/946988076 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8e781341a0 0x7f8e780a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 -- 172.21.15.47:0/946988076 >> v1:172.21.15.179:6789/0 conn(0x7f8e78134a00 legacy=0x7f8e78130e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 monclient: found mon.noname-c 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 -- 172.21.15.47:0/946988076 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7809f900 con 0x7f8e78134580 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.118+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/946988076 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 0x7f8e78130900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00bbc0 tx=0x7f8e6c00bed0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=5165c4621aea446b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 1 -- 172.21.15.47:0/946988076 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e6c017410 con 0x7f8e78134580 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: dump: 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.131 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 1 -- 172.21.15.47:0/946988076 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c017570 con 0x7f8e78134580 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 5 monclient: authenticate success, global_id 54211 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 1 -- 172.21.15.47:0/946988076 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e6c017b50 con 0x7f8e78134580 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 4 set_mon_vals no callback set 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e767fc700 10 monclient: dump: 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.133 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 10 monclient: shutdown 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/946988076 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 msgr2=0x7f8e78130900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/946988076 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 0x7f8e78130900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00bbc0 tx=0x7f8e6c00bed0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/946988076 shutdown_connections 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/946988076 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8e781341a0 0x7f8e780a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/946988076 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e78134580 0x7f8e78130900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/946988076 >> 172.21.15.47:0/946988076 conn(0x7f8e780a2db0 msgr2=0x7f8e780a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/946988076 shutdown_connections 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.119+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/946988076 wait complete. 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.120+0000 7f8e7e0d1700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.120+0000 7f8e7e0d1700 10 monclient: monmap: 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:40.135 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.120+0000 7f8e7e0d1700 1 librados: starting msgr at 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.120+0000 7f8e7e0d1700 1 librados: starting objecter 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.120+0000 7f8e7e0d1700 1 Processor -- start 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 -- start start 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 librados: setting wanted keys 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 librados: calling monclient init 2024-04-20T02:22:40.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient: init 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient: _add_conns ranks=[4,1,0,7,6,2,3,8,5] 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 0x7f8e781c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): picked mon.noname-c con 0x7f8e781341a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e78134580 0x7f8e781c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): picked mon.noname-f con 0x7f8e78134580 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78134a00 0x7f8e781c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): picked mon.noname-a con 0x7f8e78134a00 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7812c950 con 0x7f8e78134a00 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7812cc30 con 0x7f8e78134580 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7812cf10 con 0x7f8e781341a0 2024-04-20T02:22:40.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e7e0d1700 10 monclient(hunting): authenticate will time out at 2039.522658s 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 0x7f8e781c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 0x7f8e781c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:35994/0 (socket says 172.21.15.47:35994) 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 1 -- 172.21.15.47:0/3111594643 learned_addr learned my addr 172.21.15.47:0/3111594643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): get_auth_request con 0x7f8e781341a0 auth_method 0 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78134a00 0x7f8e781c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e78134580 0x7f8e781c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): get_auth_request con 0x7f8e78134a00 auth_method 0 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e777fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): get_auth_request con 0x7f8e78134580 auth_method 0 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e77fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.121+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 10 monclient(hunting): handle_auth_done global_id 54220 payload 824 2024-04-20T02:22:40.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e78134580 msgr2=0x7f8e781c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e78134580 0x7f8e781c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78134a00 msgr2=0x7f8e781c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e78134a00 0x7f8e781c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 monclient: found mon.noname-c 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c00b7f0 con 0x7f8e781341a0 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e777fe700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 0x7f8e781c81e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e810 tx=0x7f8e6800eb20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 client.?.objecter ms_handle_connect 0x7f8e781341a0 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e6800c070 con 0x7f8e781341a0 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: dump: 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.141 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 5 monclient: authenticate success, global_id 54220 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e68013880 con 0x7f8e781341a0 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 10 monclient: _renew_subs 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7809cda0 con 0x7f8e781341a0 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e68013e60 con 0x7f8e781341a0 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 10 client.54220.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e75ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e5ffff700 10 monclient: dump: 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 10 monclient: _renew_subs 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7812d980 con 0x7f8e781341a0 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.122+0000 7f8e7e0d1700 1 librados: init done 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e7e0d1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 0x7f8e78099090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7812db70 con 0x7f8e7806aac0 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8e680139f0 con 0x7f8e781341a0 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 0x7f8e78099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient: get_auth_request con 0x7f8e7806aac0 auth_method 0 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e6403a2f0 0x7f8e6403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(302..302 src has 1..302) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f8e6804a470 con 0x7f8e781341a0 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 10 client.54220.objecter ms_dispatch 0x7f8e78001550 osd_map(302..302 src has 1..302) v4 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 3 client.54220.objecter handle_osd_map got epochs [302,302] > 0 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 3 client.54220.objecter handle_osd_map decoding full epoch 302 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.123+0000 7f8e5ffff700 20 client.54220.objecter dump_active .. 0 homeless 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_done global_id 35099 payload 824 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e76ffd700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e7812cac0 con 0x7f8e7806aac0 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e77fff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e6403a2f0 0x7f8e6403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e77fff700 10 monclient: get_auth_request con 0x7f8e6403a2f0 auth_method 0 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 0x7f8e78099090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e7806e170 tx=0x7f8e6c004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 client.54220.objecter ms_handle_connect 0x7f8e7806aac0 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 client.54220.objecter resend_mon_ops 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e6c006c60 con 0x7f8e7806aac0 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 monclient: dump: 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.146 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.147 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8e6c00a040 con 0x7f8e7806aac0 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 msgr2=0x7f8e78099090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 0x7f8e78099090 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e7806e170 tx=0x7f8e6c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 msgr2=0x7f8e78099090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e7806aac0 0x7f8e78099090 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e7806e170 tx=0x7f8e6c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e77fff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e6403a2f0 0x7f8e6403c7b0 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 crypto rx=0x7f8e7812b780 tx=0x7f8e70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.124+0000 7f8e5ffff700 10 client.54220.objecter ms_handle_connect 0x7f8e6403a2f0 2024-04-20T02:22:40.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e7e0d1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:40.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 0x7f8e780999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e7e0d1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8e78067740 con 0x7f8e780995d0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 0x7f8e780999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient: get_auth_request con 0x7f8e780995d0 auth_method 0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 10 monclient(hunting): handle_auth_done global_id 35108 payload 824 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.132+0000 7f8e76ffd700 1 -- 172.21.15.47:0/3111594643 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8e7812cda0 con 0x7f8e780995d0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 client.54220.objecter ms_handle_connect 0x7f8e780995d0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 client.54220.objecter resend_mon_ops 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e76ffd700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 0x7f8e780999e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00bf10 tx=0x7f8e6c012000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e6c02c030 con 0x7f8e780995d0 2024-04-20T02:22:40.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 monclient: dump: 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.152 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f8e6c02c190 con 0x7f8e780995d0 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 msgr2=0x7f8e780999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 0x7f8e780999e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00bf10 tx=0x7f8e6c012000 comp rx=0 tx=0).stop 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 msgr2=0x7f8e780999e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e5ffff700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8e780995d0 0x7f8e780999e0 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00bf10 tx=0x7f8e6c012000 comp rx=0 tx=0).stop 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:40.154 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.155 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.158 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:40.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.163 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.165 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:40.167 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.170 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.173 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:40.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:40.175 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:40.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e7e0d1700 10 librados: watch_flush enter 2024-04-20T02:22:40.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e7e0d1700 10 librados: watch_flush exit 2024-04-20T02:22:40.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e7e0d1700 20 client.54220.objecter shutdown clearing up homeless session... 2024-04-20T02:22:40.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.133+0000 7f8e7e0d1700 10 client.54220.objecter successfully canceled tick 2024-04-20T02:22:40.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e6403a2f0 msgr2=0x7f8e6403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8e6403a2f0 0x7f8e6403c7b0 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 crypto rx=0x7f8e7812b780 tx=0x7f8e70009040 comp rx=0 tx=0).stop 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 10 monclient: shutdown 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 msgr2=0x7f8e781c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 0x7f8e781c81e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e810 tx=0x7f8e6800eb20 comp rx=0 tx=0).stop 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e77fff700 1 -- 172.21.15.47:0/3111594643 reap_dead start 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 shutdown_connections 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 --2- 172.21.15.47:0/3111594643 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8e781341a0 0x7f8e781c81e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 >> 172.21.15.47:0/3111594643 conn(0x7f8e780a2db0 msgr2=0x7f8e78130810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 shutdown_connections 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 -- 172.21.15.47:0/3111594643 wait complete. 2024-04-20T02:22:40.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.134+0000 7f8e7e0d1700 1 librados: shutdown 2024-04-20T02:22:40.179 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:40.179 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:22:40.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:22:40.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.317+0000 7f3e95958700 10 monclient: get_monmap_and_config 2024-04-20T02:22:40.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.317+0000 7f3e95958700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.317+0000 7f3e95958700 10 monclient: monmap: 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:40.318946+0000 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:40.318946+0000 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.318 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:40.319 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.318+0000 7f3e95958700 1 Processor -- start 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.318+0000 7f3e95958700 1 -- start start 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.318+0000 7f3e95958700 10 monclient: init 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.318+0000 7f3e95958700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.318+0000 7f3e95958700 10 monclient: _add_conns ranks=[6,7,5,3,4,9,0,8,10,2,1] 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 0x7f3e900a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): picked mon.noname-d con 0x7f3e90137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e900a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): picked mon.noname-i con 0x7f3e90138170 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3e901385b0 0x7f3e900a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): picked mon.noname-h con 0x7f3e901385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9009fa40 con 0x7f3e901385b0 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9009dfc0 con 0x7f3e90137d90 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900a7bc0 con 0x7f3e90138170 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e95958700 10 monclient(hunting): authenticate will time out at 2039.720128s 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 0x7f3e900a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 0x7f3e900a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48256/0 (socket says 172.21.15.47:48256) 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 -- 172.21.15.47:0/2050011306 learned_addr learned my addr 172.21.15.47:0/2050011306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): get_auth_request con 0x7f3e90137d90 auth_method 0 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8f7fe700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3e901385b0 0x7f3e900a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8e7fc700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e900a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8f7fe700 10 monclient(hunting): get_auth_request con 0x7f3e901385b0 auth_method 0 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8e7fc700 10 monclient(hunting): get_auth_request con 0x7f3e90138170 auth_method 0 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient(hunting): handle_auth_done global_id 54258 payload 274 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 -- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 msgr2=0x7f3e900a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e900a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 -- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3e901385b0 msgr2=0x7f3e900a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3e901385b0 0x7f3e900a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 monclient: found mon.noname-d 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 -- 172.21.15.47:0/2050011306 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900751e0 con 0x7f3e90137d90 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8effd700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 0x7f3e900a09e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009950 tx=0x7f3e80009c60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=21498c0f41eecec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8dffb700 1 -- 172.21.15.47:0/2050011306 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e80009ea0 con 0x7f3e90137d90 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8dffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.319+0000 7f3e8dffb700 10 monclient: dump: 2024-04-20T02:22:40.333 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.334 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 1 -- 172.21.15.47:0/2050011306 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8000bc90 con 0x7f3e90137d90 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 5 monclient: authenticate success, global_id 54258 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 1 -- 172.21.15.47:0/2050011306 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e8000be00 con 0x7f3e90137d90 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 4 set_mon_vals no callback set 2024-04-20T02:22:40.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e8dffb700 10 monclient: dump: 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.336 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 10 monclient: shutdown 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 -- 172.21.15.47:0/2050011306 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 msgr2=0x7f3e900a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 0x7f3e900a09e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009950 tx=0x7f3e80009c60 comp rx=0 tx=0).stop 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 -- 172.21.15.47:0/2050011306 shutdown_connections 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3e901385b0 0x7f3e900a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3e90137d90 0x7f3e900a09e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 --2- 172.21.15.47:0/2050011306 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e900a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 -- 172.21.15.47:0/2050011306 >> 172.21.15.47:0/2050011306 conn(0x7f3e900aa8b0 msgr2=0x7f3e900a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 -- 172.21.15.47:0/2050011306 shutdown_connections 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.320+0000 7f3e95958700 1 -- 172.21.15.47:0/2050011306 wait complete. 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 10 monclient: monmap: 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.338 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 1 librados: starting msgr at 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 1 librados: starting objecter 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 1 Processor -- start 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 1 -- start start 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 1 librados: setting wanted keys 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 1 librados: calling monclient init 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.321+0000 7f3e95958700 10 monclient: init 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient: _add_conns ranks=[1,7,3,8,4,6,5,2,0] 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90137d90 0x7f3e901cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): picked mon.noname-f con 0x7f3e90137d90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e90136d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): picked mon.noname-i con 0x7f3e90138170 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:40.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e901385b0 0x7f3e901372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): picked mon.noname-g con 0x7f3e901385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90123e80 con 0x7f3e90137d90 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901cc690 con 0x7f3e901385b0 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9009dc20 con 0x7f3e90138170 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e95958700 10 monclient(hunting): authenticate will time out at 2039.723471s 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90137d90 0x7f3e901cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90137d90 0x7f3e901cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:51294/0 (socket says 172.21.15.47:51294) 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 1 -- 172.21.15.47:0/2328631632 learned_addr learned my addr 172.21.15.47:0/2328631632 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): get_auth_request con 0x7f3e90137d90 auth_method 0 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e90136d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e901385b0 0x7f3e901372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): get_auth_request con 0x7f3e90138170 auth_method 0 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): get_auth_request con 0x7f3e901385b0 auth_method 0 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.322+0000 7f3e8effd700 10 monclient(hunting): handle_auth_done global_id 54298 payload 824 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 msgr2=0x7f3e90136d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3e90138170 0x7f3e90136d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e901385b0 msgr2=0x7f3e901372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3e901385b0 0x7f3e901372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 monclient: found mon.noname-f 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:22:40.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009550 con 0x7f3e90137d90 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8effd700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90137d90 0x7f3e901cc150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009130 tx=0x7f3e80004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 client.?.objecter ms_handle_connect 0x7f3e90137d90 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e800036a0 con 0x7f3e90137d90 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: dump: 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.343 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80003860 con 0x7f3e90137d90 2024-04-20T02:22:40.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 5 monclient: authenticate success, global_id 54298 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 10 monclient: _renew_subs 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9009e9e0 con 0x7f3e90137d90 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e80003e40 con 0x7f3e90137d90 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e8d7fa700 4 set_mon_vals no callback set 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e87fff700 10 monclient: dump: 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.345 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 10 client.54298.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 10 monclient: _renew_subs 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e901377e0 con 0x7f3e90137d90 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.323+0000 7f3e95958700 1 librados: init done 2024-04-20T02:22:40.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e95958700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e95958700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 0x7f3e901208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90068180 con 0x7f3e9006aac0 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3e80021020 con 0x7f3e90137d90 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 0x7f3e901208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e7003a2f0 0x7f3e7003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient: get_auth_request con 0x7f3e9006aac0 auth_method 0 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(302..302 src has 1..302) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f3e8004b8c0 con 0x7f3e90137d90 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 10 client.54298.objecter ms_dispatch 0x7f3e90001550 osd_map(302..302 src has 1..302) v4 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 3 client.54298.objecter handle_osd_map got epochs [302,302] > 0 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 3 client.54298.objecter handle_osd_map decoding full epoch 302 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e87fff700 20 client.54298.objecter dump_active .. 0 homeless 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8f7fe700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e7003a2f0 0x7f3e7003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8f7fe700 10 monclient: get_auth_request con 0x7f3e7003a2f0 auth_method 0 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_done global_id 54307 payload 824 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.324+0000 7f3e8e7fc700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e9009f760 con 0x7f3e9006aac0 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 client.54298.objecter ms_handle_connect 0x7f3e9006aac0 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 client.54298.objecter resend_mon_ops 2024-04-20T02:22:40.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e8e7fc700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 0x7f3e901208d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a4770 tx=0x7f3e78011960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 client.54298.objecter ms_handle_connect 0x7f3e7003a2f0 2024-04-20T02:22:40.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e8f7fe700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e7003a2f0 0x7f3e7003c7b0 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7f3e90074d10 tx=0x7f3e88009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e7800c070 con 0x7f3e9006aac0 2024-04-20T02:22:40.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:40.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 monclient: dump: 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.351 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.352 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3e78004030 con 0x7f3e9006aac0 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 msgr2=0x7f3e901208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 0x7f3e901208d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a4770 tx=0x7f3e78011960 comp rx=0 tx=0).stop 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 msgr2=0x7f3e901208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.325+0000 7f3e87fff700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e9006aac0 0x7f3e901208d0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a4770 tx=0x7f3e78011960 comp rx=0 tx=0).stop 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e95958700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e95958700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 0x7f3e90121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e95958700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90124010 con 0x7f3e90120e10 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 0x7f3e90121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient: get_auth_request con 0x7f3e90120e10 auth_method 0 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.333+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e8e7fc700 10 monclient(hunting): handle_auth_done global_id 54316 payload 824 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e8e7fc700 1 -- 172.21.15.47:0/2328631632 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3e90121760 con 0x7f3e90120e10 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e8e7fc700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 0x7f3e90121220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e78006b20 tx=0x7f3e78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 client.54298.objecter ms_handle_connect 0x7f3e90120e10 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 client.54298.objecter resend_mon_ops 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3e78015cf0 con 0x7f3e90120e10 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 monclient: dump: 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.358 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f3e78015e50 con 0x7f3e90120e10 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:40.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 msgr2=0x7f3e90121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 0x7f3e90121220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e78006b20 tx=0x7f3e78004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 msgr2=0x7f3e90121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.334+0000 7f3e87fff700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90120e10 0x7f3e90121220 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e78006b20 tx=0x7f3e78004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:40.360 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.362 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.364 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.365 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.365 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.365 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.365 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:40.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.365 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:40.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.366 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:40.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:40.366 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:40.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.368 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.376 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:40.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.378 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.378 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.380 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.383 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.384 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:40.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.385 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:40.385 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 10 librados: watch_flush enter 2024-04-20T02:22:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 10 librados: watch_flush exit 2024-04-20T02:22:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 20 client.54298.objecter shutdown clearing up homeless session... 2024-04-20T02:22:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 10 client.54298.objecter successfully canceled tick 2024-04-20T02:22:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e7003a2f0 msgr2=0x7f3e7003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3e7003a2f0 0x7f3e7003c7b0 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 crypto rx=0x7f3e90074d10 tx=0x7f3e88009780 comp rx=0 tx=0).stop 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 10 monclient: shutdown 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90137d90 msgr2=0x7f3e901cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 --2- 172.21.15.47:0/2328631632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e90137d90 0x7f3e901cc150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009130 tx=0x7f3e80004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e8f7fe700 1 -- 172.21.15.47:0/2328631632 reap_dead start 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e8f7fe700 1 -- 172.21.15.47:0/2328631632 reap_dead start 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 shutdown_connections 2024-04-20T02:22:40.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 >> 172.21.15.47:0/2328631632 conn(0x7f3e900aa8b0 msgr2=0x7f3e900a5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 shutdown_connections 2024-04-20T02:22:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 -- 172.21.15.47:0/2328631632 wait complete. 2024-04-20T02:22:40.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.335+0000 7f3e95958700 1 librados: shutdown 2024-04-20T02:22:40.390 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:40.390 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:22:40.391 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:22:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.532+0000 7f9974fa1700 10 monclient: get_monmap_and_config 2024-04-20T02:22:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.532+0000 7f9974fa1700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.532+0000 7f9974fa1700 10 monclient: monmap: 2024-04-20T02:22:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.532 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:40.533739+0000 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:40.533739+0000 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.533 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:40.534 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:40.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 1 Processor -- start 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 1 -- start start 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient: init 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient: _add_conns ranks=[2,4,0,3,10,9,8,5,7,6,1] 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a4f50 0x7f997009f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient(hunting): picked mon.noname-b con 0x7f99700a4f50 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 0x7f99700a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient(hunting): picked mon.noname-c con 0x7f99700a13e0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:40.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f99700a36a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99700751e0 con 0x7f99700a36a0 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.533+0000 7f9974fa1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970072200 con 0x7f99700a4f50 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99700a5330 con 0x7f99700a13e0 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 10 monclient(hunting): authenticate will time out at 2039.935020s 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 0x7f99700a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 0x7f99700a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36002/0 (socket says 172.21.15.47:36002) 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f99700a36a0 0x7f99700a3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:46342/0 (socket says 172.21.15.47:46342) 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 -- 172.21.15.47:0/2954163213 learned_addr learned my addr 172.21.15.47:0/2954163213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): get_auth_request con 0x7f99700a13e0 auth_method 0 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 1 --2- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a4f50 0x7f997009f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): get_auth_request con 0x7f99700a4f50 auth_method 0 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient(hunting): handle_auth_done global_id 54229 payload 274 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 -- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a4f50 msgr2=0x7f997009f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 1 -- 172.21.15.47:0/2954163213 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601934130 0 0) 0x7f99700751e0 con 0x7f99700a36a0 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 --2- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a4f50 0x7f997009f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 -- 172.21.15.47:0/2954163213 >> v1:172.21.15.47:6789/0 conn(0x7f99700a36a0 legacy=0x7f99700a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 monclient: found mon.noname-c 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 -- 172.21.15.47:0/2954163213 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970071c40 con 0x7f99700a13e0 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996effd700 1 --2- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 0x7f99700a17e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9964009920 tx=0x7f9964009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8f6dd8f8abca3fff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 1 -- 172.21.15.47:0/2954163213 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9964011aa0 con 0x7f99700a13e0 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:40.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: dump: 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 5 monclient: authenticate success, global_id 54229 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 1 -- 172.21.15.47:0/2954163213 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964011c00 con 0x7f99700a13e0 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f9974fa1700 4 set_mon_vals no callback set 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 1 -- 172.21.15.47:0/2954163213 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9964011d70 con 0x7f99700a13e0 2024-04-20T02:22:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.534+0000 7f996e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f996e7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f996e7fc700 10 monclient: dump: 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.552 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 10 monclient: shutdown 2024-04-20T02:22:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 -- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 msgr2=0x7f99700a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 --2- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 0x7f99700a17e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9964009920 tx=0x7f9964009c30 comp rx=0 tx=0).stop 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 -- 172.21.15.47:0/2954163213 shutdown_connections 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 --2- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a4f50 0x7f997009f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 --2- 172.21.15.47:0/2954163213 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f99700a13e0 0x7f99700a17e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 -- 172.21.15.47:0/2954163213 >> 172.21.15.47:0/2954163213 conn(0x7f99700a83a0 msgr2=0x7f99700a4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 -- 172.21.15.47:0/2954163213 shutdown_connections 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.535+0000 7f9974fa1700 1 -- 172.21.15.47:0/2954163213 wait complete. 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 10 monclient: monmap: 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:40.553 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 1 librados: starting msgr at 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 1 librados: starting objecter 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 1 Processor -- start 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 1 -- start start 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 1 librados: setting wanted keys 2024-04-20T02:22:40.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 1 librados: calling monclient init 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.536+0000 7f9974fa1700 10 monclient: init 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient: _add_conns ranks=[0,2,7,3,4,8,6,5,1] 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99700a13e0 0x7f99701cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): picked mon.noname-a con 0x7f99700a13e0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a36a0 0x7f9970135a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): picked mon.noname-b con 0x7f99700a36a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99700a4f50 0x7f99701c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): picked mon.noname-i con 0x7f99700a4f50 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701cc8e0 con 0x7f99700a13e0 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970070fe0 con 0x7f99700a36a0 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99700a1f50 con 0x7f99700a4f50 2024-04-20T02:22:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f9974fa1700 10 monclient(hunting): authenticate will time out at 2039.938372s 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99700a13e0 0x7f99701cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99700a13e0 0x7f99701cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60996/0 (socket says 172.21.15.47:60996) 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 1 -- 172.21.15.47:0/758401646 learned_addr learned my addr 172.21.15.47:0/758401646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): get_auth_request con 0x7f99700a13e0 auth_method 0 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a36a0 0x7f9970135a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): get_auth_request con 0x7f99700a36a0 auth_method 0 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996ffff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99700a4f50 0x7f99701c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996ffff700 10 monclient(hunting): get_auth_request con 0x7f99700a4f50 auth_method 0 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient(hunting): handle_auth_done global_id 23933 payload 824 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99700a4f50 msgr2=0x7f99701c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99700a4f50 0x7f99701c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99700a13e0 msgr2=0x7f99701cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99700a13e0 0x7f99701cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 monclient: found mon.noname-b 2024-04-20T02:22:40.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996effd700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964009550 con 0x7f99700a36a0 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.537+0000 7f996f7fe700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99700a13e0 0x7f99701cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f996effd700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a36a0 0x7f9970135a10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9964016000 tx=0x7f9964004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 client.?.objecter ms_handle_connect 0x7f99700a36a0 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9964004310 con 0x7f99700a36a0 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: dump: 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.558 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964004490 con 0x7f99700a36a0 2024-04-20T02:22:40.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 5 monclient: authenticate success, global_id 23933 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 10 monclient: _renew_subs 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997006e2e0 con 0x7f99700a36a0 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f996401a420 con 0x7f99700a36a0 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9957fff700 10 monclient: dump: 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.561 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f996dffb700 4 set_mon_vals no callback set 2024-04-20T02:22:40.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 10 client.23933.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 10 monclient: _renew_subs 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9970124e50 con 0x7f99700a36a0 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.538+0000 7f9974fa1700 1 librados: init done 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9974fa1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9974fa1700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 0x7f99701c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970068180 con 0x7f9970123e30 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9964022030 con 0x7f99700a36a0 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 0x7f99701c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient: get_auth_request con 0x7f9970123e30 auth_method 0 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f995c03a2f0 0x7f995c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(302..302 src has 1..302) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f9964020070 con 0x7f99700a36a0 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 10 client.23933.objecter ms_dispatch 0x7f9970001550 osd_map(302..302 src has 1..302) v4 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 3 client.23933.objecter handle_osd_map got epochs [302,302] > 0 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 3 client.23933.objecter handle_osd_map decoding full epoch 302 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f9957fff700 20 client.23933.objecter dump_active .. 0 homeless 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 10 monclient(hunting): handle_auth_done global_id 22035 payload 824 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f997006a740 con 0x7f9970123e30 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996ffff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f995c03a2f0 0x7f995c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996ffff700 10 monclient: get_auth_request con 0x7f995c03a2f0 auth_method 0 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 client.23933.objecter ms_handle_connect 0x7f9970123e30 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 client.23933.objecter resend_mon_ops 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.539+0000 7f996f7fe700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 0x7f99701c8ea0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9960009f40 tx=0x7f996000f520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f996000fd80 con 0x7f9970123e30 2024-04-20T02:22:40.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 monclient: dump: 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.565 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 client.23933.objecter ms_handle_connect 0x7f995c03a2f0 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f996ffff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f995c03a2f0 0x7f995c03c7b0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f997006deb0 tx=0x7f9968001880 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9960014440 con 0x7f9970123e30 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 msgr2=0x7f99701c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 0x7f99701c8ea0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9960009f40 tx=0x7f996000f520 comp rx=0 tx=0).stop 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 msgr2=0x7f99701c8ea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.540+0000 7f9957fff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970123e30 0x7f99701c8ea0 secure :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9960009f40 tx=0x7f996000f520 comp rx=0 tx=0).stop 2024-04-20T02:22:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f9974fa1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f9974fa1700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 0x7f99701c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f9974fa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997006a400 con 0x7f9970135f50 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 0x7f99701c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient: get_auth_request con 0x7f9970135f50 auth_method 0 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 10 monclient(hunting): handle_auth_done global_id 22044 payload 824 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.549+0000 7f996f7fe700 1 -- 172.21.15.47:0/758401646 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9970071f20 con 0x7f9970135f50 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f996f7fe700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 0x7f99701c93e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f996001b000 tx=0x7f9960004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 client.23933.objecter ms_handle_connect 0x7f9970135f50 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 client.23933.objecter resend_mon_ops 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f996002b070 con 0x7f9970135f50 2024-04-20T02:22:40.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 monclient: dump: 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.570 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f9960023040 con 0x7f9970135f50 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 msgr2=0x7f99701c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 0x7f99701c93e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f996001b000 tx=0x7f9960004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 msgr2=0x7f99701c93e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9957fff700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9970135f50 0x7f99701c93e0 secure :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f996001b000 tx=0x7f9960004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.574 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:40.576 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.577 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:40.578 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:40.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:40.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.587 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:40.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.589 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:40.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.596 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.597 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:40.598 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 10 librados: watch_flush enter 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 10 librados: watch_flush exit 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 20 client.23933.objecter shutdown clearing up homeless session... 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 10 client.23933.objecter successfully canceled tick 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f995c03a2f0 msgr2=0x7f995c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f995c03a2f0 0x7f995c03c7b0 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 crypto rx=0x7f997006deb0 tx=0x7f9968001880 comp rx=0 tx=0).stop 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 10 monclient: shutdown 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a36a0 msgr2=0x7f9970135a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f9974fa1700 1 --2- 172.21.15.47:0/758401646 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99700a36a0 0x7f9970135a10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9964016000 tx=0x7f9964004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.550+0000 7f996ffff700 1 -- 172.21.15.47:0/758401646 reap_dead start 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.551+0000 7f996ffff700 1 -- 172.21.15.47:0/758401646 reap_dead start 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.551+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 shutdown_connections 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.551+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 >> 172.21.15.47:0/758401646 conn(0x7f99700a83a0 msgr2=0x7f99700a01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.551+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 shutdown_connections 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.551+0000 7f9974fa1700 1 -- 172.21.15.47:0/758401646 wait complete. 2024-04-20T02:22:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.551+0000 7f9974fa1700 1 librados: shutdown 2024-04-20T02:22:40.602 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:40.603 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:22:40.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:22:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.741+0000 7f5bb048e700 10 monclient: get_monmap_and_config 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.741+0000 7f5bb048e700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.741+0000 7f5bb048e700 10 monclient: monmap: 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:40.743105+0000 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:40.743105+0000 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.742 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:40.743 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.742+0000 7f5bb048e700 1 Processor -- start 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.742+0000 7f5bb048e700 1 -- start start 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.742+0000 7f5bb048e700 10 monclient: init 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient: _add_conns ranks=[8,9,10,7,2,6,4,1,0,5,3] 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 0x7f5ba80a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): picked mon.noname-e con 0x7f5ba8066610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 0x7f5ba80a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): picked mon.noname-a con 0x7f5ba80a2450 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5ba813c6b0 0x7f5ba813eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): picked mon.noname-f con 0x7f5ba813c6b0 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:40.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba806cd70 con 0x7f5ba80a2450 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80669f0 con 0x7f5ba813c6b0 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba809c420 con 0x7f5ba8066610 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bb048e700 10 monclient(hunting): authenticate will time out at 2040.144450s 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 0x7f5ba80a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 0x7f5ba80a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 0x7f5ba80a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:32780/0 (socket says 172.21.15.47:32780) 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 0x7f5ba80a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58354/0 (socket says 172.21.15.47:58354) 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 1 -- 172.21.15.47:0/2496796552 learned_addr learned my addr 172.21.15.47:0/2496796552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): get_auth_request con 0x7f5ba80a2450 auth_method 0 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): get_auth_request con 0x7f5ba8066610 auth_method 0 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5baea2b700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.179:3300/0 conn(0x7f5ba813c6b0 0x7f5ba813eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5baea2b700 10 monclient(hunting): get_auth_request con 0x7f5ba813c6b0 auth_method 0 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5baea2b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5baea2b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5baea2b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient(hunting): handle_auth_done global_id 35117 payload 274 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 -- 172.21.15.47:0/2496796552 >> v2:172.21.15.179:3300/0 conn(0x7f5ba813c6b0 msgr2=0x7f5ba813eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.179:3300/0 conn(0x7f5ba813c6b0 0x7f5ba813eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 -- 172.21.15.47:0/2496796552 >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 msgr2=0x7f5ba80a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 0x7f5ba80a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 monclient: found mon.noname-e 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bae22a700 1 -- 172.21.15.47:0/2496796552 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80751e0 con 0x7f5ba8066610 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5bada29700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 0x7f5ba80a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.743+0000 7f5baea2b700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.179:3300/0 conn(0x7f5ba813c6b0 0x7f5ba813eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bae22a700 1 --2- 172.21.15.47:0/2496796552 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 0x7f5ba80a1f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806ec70 tx=0x7f5ba4009b20 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9d84f39f111b77ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 1 -- 172.21.15.47:0/2496796552 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ba4011aa0 con 0x7f5ba8066610 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: dump: 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.758 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 1 -- 172.21.15.47:0/2496796552 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4011c00 con 0x7f5ba8066610 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 5 monclient: authenticate success, global_id 35117 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 4 set_mon_vals no callback set 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 1 -- 172.21.15.47:0/2496796552 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ba4011d70 con 0x7f5ba8066610 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bad228700 10 monclient: dump: 2024-04-20T02:22:40.759 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.760 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 10 monclient: shutdown 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 1 -- 172.21.15.47:0/2496796552 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 msgr2=0x7f5ba80a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 1 --2- 172.21.15.47:0/2496796552 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 0x7f5ba80a1f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806ec70 tx=0x7f5ba4009b20 comp rx=0 tx=0).stop 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 1 -- 172.21.15.47:0/2496796552 shutdown_connections 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.744+0000 7f5bb048e700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.179:3300/0 conn(0x7f5ba813c6b0 0x7f5ba813eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 --2- 172.21.15.47:0/2496796552 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5ba8066610 0x7f5ba80a1f10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 --2- 172.21.15.47:0/2496796552 >> v2:172.21.15.47:3300/0 conn(0x7f5ba80a2450 0x7f5ba80a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 -- 172.21.15.47:0/2496796552 >> 172.21.15.47:0/2496796552 conn(0x7f5ba80a3000 msgr2=0x7f5ba80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 -- 172.21.15.47:0/2496796552 shutdown_connections 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 -- 172.21.15.47:0/2496796552 wait complete. 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 10 monclient: monmap: 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:40.761 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 librados: starting msgr at 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.745+0000 7f5bb048e700 1 librados: starting objecter 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 Processor -- start 2024-04-20T02:22:40.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 -- start start 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 librados: setting wanted keys 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 librados: calling monclient init 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient: init 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient: _add_conns ranks=[2,5,6,1,0,3,4,8,7] 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5ba8066610 0x7f5ba8071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): picked mon.noname-b con 0x7f5ba8066610 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba80a2450 0x7f5ba8071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): picked mon.noname-h con 0x7f5ba80a2450 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5ba813c6b0 0x7f5ba80723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): picked mon.noname-d con 0x7f5ba813c6b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80927e0 con 0x7f5ba8066610 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8092ac0 con 0x7f5ba80a2450 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80906f0 con 0x7f5ba813c6b0 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bb048e700 10 monclient(hunting): authenticate will time out at 2040.147750s 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5ba8066610 0x7f5ba8071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5ba8066610 0x7f5ba8071960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60886/0 (socket says 172.21.15.47:60886) 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 1 -- 172.21.15.47:0/1707954318 learned_addr learned my addr 172.21.15.47:0/1707954318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): get_auth_request con 0x7f5ba8066610 auth_method 0 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5baea2b700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5ba813c6b0 0x7f5ba80723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5baea2b700 10 monclient(hunting): get_auth_request con 0x7f5ba813c6b0 auth_method 0 2024-04-20T02:22:40.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5baea2b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5baea2b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5baea2b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bada29700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba80a2450 0x7f5ba8071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.746+0000 7f5bae22a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): get_auth_request con 0x7f5ba80a2450 auth_method 0 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5baea2b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5baea2b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5baea2b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 10 monclient(hunting): handle_auth_done global_id 23942 payload 824 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5ba813c6b0 msgr2=0x7f5ba80723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5ba813c6b0 0x7f5ba80723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba80a2450 msgr2=0x7f5ba8071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba80a2450 0x7f5ba8071ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 monclient: found mon.noname-b 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98009550 con 0x7f5ba8066610 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bae22a700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5ba8066610 0x7f5ba8071960 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40090b0 tx=0x7f5ba4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 client.?.objecter ms_handle_connect 0x7f5ba8066610 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ba4004470 con 0x7f5ba8066610 2024-04-20T02:22:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: dump: 2024-04-20T02:22:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba401f070 con 0x7f5ba8066610 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 5 monclient: authenticate success, global_id 23942 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 10 monclient: _renew_subs 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba40094a0 con 0x7f5ba8066610 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ba401b430 con 0x7f5ba8066610 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5b9effd700 10 monclient: dump: 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.769 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5baca27700 4 set_mon_vals no callback set 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 10 client.23942.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 10 monclient: _renew_subs 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba809a890 con 0x7f5ba8066610 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.747+0000 7f5bb048e700 1 librados: init done 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bb048e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bb048e700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 0x7f5ba81cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8068230 con 0x7f5ba806aac0 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5ba4021030 con 0x7f5ba8066610 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 0x7f5ba81cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5b9effd700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5b8c03a220 0x7f5b8c03c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient: get_auth_request con 0x7f5ba806aac0 auth_method 0 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(302..302 src has 1..302) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f5ba404bc60 con 0x7f5ba8066610 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5b9effd700 10 client.23942.objecter ms_dispatch 0x7f5ba8001550 osd_map(302..302 src has 1..302) v4 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5b9effd700 3 client.23942.objecter handle_osd_map got epochs [302,302] > 0 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5b9effd700 3 client.23942.objecter handle_osd_map decoding full epoch 302 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.748+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 20 client.23942.objecter dump_active .. 0 homeless 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5baea2b700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5b8c03a220 0x7f5b8c03c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5baea2b700 10 monclient: get_auth_request con 0x7f5b8c03a220 auth_method 0 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5bada29700 10 monclient(hunting): handle_auth_done global_id 44456 payload 824 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5bada29700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ba809b960 con 0x7f5ba806aac0 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 client.23942.objecter ms_handle_connect 0x7f5ba806aac0 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 client.23942.objecter resend_mon_ops 2024-04-20T02:22:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5bada29700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 0x7f5ba81cd020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a640 tx=0x7f5b98011f40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 client.23942.objecter ms_handle_connect 0x7f5b8c03a220 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5baea2b700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5b8c03a220 0x7f5b8c03c6e0 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a820 tx=0x7f5ba00097d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b98014440 con 0x7f5ba806aac0 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 monclient: dump: 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5b9800fa90 con 0x7f5ba806aac0 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 msgr2=0x7f5ba81cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 0x7f5ba81cd020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a640 tx=0x7f5b98011f40 comp rx=0 tx=0).stop 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 msgr2=0x7f5ba81cd020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.749+0000 7f5b9effd700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba806aac0 0x7f5ba81cd020 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a640 tx=0x7f5b98011f40 comp rx=0 tx=0).stop 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.757+0000 7f5bb048e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.757+0000 7f5bb048e700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 0x7f5ba802bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.757+0000 7f5bb048e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.757+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8092590 con 0x7f5ba802bbc0 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 0x7f5ba802bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient: get_auth_request con 0x7f5ba802bbc0 auth_method 0 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 10 monclient(hunting): handle_auth_done global_id 44465 payload 824 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 1 -- 172.21.15.47:0/1707954318 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5ba809bc40 con 0x7f5ba802bbc0 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5b9effd700 10 client.23942.objecter ms_handle_connect 0x7f5ba802bbc0 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5bada29700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 0x7f5ba802bfd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a640 tx=0x7f5b98004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5b9effd700 10 client.23942.objecter resend_mon_ops 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b98014cd0 con 0x7f5ba802bbc0 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5b9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5b9effd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.758+0000 7f5b9effd700 10 monclient: dump: 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.777 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f5b98022580 con 0x7f5ba802bbc0 2024-04-20T02:22:40.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 msgr2=0x7f5ba802bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 0x7f5ba802bfd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a640 tx=0x7f5b98004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 msgr2=0x7f5ba802bfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5b9effd700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5ba802bbc0 0x7f5ba802bfd0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a640 tx=0x7f5b98004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:40.779 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.780 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:40.782 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.783 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:40.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.788 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:40.790 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:40.792 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.793 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:40.794 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:40.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:40.797 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:40.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:40.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:40.800 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:40.800 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 10 librados: watch_flush enter 2024-04-20T02:22:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 10 librados: watch_flush exit 2024-04-20T02:22:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 20 client.23942.objecter shutdown clearing up homeless session... 2024-04-20T02:22:40.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 10 client.23942.objecter successfully canceled tick 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5b8c03a220 msgr2=0x7f5b8c03c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5b8c03a220 0x7f5b8c03c6e0 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 crypto rx=0x7f5ba806a820 tx=0x7f5ba00097d0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 10 monclient: shutdown 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5ba8066610 msgr2=0x7f5ba8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 --2- 172.21.15.47:0/1707954318 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5ba8066610 0x7f5ba8071960 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40090b0 tx=0x7f5ba4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5baea2b700 1 -- 172.21.15.47:0/1707954318 reap_dead start 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5baea2b700 1 -- 172.21.15.47:0/1707954318 reap_dead start 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 shutdown_connections 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 >> 172.21.15.47:0/1707954318 conn(0x7f5ba80a3000 msgr2=0x7f5ba813be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 shutdown_connections 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 -- 172.21.15.47:0/1707954318 wait complete. 2024-04-20T02:22:40.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.759+0000 7f5bb048e700 1 librados: shutdown 2024-04-20T02:22:40.803 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:40.803 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:22:40.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.940+0000 7ff7c6bfc700 10 monclient: get_monmap_and_config 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.941+0000 7ff7c6bfc700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.941+0000 7ff7c6bfc700 10 monclient: monmap: 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:40.942180+0000 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:40.942180+0000 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.942 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.943 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.943 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:40.943 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:40.943 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.941+0000 7ff7c6bfc700 1 Processor -- start 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.941+0000 7ff7c6bfc700 1 -- start start 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.941+0000 7ff7c6bfc700 10 monclient: init 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient: _add_conns ranks=[10,6,5,4,0,7,2,8,3,9,1] 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff7c0071610 0x7ff7c00719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): picked mon.noname-f con 0x7ff7c0071610 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 0x7ff7c00a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): picked mon.noname-d con 0x7ff7c0071f30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:40.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c00a2e90 0x7ff7c00a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): picked mon.noname-h con 0x7ff7c00a2e90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c00727b0 con 0x7ff7c0071610 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00a0880 con 0x7ff7c00a2e90 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c009e8f0 con 0x7ff7c0071f30 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c6bfc700 10 monclient(hunting): authenticate will time out at 2040.343564s 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 0x7ff7c00a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 0x7ff7c00a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48276/0 (socket says 172.21.15.47:48276) 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 1 -- 172.21.15.47:0/3749274694 learned_addr learned my addr 172.21.15.47:0/3749274694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): get_auth_request con 0x7ff7c0071f30 auth_method 0 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 1 --2- 172.21.15.47:0/3749274694 >> v2:172.21.15.179:3300/0 conn(0x7ff7c0071610 0x7ff7c00719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c5199700 1 --2- 172.21.15.47:0/3749274694 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c00a2e90 0x7ff7c00a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7bffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): get_auth_request con 0x7ff7c0071610 auth_method 0 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c5199700 10 monclient(hunting): get_auth_request con 0x7ff7c00a2e90 auth_method 0 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c5199700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c5199700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c5199700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.942+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c5199700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c5199700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c5199700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 10 monclient(hunting): handle_auth_done global_id 54276 payload 274 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 -- 172.21.15.47:0/3749274694 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c00a2e90 msgr2=0x7ff7c00a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 --2- 172.21.15.47:0/3749274694 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c00a2e90 0x7ff7c00a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 -- 172.21.15.47:0/3749274694 >> v2:172.21.15.179:3300/0 conn(0x7ff7c0071610 msgr2=0x7ff7c00719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:40.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 --2- 172.21.15.47:0/3749274694 >> v2:172.21.15.179:3300/0 conn(0x7ff7c0071610 0x7ff7c00719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 monclient: found mon.noname-d 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 -- 172.21.15.47:0/3749274694 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c00751e0 con 0x7ff7c0071f30 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bffff700 1 --2- 172.21.15.47:0/3749274694 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 0x7ff7c00a2950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400bb90 tx=0x7ff7b400bea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e80b657c683dcba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 1 -- 172.21.15.47:0/3749274694 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4010470 con 0x7ff7c0071f30 2024-04-20T02:22:40.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: dump: 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.970 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.971 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.971 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.971 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.971 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.971 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.971 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 1 -- 172.21.15.47:0/3749274694 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b40105d0 con 0x7ff7c0071f30 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c6bfc700 5 monclient: authenticate success, global_id 54276 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c6bfc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:40.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c6bfc700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 1 -- 172.21.15.47:0/3749274694 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4010bb0 con 0x7ff7c0071f30 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7c6bfc700 4 set_mon_vals no callback set 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.943+0000 7ff7bf7fe700 10 monclient: dump: 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.973 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 10 monclient: shutdown 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/3749274694 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 msgr2=0x7ff7c00a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/3749274694 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 0x7ff7c00a2950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400bb90 tx=0x7ff7b400bea0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/3749274694 shutdown_connections 2024-04-20T02:22:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/3749274694 >> v2:172.21.15.179:3300/0 conn(0x7ff7c0071610 0x7ff7c00719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/3749274694 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff7c00a2e90 0x7ff7c00a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/3749274694 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff7c0071f30 0x7ff7c00a2950 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/3749274694 >> 172.21.15.47:0/3749274694 conn(0x7ff7c00a7570 msgr2=0x7ff7c00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/3749274694 shutdown_connections 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/3749274694 wait complete. 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 10 monclient: build_initial_monmap 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.944+0000 7ff7c6bfc700 10 monclient: monmap: 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:40.976 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 1 librados: starting msgr at 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 1 librados: starting objecter 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 1 Processor -- start 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 1 -- start start 2024-04-20T02:22:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 1 librados: setting wanted keys 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 1 librados: calling monclient init 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.945+0000 7ff7c6bfc700 10 monclient: init 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient: _add_conns ranks=[3,8,4,0,6,2,1,7,5] 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c0071610 0x7ff7c01c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): picked mon.noname-g con 0x7ff7c0071610 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff7c0071f30 0x7ff7c01b8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): picked mon.noname-e con 0x7ff7c0071f30 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c00a2e90 0x7ff7c01b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): picked mon.noname-c con 0x7ff7c00a2e90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c0072460 con 0x7ff7c0071610 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01a8040 con 0x7ff7c00a2e90 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01c9630 con 0x7ff7c0071f30 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:40.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c6bfc700 10 monclient(hunting): authenticate will time out at 2040.347628s 2024-04-20T02:22:40.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff7c0071f30 0x7ff7c01b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c00a2e90 0x7ff7c01b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff7c0071f30 0x7ff7c01b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58360/0 (socket says 172.21.15.47:58360) 2024-04-20T02:22:40.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 1 -- 172.21.15.47:0/290196613 learned_addr learned my addr 172.21.15.47:0/290196613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:40.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): get_auth_request con 0x7ff7c0071f30 auth_method 0 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): get_auth_request con 0x7ff7c00a2e90 auth_method 0 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7bffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c5199700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.946+0000 7ff7c4998700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c0071610 0x7ff7c01c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c4998700 10 monclient(hunting): get_auth_request con 0x7ff7c0071610 auth_method 0 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c4998700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c4998700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c4998700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 10 monclient(hunting): handle_auth_done global_id 35126 payload 824 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c00a2e90 msgr2=0x7ff7c01b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c00a2e90 0x7ff7c01b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c0071610 msgr2=0x7ff7c01c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c0071610 0x7ff7c01c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 monclient: found mon.noname-e 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b400b7f0 con 0x7ff7c0071f30 2024-04-20T02:22:40.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c4998700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff7c0071610 0x7ff7c01c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c5199700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7c00a2e90 0x7ff7c01b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bffff700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff7c0071f30 0x7ff7c01b8910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400bee0 tx=0x7ff7b4004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 client.?.objecter ms_handle_connect 0x7ff7c0071f30 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4010470 con 0x7ff7c0071f30 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: dump: 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.985 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.986 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: _finish_auth 0 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 5 monclient: authenticate success, global_id 35126 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b4010dc0 con 0x7ff7c0071f30 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b4003b10 con 0x7ff7c0071f30 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 10 monclient: _renew_subs 2024-04-20T02:22:40.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c01a9a70 con 0x7ff7c0071f30 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7beffd700 4 set_mon_vals no callback set 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7bd7fa700 10 monclient: dump: 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.988 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 10 client.35126.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 10 monclient: _renew_subs 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01aa030 con 0x7ff7c0071f30 2024-04-20T02:22:40.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.947+0000 7ff7c6bfc700 1 librados: init done 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff7b4003c70 con 0x7ff7c0071f30 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c6bfc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 0x7ff7c01cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c0068310 con 0x7ff7c006aac0 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7ac03a2e0 0x7ff7ac03c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(302..302 src has 1..302) v4 ==== 7062+0+0 (secure 0 0 0) 0x7ff7b404bfe0 con 0x7ff7c0071f30 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 10 client.35126.objecter ms_dispatch 0x7ff7c0001550 osd_map(302..302 src has 1..302) v4 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 3 client.35126.objecter handle_osd_map got epochs [302,302] > 0 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 3 client.35126.objecter handle_osd_map decoding full epoch 302 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 0x7ff7c01cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient: get_auth_request con 0x7ff7c006aac0 auth_method 0 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7bd7fa700 20 client.35126.objecter dump_active .. 0 homeless 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c5199700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7ac03a2e0 0x7ff7ac03c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.948+0000 7ff7c5199700 10 monclient: get_auth_request con 0x7ff7ac03a2e0 auth_method 0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7c4998700 10 monclient(hunting): handle_auth_done global_id 54241 payload 824 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7c4998700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7c009ffe0 con 0x7ff7c006aac0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 client.35126.objecter ms_handle_connect 0x7ff7c006aac0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7c4998700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 0x7ff7c01cc090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0011830 tx=0x7ff7b0011bf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 client.35126.objecter resend_mon_ops 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 client.35126.objecter ms_handle_connect 0x7ff7ac03a2e0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7c5199700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7ac03a2e0 0x7ff7ac03c7a0 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009fd0 tx=0x7ff7b800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b0015ae0 con 0x7ff7c006aac0 2024-04-20T02:22:40.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 monclient: dump: 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.993 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff7b0015c40 con 0x7ff7c006aac0 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 msgr2=0x7ff7c01cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 0x7ff7c01cc090 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0011830 tx=0x7ff7b0011bf0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 msgr2=0x7ff7c01cc090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.949+0000 7ff7bd7fa700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c006aac0 0x7ff7c01cc090 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0011830 tx=0x7ff7b0011bf0 comp rx=0 tx=0).stop 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c6bfc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 0x7ff7c01cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c6bfc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:40.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c006a400 con 0x7ff7c01c9090 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c4998700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 0x7ff7c01cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c4998700 10 monclient: get_auth_request con 0x7ff7c01c9090 auth_method 0 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c4998700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c4998700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.957+0000 7ff7c4998700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7c4998700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7c4998700 10 monclient(hunting): handle_auth_done global_id 54250 payload 824 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7c4998700 1 -- 172.21.15.47:0/290196613 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff7c00a02c0 con 0x7ff7c01c9090 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7c4998700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 0x7ff7c01cc5d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7b001c000 tx=0x7ff7b0004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 client.35126.objecter ms_handle_connect 0x7ff7c01c9090 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 client.35126.objecter resend_mon_ops 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff7b0013870 con 0x7ff7c01c9090 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 monclient: dump: 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7ff7b0013400 con 0x7ff7c01c9090 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 msgr2=0x7ff7c01cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 0x7ff7c01cc5d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7b001c000 tx=0x7ff7b0004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 msgr2=0x7ff7c01cc5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.958+0000 7ff7bd7fa700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff7c01c9090 0x7ff7c01cc5d0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7b001c000 tx=0x7ff7b0004750 comp rx=0 tx=0).stop 2024-04-20T02:22:40.997 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 52, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:40.998 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:40.999 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:41.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:41.001 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:41.002 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.004 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:41.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.010 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.012 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:41.013 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:41.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:41.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:41.018 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 10 librados: watch_flush enter 2024-04-20T02:22:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 10 librados: watch_flush exit 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 20 client.35126.objecter shutdown clearing up homeless session... 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 10 client.35126.objecter successfully canceled tick 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7ac03a2e0 msgr2=0x7ff7ac03c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff7ac03a2e0 0x7ff7ac03c7a0 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009fd0 tx=0x7ff7b800b040 comp rx=0 tx=0).stop 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 10 monclient: shutdown 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff7c0071f30 msgr2=0x7ff7c01b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 --2- 172.21.15.47:0/290196613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff7c0071f30 0x7ff7c01b8910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400bee0 tx=0x7ff7b4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c5199700 1 -- 172.21.15.47:0/290196613 reap_dead start 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c5199700 1 -- 172.21.15.47:0/290196613 reap_dead start 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 shutdown_connections 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 >> 172.21.15.47:0/290196613 conn(0x7ff7c00a7570 msgr2=0x7ff7c01bb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 shutdown_connections 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 -- 172.21.15.47:0/290196613 wait complete. 2024-04-20T02:22:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:40.959+0000 7ff7c6bfc700 1 librados: shutdown 2024-04-20T02:22:41.022 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:22:41.022 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h'] 2024-04-20T02:22:41.022 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-20T02:22:41.022 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-04-20T02:22:41.022 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h store 2024-04-20T02:22:41.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h sync_force --yes-i-really-mean-it 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.171+0000 7f68ce474700 10 monclient: get_monmap_and_config 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.171+0000 7f68ce474700 10 monclient: build_initial_monmap 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.171+0000 7f68ce474700 10 monclient: monmap: 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:41.172351+0000 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:41.172351+0000 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:41.172 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.171+0000 7f68ce474700 1 Processor -- start 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 -- start start 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient: init 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient: _add_conns ranks=[3,7,5,6,8,9,1,0,10,2,4] 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 0x7f68c80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): picked mon.noname-g con 0x7f68c80a2870 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 0x7f68c8137a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): picked mon.noname-i con 0x7f68c80a3190 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8137f60 0x7f68c81383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): picked mon.noname-h con 0x7f68c8137f60 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c809ee70 con 0x7f68c80a2870 2024-04-20T02:22:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80673d0 con 0x7f68c8137f60 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8067610 con 0x7f68c80a3190 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68ce474700 10 monclient(hunting): authenticate will time out at 2040.573595s 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 0x7f68c80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c77fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 0x7f68c8137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c77fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 0x7f68c8137a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:47748/0 (socket says 172.21.15.47:47748) 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 0x7f68c80a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:50270/0 (socket says 172.21.15.47:50270) 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 1 -- 172.21.15.47:0/2938628353 learned_addr learned my addr 172.21.15.47:0/2938628353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 10 monclient(hunting): get_auth_request con 0x7f68c80a2870 auth_method 0 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68cca11700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8137f60 0x7f68c81383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c77fe700 10 monclient(hunting): get_auth_request con 0x7f68c80a3190 auth_method 0 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68cca11700 10 monclient(hunting): get_auth_request con 0x7f68c8137f60 auth_method 0 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68cca11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68cca11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.172+0000 7f68cca11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68cca11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68cca11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68cca11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 10 monclient(hunting): handle_auth_done global_id 22062 payload 274 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 -- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 msgr2=0x7f68c8137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 0x7f68c8137a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 -- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8137f60 msgr2=0x7f68c81383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8137f60 0x7f68c81383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 monclient: found mon.noname-g 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 -- 172.21.15.47:0/2938628353 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80751e0 con 0x7f68c80a2870 2024-04-20T02:22:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c77fe700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 0x7f68c8137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68cca11700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8137f60 0x7f68c81383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c7fff700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 0x7f68c80a2c50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009950 tx=0x7f68b8009c60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=17eef94e34a79665 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 1 -- 172.21.15.47:0/2938628353 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68b8011aa0 con 0x7f68c80a2870 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: dump: 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:41.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 1 -- 172.21.15.47:0/2938628353 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b8011c20 con 0x7f68c80a2870 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 1 -- 172.21.15.47:0/2938628353 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68b8011d90 con 0x7f68c80a2870 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68c6ffd700 10 monclient: dump: 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68ce474700 5 monclient: authenticate success, global_id 22062 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68ce474700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.173+0000 7f68ce474700 4 set_mon_vals no callback set 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 10 monclient: shutdown 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 -- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 msgr2=0x7f68c80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 0x7f68c80a2c50 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009950 tx=0x7f68b8009c60 comp rx=0 tx=0).stop 2024-04-20T02:22:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 -- 172.21.15.47:0/2938628353 shutdown_connections 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8137f60 0x7f68c81383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a2870 0x7f68c80a2c50 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 --2- 172.21.15.47:0/2938628353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a3190 0x7f68c8137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 -- 172.21.15.47:0/2938628353 >> 172.21.15.47:0/2938628353 conn(0x7f68c80a3f90 msgr2=0x7f68c806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 -- 172.21.15.47:0/2938628353 shutdown_connections 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.174+0000 7f68ce474700 1 -- 172.21.15.47:0/2938628353 wait complete. 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 10 monclient: build_initial_monmap 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 10 monclient: monmap: 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 1 librados: starting msgr at 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 1 librados: starting objecter 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 1 Processor -- start 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 1 -- start start 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 1 librados: setting wanted keys 2024-04-20T02:22:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 1 librados: calling monclient init 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.175+0000 7f68ce474700 10 monclient: init 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient: _add_conns ranks=[7,3,8,4,0,6,1,2,5] 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a2870 0x7f68c81cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): picked mon.noname-i con 0x7f68c80a2870 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a3190 0x7f68c81ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): picked mon.noname-g con 0x7f68c80a3190 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68c8137f60 0x7f68c81218c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): picked mon.noname-e con 0x7f68c8137f60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81249c0 con 0x7f68c80a3190 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8071fe0 con 0x7f68c80a2870 2024-04-20T02:22:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f68c809e510 con 0x7f68c8137f60 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68ce474700 10 monclient(hunting): authenticate will time out at 2040.577215s 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68c8137f60 0x7f68c81218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68c8137f60 0x7f68c81218c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:58368/0 (socket says 172.21.15.47:58368) 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 -- 172.21.15.47:0/838248089 learned_addr learned my addr 172.21.15.47:0/838248089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): get_auth_request con 0x7f68c8137f60 auth_method 0 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a2870 0x7f68c81cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): get_auth_request con 0x7f68c80a2870 auth_method 0 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a3190 0x7f68c81ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): get_auth_request con 0x7f68c80a3190 auth_method 0 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient(hunting): handle_auth_done global_id 35135 payload 824 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a2870 msgr2=0x7f68c81cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f68c80a2870 0x7f68c81cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a3190 msgr2=0x7f68c81ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68c80a3190 0x7f68c81ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient: _finish_hunting 0 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 monclient: found mon.noname-e 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00094a0 con 0x7f68c8137f60 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68cca11700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68c8137f60 0x7f68c81218c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f68c000efc0 tx=0x7f68c000c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c4ff9700 10 client.?.objecter ms_handle_connect 0x7f68c8137f60 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.176+0000 7f68c4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68c0014480 con 0x7f68c8137f60 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: dump: 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c000f040 con 0x7f68c8137f60 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 5 monclient: authenticate success, global_id 35135 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 10 monclient: _renew_subs 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c0009a70 con 0x7f68c8137f60 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68c0014a90 con 0x7f68c8137f60 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 10 client.35135.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 10 monclient: dump: 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c67fc700 4 set_mon_vals no callback set 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 10 monclient: _renew_subs 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8009550 con 0x7f68c8137f60 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68ce474700 1 librados: init done 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.177+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f68c0014bf0 con 0x7f68c8137f60 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68ce474700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68ce474700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 0x7f68c8121e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8068280 con 0x7f68c8125bd0 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68a803a2f0 0x7f68a803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f68c004b480 con 0x7f68c8137f60 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 10 client.35135.objecter ms_dispatch 0x7f68c8001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 3 client.35135.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 3 client.35135.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 0x7f68c8121e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient: get_auth_request con 0x7f68c8125bd0 auth_method 0 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 20 client.35135.objecter dump_active .. 0 homeless 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c77fe700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68a803a2f0 0x7f68a803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c77fe700 10 monclient: get_auth_request con 0x7f68a803a2f0 auth_method 0 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 10 monclient(hunting): handle_auth_done global_id 44492 payload 824 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68c809ddb0 con 0x7f68c8125bd0 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 10 client.35135.objecter ms_handle_connect 0x7f68a803a2f0 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c77fe700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68a803a2f0 0x7f68a803c7b0 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7f68c8123a60 tx=0x7f68b0009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 10 client.35135.objecter ms_handle_connect 0x7f68c8125bd0 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c4ff9700 10 client.35135.objecter resend_mon_ops 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.178+0000 7f68c7fff700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 0x7f68c8121e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009950 tx=0x7f68b8004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68b8020070 con 0x7f68c8125bd0 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 10 monclient: dump: 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.202 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f68b8004100 con 0x7f68c8125bd0 2024-04-20T02:22:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 msgr2=0x7f68c8121e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 0x7f68c8121e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009950 tx=0x7f68b8004750 comp rx=0 tx=0).stop 2024-04-20T02:22:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 msgr2=0x7f68c8121e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.179+0000 7f68c4ff9700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8125bd0 0x7f68c8121e00 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009950 tx=0x7f68b8004750 comp rx=0 tx=0).stop 2024-04-20T02:22:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68ce474700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68ce474700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 0x7f68c8122340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68ce474700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68c806a400 con 0x7f68c8124d60 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 0x7f68c8122340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient: get_auth_request con 0x7f68c8124d60 auth_method 0 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.188+0000 7f68c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c7fff700 10 monclient(hunting): handle_auth_done global_id 44501 payload 824 2024-04-20T02:22:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c7fff700 1 -- 172.21.15.47:0/838248089 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f68c80a35d0 con 0x7f68c8124d60 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c7fff700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 0x7f68c8122340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009130 tx=0x7f68b8011000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 client.35135.objecter ms_handle_connect 0x7f68c8124d60 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 client.35135.objecter resend_mon_ops 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68b802e300 con 0x7f68c8124d60 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 monclient: dump: 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.207 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f68b8003b90 con 0x7f68c8124d60 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 msgr2=0x7f68c8122340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 0x7f68c8122340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009130 tx=0x7f68b8011000 comp rx=0 tx=0).stop 2024-04-20T02:22:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 msgr2=0x7f68c8122340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.189+0000 7f68c4ff9700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f68c8124d60 0x7f68c8122340 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009130 tx=0x7f68b8011000 comp rx=0 tx=0).stop 2024-04-20T02:22:41.209 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:22:41.209 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:22:41.209 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 10 librados: watch_flush enter 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 10 librados: watch_flush exit 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 20 client.35135.objecter shutdown clearing up homeless session... 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 10 client.35135.objecter successfully canceled tick 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68a803a2f0 msgr2=0x7f68a803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68a803a2f0 0x7f68a803c7b0 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 crypto rx=0x7f68c8123a60 tx=0x7f68b0009210 comp rx=0 tx=0).stop 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 10 monclient: shutdown 2024-04-20T02:22:41.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68c8137f60 msgr2=0x7f68c81218c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 --2- 172.21.15.47:0/838248089 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68c8137f60 0x7f68c81218c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f68c000efc0 tx=0x7f68c000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68cca11700 1 -- 172.21.15.47:0/838248089 reap_dead start 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68cca11700 1 -- 172.21.15.47:0/838248089 reap_dead start 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 shutdown_connections 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 >> 172.21.15.47:0/838248089 conn(0x7f68c80a3f90 msgr2=0x7f68c80a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 shutdown_connections 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 -- 172.21.15.47:0/838248089 wait complete. 2024-04-20T02:22:41.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.190+0000 7f68ce474700 1 librados: shutdown 2024-04-20T02:22:41.212 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:22:41.213 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-04-20T02:22:41.213 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:22:41.227 INFO:tasks.ceph.mon.h:Stopped 2024-04-20T02:22:41.227 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-04-20T02:22:41.227 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:41.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.362+0000 7fb1c7287700 10 monclient: get_monmap_and_config 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.362+0000 7fb1c7287700 10 monclient: build_initial_monmap 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.362+0000 7fb1c7287700 10 monclient: monmap: 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:41.363844+0000 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:41.363844+0000 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:41.363 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.363+0000 7fb1c7287700 1 Processor -- start 2024-04-20T02:22:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.363+0000 7fb1c7287700 1 -- start start 2024-04-20T02:22:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.363+0000 7fb1c7287700 10 monclient: init 2024-04-20T02:22:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:41.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient: _add_conns ranks=[10,4,1,0,9,5,8,3,7,6,2] 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb1c00a1810 0x7fb1c0135d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): picked mon.noname-f con 0x7fb1c00a1810 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 0x7fb1c00a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): picked mon.noname-c con 0x7fb1c00a1bf0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb1c0070170 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00751e0 con 0x7fb1c0070170 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c009f450 con 0x7fb1c00a1810 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0071610 con 0x7fb1c00a1bf0 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c7287700 10 monclient(hunting): authenticate will time out at 2040.765202s 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 0x7fb1c00a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 0x7fb1c00a2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36020/0 (socket says 172.21.15.47:36020) 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 -- 172.21.15.47:0/1852280403 learned_addr learned my addr 172.21.15.47:0/1852280403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): get_auth_request con 0x7fb1c00a1bf0 auth_method 0 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c5023700 1 --2- 172.21.15.47:0/1852280403 >> v2:172.21.15.179:3300/0 conn(0x7fb1c00a1810 0x7fb1c0135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c5023700 10 monclient(hunting): get_auth_request con 0x7fb1c00a1810 auth_method 0 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c5023700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c5023700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c5023700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient(hunting): handle_auth_done global_id 54247 payload 274 2024-04-20T02:22:41.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 -- 172.21.15.47:0/1852280403 >> v2:172.21.15.179:3300/0 conn(0x7fb1c00a1810 msgr2=0x7fb1c0135d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 --2- 172.21.15.47:0/1852280403 >> v2:172.21.15.179:3300/0 conn(0x7fb1c00a1810 0x7fb1c0135d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 -- 172.21.15.47:0/1852280403 >> v1:172.21.15.179:6789/0 conn(0x7fb1c0070170 legacy=0x7fb1c0070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient: _finish_hunting 0 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 monclient: found mon.noname-c 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 -- 172.21.15.47:0/1852280403 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c0072430 con 0x7fb1c00a1bf0 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.364+0000 7fb1c4822700 1 --2- 172.21.15.47:0/1852280403 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 0x7fb1c00a2030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40098f0 tx=0x7fb1b4009c00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=dd4264ee6edaabe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 1 -- 172.21.15.47:0/1852280403 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1b40044a0 con 0x7fb1c00a1bf0 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: dump: 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.376 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 5 monclient: authenticate success, global_id 54247 2024-04-20T02:22:41.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 1 -- 172.21.15.47:0/1852280403 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b40036a0 con 0x7fb1c00a1bf0 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 4 set_mon_vals no callback set 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 1 -- 172.21.15.47:0/1852280403 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1b4003c80 con 0x7fb1c00a1bf0 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1b3fff700 10 monclient: dump: 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.378 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 10 monclient: shutdown 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 -- 172.21.15.47:0/1852280403 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 msgr2=0x7fb1c00a2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 --2- 172.21.15.47:0/1852280403 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 0x7fb1c00a2030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40098f0 tx=0x7fb1b4009c00 comp rx=0 tx=0).stop 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 -- 172.21.15.47:0/1852280403 shutdown_connections 2024-04-20T02:22:41.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 --2- 172.21.15.47:0/1852280403 >> v2:172.21.15.179:3300/0 conn(0x7fb1c00a1810 0x7fb1c0135d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 --2- 172.21.15.47:0/1852280403 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c00a1bf0 0x7fb1c00a2030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 -- 172.21.15.47:0/1852280403 >> 172.21.15.47:0/1852280403 conn(0x7fb1c00a5680 msgr2=0x7fb1c0066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 -- 172.21.15.47:0/1852280403 shutdown_connections 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.365+0000 7fb1c7287700 1 -- 172.21.15.47:0/1852280403 wait complete. 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 10 monclient: build_initial_monmap 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 10 monclient: monmap: 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:41.380 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 1 librados: starting msgr at 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 1 librados: starting objecter 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 1 Processor -- start 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 1 -- start start 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 1 librados: setting wanted keys 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 1 librados: calling monclient init 2024-04-20T02:22:41.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.366+0000 7fb1c7287700 10 monclient: init 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient: _add_conns ranks=[4,1,8,2,3,7,5,6,0] 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 0x7fb1c01cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): picked mon.noname-c con 0x7fb1c0070170 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c00a1810 0x7fb1c01ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): picked mon.noname-f con 0x7fb1c00a1810 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1c00a1bf0 0x7fb1c0123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): picked mon.noname-e con 0x7fb1c00a1bf0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0127150 con 0x7fb1c00a1810 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0124df0 con 0x7fb1c0070170 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:41.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0126e70 con 0x7fb1c00a1bf0 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c7287700 10 monclient(hunting): authenticate will time out at 2040.768325s 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 0x7fb1c01cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 0x7fb1c01cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36030/0 (socket says 172.21.15.47:36030) 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 -- 172.21.15.47:0/318488509 learned_addr learned my addr 172.21.15.47:0/318488509 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): get_auth_request con 0x7fb1c0070170 auth_method 0 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1c00a1bf0 0x7fb1c0123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c00a1810 0x7fb1c01ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): get_auth_request con 0x7fb1c00a1bf0 auth_method 0 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): get_auth_request con 0x7fb1c00a1810 auth_method 0 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5824700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c4822700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient(hunting): handle_auth_done global_id 54256 payload 824 2024-04-20T02:22:41.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 -- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1c00a1bf0 msgr2=0x7fb1c0123260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1c00a1bf0 0x7fb1c0123260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 -- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c00a1810 msgr2=0x7fb1c01ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c00a1810 0x7fb1c01ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient: _finish_hunting 0 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 monclient: found mon.noname-c 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 -- 172.21.15.47:0/318488509 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4009550 con 0x7fb1c0070170 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.367+0000 7fb1c5023700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 0x7fb1c01cc7c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00e810 tx=0x7fb1bc00eb20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 client.?.objecter ms_handle_connect 0x7fb1c0070170 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1bc00c070 con 0x7fb1c0070170 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: dump: 2024-04-20T02:22:41.385 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.386 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc013880 con 0x7fb1c0070170 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 5 monclient: authenticate success, global_id 54256 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 10 monclient: _renew_subs 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c009ed30 con 0x7fb1c0070170 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1bc013e60 con 0x7fb1c0070170 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b1ffb700 10 monclient: dump: 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:41.387 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1b37fe700 4 set_mon_vals no callback set 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 10 client.54256.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:41.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 10 monclient: _renew_subs 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c0125680 con 0x7fb1c0070170 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.368+0000 7fb1c7287700 1 librados: init done 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb1bc0139f0 con 0x7fb1c0070170 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb1a403a320 0x7fb1a403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7fb1bc04a7d0 con 0x7fb1c0070170 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 10 client.54256.objecter ms_dispatch 0x7fb1c0001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 3 client.54256.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 3 client.54256.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1c7287700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1c7287700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1c7287700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c01268b0 con 0x7fb1c0070170 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1b1ffb700 20 client.54256.objecter dump_active .. 0 homeless 2024-04-20T02:22:41.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1c4822700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb1a403a320 0x7fb1a403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:41.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.369+0000 7fb1c4822700 10 monclient: get_auth_request con 0x7fb1a403a320 auth_method 0 2024-04-20T02:22:41.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.370+0000 7fb1b1ffb700 10 client.54256.objecter ms_handle_connect 0x7fb1a403a320 2024-04-20T02:22:41.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.370+0000 7fb1c4822700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb1a403a320 0x7fb1a403c7e0 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009130 tx=0x7fb1b4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:41.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.374+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fb1bc017070 con 0x7fb1c0070170 2024-04-20T02:22:41.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.374+0000 7fb1b1ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:41.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.374+0000 7fb1b1ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.681+0000 7fb1c7287700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:41.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.681+0000 7fb1c7287700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.681+0000 7fb1c7287700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.681+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb1c00718a0 con 0x7fb1c0070170 2024-04-20T02:22:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.682+0000 7fb1b1ffb700 1 -- 172.21.15.47:0/318488509 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fb1bc02ac80 con 0x7fb1c0070170 2024-04-20T02:22:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.682+0000 7fb1b1ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:41.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.682+0000 7fb1b1ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:41.683 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 10 librados: watch_flush enter 2024-04-20T02:22:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 10 librados: watch_flush exit 2024-04-20T02:22:41.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 20 client.54256.objecter shutdown clearing up homeless session... 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 10 client.54256.objecter successfully canceled tick 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb1a403a320 msgr2=0x7fb1a403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb1a403a320 0x7fb1a403c7e0 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009130 tx=0x7fb1b4008030 comp rx=0 tx=0).stop 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 10 monclient: shutdown 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 msgr2=0x7fb1c01cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 0x7fb1c01cc7c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00e810 tx=0x7fb1bc00eb20 comp rx=0 tx=0).stop 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 shutdown_connections 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1c00a1bf0 0x7fb1c0123260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb1c0070170 0x7fb1c01cc7c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb1a403a320 0x7fb1a403c7e0 unknown :-1 s=CLOSED pgs=3195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 --2- 172.21.15.47:0/318488509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1c00a1810 0x7fb1c01ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 >> 172.21.15.47:0/318488509 conn(0x7fb1c00a5680 msgr2=0x7fb1c0135350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 shutdown_connections 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 -- 172.21.15.47:0/318488509 wait complete. 2024-04-20T02:22:41.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:41.684+0000 7fb1c7287700 1 librados: shutdown 2024-04-20T02:22:44.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.842+0000 7fec32067700 10 monclient: get_monmap_and_config 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.842+0000 7fec32067700 10 monclient: build_initial_monmap 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.842+0000 7fec32067700 10 monclient: monmap: 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:44.843239+0000 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:44.843239+0000 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:44.842 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:44.843 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.842+0000 7fec32067700 1 Processor -- start 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.842+0000 7fec32067700 1 -- start start 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.842+0000 7fec32067700 10 monclient: init 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient: _add_conns ranks=[9,3,6,8,2,10,5,1,7,0,4] 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec2c0a5080 0x7fec2c0a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): picked mon.noname-a con 0x7fec2c0a5080 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a1a80 0x7fec2c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): picked mon.noname-g con 0x7fec2c0a1a80 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 0x7fec2c0a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): picked mon.noname-d con 0x7fec2c0a3dc0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c072380 con 0x7fec2c0a5080 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0a5460 con 0x7fec2c0a1a80 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c070b10 con 0x7fec2c0a3dc0 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec32067700 10 monclient(hunting): authenticate will time out at 2044.244534s 2024-04-20T02:22:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 0x7fec2c0a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec2c0a5080 0x7fec2c0a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fec2c0a5080 0x7fec2c0a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38468/0 (socket says 172.21.15.47:38468) 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 0x7fec2c0a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57448/0 (socket says 172.21.15.47:57448) 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 1 -- 172.21.15.47:0/3274409 learned_addr learned my addr 172.21.15.47:0/3274409 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): get_auth_request con 0x7fec2c0a5080 auth_method 0 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): get_auth_request con 0x7fec2c0a3dc0 auth_method 0 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2affd700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a1a80 0x7fec2c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2affd700 10 monclient(hunting): get_auth_request con 0x7fec2c0a1a80 auth_method 0 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.843+0000 7fec2affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_done global_id 35505 payload 274 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 -- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 msgr2=0x7fec2c0a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 0x7fec2c0a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 -- 172.21.15.47:0/3274409 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a1a80 msgr2=0x7fec2c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a1a80 0x7fec2c0a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 monclient: found mon.noname-a 2024-04-20T02:22:44.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 -- 172.21.15.47:0/3274409 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0751e0 con 0x7fec2c0a5080 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2bfff700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 0x7fec2c0a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2b7fe700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a5080 0x7fec2c0a1540 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0098f0 tx=0x7fec1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe6fd518469c295 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 1 -- 172.21.15.47:0/3274409 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fec1c011aa0 con 0x7fec2c0a5080 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: dump: 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec32067700 5 monclient: authenticate success, global_id 35505 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec32067700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:44.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 1 -- 172.21.15.47:0/3274409 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c011c00 con 0x7fec2c0a5080 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec32067700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec32067700 4 set_mon_vals no callback set 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 1 -- 172.21.15.47:0/3274409 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fec1c011d70 con 0x7fec2c0a5080 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.844+0000 7fec2a7fc700 10 monclient: dump: 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 10 monclient: shutdown 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 -- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a5080 msgr2=0x7fec2c0a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a5080 0x7fec2c0a1540 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0098f0 tx=0x7fec1c009c00 comp rx=0 tx=0).stop 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 -- 172.21.15.47:0/3274409 shutdown_connections 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a5080 0x7fec2c0a1540 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fec2c0a3dc0 0x7fec2c0a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 --2- 172.21.15.47:0/3274409 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a1a80 0x7fec2c0a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 -- 172.21.15.47:0/3274409 >> 172.21.15.47:0/3274409 conn(0x7fec2c0a84a0 msgr2=0x7fec2c0a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 -- 172.21.15.47:0/3274409 shutdown_connections 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 1 -- 172.21.15.47:0/3274409 wait complete. 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 10 monclient: build_initial_monmap 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.845+0000 7fec32067700 10 monclient: monmap: 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:44.861 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 1 librados: starting msgr at 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 1 librados: starting objecter 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 1 Processor -- start 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 1 -- start start 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 1 librados: setting wanted keys 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 1 librados: calling monclient init 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.846+0000 7fec32067700 10 monclient: init 2024-04-20T02:22:44.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient: _add_conns ranks=[0,5,3,6,7,4,2,1,8] 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 0x7fec2c1ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): picked mon.noname-a con 0x7fec2c0a1a80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fec2c0a3dc0 0x7fec2c134b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): picked mon.noname-h con 0x7fec2c0a3dc0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a5080 0x7fec2c135070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): picked mon.noname-g con 0x7fec2c0a5080 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0712d0 con 0x7fec2c0a1a80 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c1cad10 con 0x7fec2c0a5080 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0a21e0 con 0x7fec2c0a3dc0 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:44.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient(hunting): authenticate will time out at 2044.248138s 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 0x7fec2c1ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 0x7fec2c1ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38478/0 (socket says 172.21.15.47:38478) 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 -- 172.21.15.47:0/2387653926 learned_addr learned my addr 172.21.15.47:0/2387653926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): get_auth_request con 0x7fec2c0a1a80 auth_method 0 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2affd700 1 -- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fec2c0a3dc0 msgr2=0x7fec2c134b30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2affd700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fec2c0a3dc0 0x7fec2c134b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2bfff700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a5080 0x7fec2c135070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient(hunting): handle_auth_done global_id 35514 payload 824 2024-04-20T02:22:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 -- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fec2c0a3dc0 msgr2=0x7fec2c134b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fec2c0a3dc0 0x7fec2c134b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 -- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a5080 msgr2=0x7fec2c135070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a5080 0x7fec2c135070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 monclient: found mon.noname-a 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 -- 172.21.15.47:0/2387653926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24009a70 con 0x7fec2c0a1a80 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2bfff700 10 monclient: get_auth_request con 0x7fec2c0a5080 auth_method 0 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2bfff700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a5080 0x7fec2c135070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec2b7fe700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 0x7fec2c1ca060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0a4840 tx=0x7fec1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 client.?.objecter ms_handle_connect 0x7fec2c0a1a80 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fec1c0045f0 con 0x7fec2c0a1a80 2024-04-20T02:22:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: dump: 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: _finish_auth 0 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 5 monclient: authenticate success, global_id 35514 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient: _renew_subs 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c003b10 con 0x7fec2c0a1a80 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c009550 con 0x7fec2c0a1a80 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fec1c003cc0 con 0x7fec2c0a1a80 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:44.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec32067700 10 client.35514.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec13fff700 10 monclient: dump: 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:44.868 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.847+0000 7fec29ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 10 monclient: _renew_subs 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c121a20 con 0x7fec2c0a1a80 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 1 librados: init done 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.848+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c0008d0 con 0x7fec2c0a1a80 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fec1c021070 con 0x7fec2c0a1a80 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec1803a320 0x7fec1803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7fec1c04bf80 con 0x7fec2c0a1a80 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 10 client.35514.objecter ms_dispatch 0x7fec2c001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:44.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 3 client.35514.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 3 client.35514.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec13fff700 20 client.35514.objecter dump_active .. 0 homeless 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec2affd700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec1803a320 0x7fec1803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.849+0000 7fec2affd700 10 monclient: get_auth_request con 0x7fec1803a320 auth_method 0 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.850+0000 7fec2affd700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec1803a320 0x7fec1803c7e0 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7fec20006f10 tx=0x7fec20006ea0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.850+0000 7fec13fff700 10 client.35514.objecter ms_handle_connect 0x7fec1803a320 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.854+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fec1c003e40 con 0x7fec2c0a1a80 2024-04-20T02:22:44.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.854+0000 7fec13fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:44.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:44.854+0000 7fec13fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec32067700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec32067700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec32067700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fec2c0a1f50 con 0x7fec2c0a1a80 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec13fff700 1 -- 172.21.15.47:0/2387653926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7fec1c015030 con 0x7fec2c0a1a80 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec13fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.163+0000 7fec13fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:45.164 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 10 librados: watch_flush enter 2024-04-20T02:22:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 10 librados: watch_flush exit 2024-04-20T02:22:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 20 client.35514.objecter shutdown clearing up homeless session... 2024-04-20T02:22:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 10 client.35514.objecter successfully canceled tick 2024-04-20T02:22:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec1803a320 msgr2=0x7fec1803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec1803a320 0x7fec1803c7e0 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 crypto rx=0x7fec20006f10 tx=0x7fec20006ea0 comp rx=0 tx=0).stop 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 10 monclient: shutdown 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 msgr2=0x7fec2c1ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.165+0000 7fec32067700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 0x7fec2c1ca060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0a4840 tx=0x7fec1c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 shutdown_connections 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec1803a320 0x7fec1803c7e0 unknown :-1 s=CLOSED pgs=3196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec2c0a1a80 0x7fec2c1ca060 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fec2c0a5080 0x7fec2c135070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 --2- 172.21.15.47:0/2387653926 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fec2c0a3dc0 0x7fec2c134b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 >> 172.21.15.47:0/2387653926 conn(0x7fec2c0a84a0 msgr2=0x7fec2c0a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 shutdown_connections 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 -- 172.21.15.47:0/2387653926 wait complete. 2024-04-20T02:22:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:45.166+0000 7fec32067700 1 librados: shutdown 2024-04-20T02:22:48.180 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.322+0000 7f58c8d9e700 10 monclient: get_monmap_and_config 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.322+0000 7f58c8d9e700 10 monclient: build_initial_monmap 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.323+0000 7f58c8d9e700 10 monclient: monmap: 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:48.324135+0000 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:48.324135+0000 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:48.324 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.323+0000 7f58c8d9e700 1 Processor -- start 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.323+0000 7f58c8d9e700 1 -- start start 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.323+0000 7f58c8d9e700 10 monclient: init 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient: _add_conns ranks=[3,0,7,5,2,10,8,9,4,1,6] 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58c40a5080 0x7f58c40a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): picked mon.noname-g con 0x7f58c40a5080 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f58c40a1a80 addr v1:172.21.15.47:6789/0 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58c40a3f90 0x7f58c413da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): picked mon.noname-i con 0x7f58c40a3f90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40751e0 con 0x7f58c40a1a80 2024-04-20T02:22:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40a5460 con 0x7f58c40a5080 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4070b10 con 0x7f58c40a3f90 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c8d9e700 10 monclient(hunting): authenticate will time out at 2047.725430s 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c1d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f58c40a1a80 0x7f58c40a3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:49120/0 (socket says 172.21.15.47:49120) 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c1d9b700 1 -- 172.21.15.47:0/4273760713 learned_addr learned my addr 172.21.15.47:0/4273760713 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58c40a5080 0x7f58c40a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58c40a3f90 0x7f58c413da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): get_auth_request con 0x7f58c40a5080 auth_method 0 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): get_auth_request con 0x7f58c40a3f90 auth_method 0 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.324+0000 7f58c2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 10 monclient(hunting): handle_auth_done global_id 22089 payload 274 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 1 -- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58c40a3f90 msgr2=0x7f58c413da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58c40a3f90 0x7f58c413da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 1 -- 172.21.15.47:0/4273760713 >> v1:172.21.15.47:6789/0 conn(0x7f58c40a1a80 legacy=0x7f58c40a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 10 monclient: _finish_hunting 0 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 1 monclient: found mon.noname-g 2024-04-20T02:22:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 1 -- 172.21.15.47:0/4273760713 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4136370 con 0x7f58c40a5080 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c2d9d700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58c40a3f90 0x7f58c413da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c259c700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58c40a5080 0x7f58c40a1540 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b4009b80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c1fd3d87a9b37724 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 1 -- 172.21.15.47:0/4273760713 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58b4011aa0 con 0x7f58c40a5080 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: dump: 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: _finish_auth 0 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: _check_auth_tickets 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c8d9e700 5 monclient: authenticate success, global_id 22089 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c8d9e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 1 -- 172.21.15.47:0/4273760713 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4011c00 con 0x7f58c40a5080 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c8d9e700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c8d9e700 4 set_mon_vals no callback set 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 1 -- 172.21.15.47:0/4273760713 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58b4011d70 con 0x7f58c40a5080 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.325+0000 7f58c159a700 10 monclient: dump: 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 10 monclient: shutdown 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 -- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58c40a5080 msgr2=0x7f58c40a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58c40a5080 0x7f58c40a1540 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b4009b80 comp rx=0 tx=0).stop 2024-04-20T02:22:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 -- 172.21.15.47:0/4273760713 shutdown_connections 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f58c40a3f90 0x7f58c413da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/4273760713 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f58c40a5080 0x7f58c40a1540 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 -- 172.21.15.47:0/4273760713 >> 172.21.15.47:0/4273760713 conn(0x7f58c40a84a0 msgr2=0x7f58c40a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 -- 172.21.15.47:0/4273760713 shutdown_connections 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.326+0000 7f58c8d9e700 1 -- 172.21.15.47:0/4273760713 wait complete. 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient: build_initial_monmap 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient: monmap: 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 librados: starting msgr at 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 librados: starting objecter 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 Processor -- start 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 -- start start 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 librados: setting wanted keys 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 librados: calling monclient init 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient: init 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient: _add_conns ranks=[1,8,5,4,0,2,7,6,3] 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58c40a1a80 0x7f58c41cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): picked mon.noname-f con 0x7f58c40a1a80 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 0x7f58c41ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): picked mon.noname-e con 0x7f58c40a3f90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58c40a5080 0x7f58c41232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): picked mon.noname-h con 0x7f58c40a5080 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4127270 con 0x7f58c40a1a80 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4071260 con 0x7f58c40a5080 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40a21e0 con 0x7f58c40a3f90 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.327+0000 7f58c8d9e700 10 monclient(hunting): authenticate will time out at 2047.728986s 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c2d9d700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58c40a5080 msgr2=0x7f58c41232a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c2d9d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58c40a5080 0x7f58c41232a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 0x7f58c41ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 0x7f58c41ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:36414/0 (socket says 172.21.15.47:36414) 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 -- 172.21.15.47:0/3143005667 learned_addr learned my addr 172.21.15.47:0/3143005667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): get_auth_request con 0x7f58c40a3f90 auth_method 0 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58c40a1a80 0x7f58c41cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): get_auth_request con 0x7f58c40a1a80 auth_method 0 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient(hunting): handle_auth_done global_id 35153 payload 824 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 -- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58c40a5080 msgr2=0x7f58c41232a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58c40a5080 0x7f58c41232a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 -- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58c40a1a80 msgr2=0x7f58c41cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58c40a1a80 0x7f58c41cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 monclient: found mon.noname-e 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:48.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 -- 172.21.15.47:0/3143005667 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc009a70 con 0x7f58c40a3f90 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c1d9b700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 0x7f58c41ccd80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58b800c3f0 tx=0x7f58b800c700 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 client.?.objecter ms_handle_connect 0x7f58c40a3f90 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58b800ce70 con 0x7f58c40a3f90 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: dump: 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:48.345 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b8001100 con 0x7f58c40a3f90 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 5 monclient: authenticate success, global_id 35153 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58b8007540 con 0x7f58c40a3f90 2024-04-20T02:22:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 10 monclient: _renew_subs 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b40094a0 con 0x7f58c40a3f90 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c0d99700 4 set_mon_vals no callback set 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58b2ffd700 10 monclient: dump: 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:48.347 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 10 client.35153.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 10 monclient: _renew_subs 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c4126960 con 0x7f58c40a3f90 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.328+0000 7f58c8d9e700 1 librados: init done 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.329+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f58b80076c0 con 0x7f58c40a3f90 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.329+0000 7f58c8d9e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.329+0000 7f58c8d9e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:48.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.329+0000 7f58c8d9e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.329+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c41270d0 con 0x7f58c40a3f90 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f58a403a320 0x7f58a403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f58b8049920 con 0x7f58c40a3f90 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 10 client.35153.objecter ms_dispatch 0x7f58c4001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 3 client.35153.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 3 client.35153.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 20 client.35153.objecter dump_active .. 0 homeless 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58c259c700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f58a403a320 0x7f58a403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58c259c700 10 monclient: get_auth_request con 0x7f58a403a320 auth_method 0 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58c259c700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f58a403a320 0x7f58a403c7e0 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f58b40090b0 tx=0x7f58b401d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:48.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.330+0000 7f58b2ffd700 10 client.35153.objecter ms_handle_connect 0x7f58a403a320 2024-04-20T02:22:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.335+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f58c41270d0 con 0x7f58c40a3f90 2024-04-20T02:22:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.335+0000 7f58b2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:48.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.335+0000 7f58b2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58c8d9e700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58c8d9e700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:48.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58c8d9e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f58c41270d0 con 0x7f58c40a3f90 2024-04-20T02:22:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58b2ffd700 1 -- 172.21.15.47:0/3143005667 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f58c41270d0 con 0x7f58c40a3f90 2024-04-20T02:22:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58b2ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:48.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.645+0000 7f58b2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:48.646 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":52,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 10 librados: watch_flush enter 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 10 librados: watch_flush exit 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 20 client.35153.objecter shutdown clearing up homeless session... 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 10 client.35153.objecter successfully canceled tick 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f58a403a320 msgr2=0x7f58a403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f58a403a320 0x7f58a403c7e0 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 crypto rx=0x7f58b40090b0 tx=0x7f58b401d040 comp rx=0 tx=0).stop 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 10 monclient: shutdown 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 msgr2=0x7f58c41ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 0x7f58c41ccd80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58b800c3f0 tx=0x7f58b800c700 comp rx=0 tx=0).stop 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 shutdown_connections 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f58c40a5080 0x7f58c41232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f58a403a320 0x7f58a403c7e0 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58c40a1a80 0x7f58c41cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 --2- 172.21.15.47:0/3143005667 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f58c40a3f90 0x7f58c41ccd80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 >> 172.21.15.47:0/3143005667 conn(0x7f58c40a84a0 msgr2=0x7f58c4134ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 shutdown_connections 2024-04-20T02:22:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 -- 172.21.15.47:0/3143005667 wait complete. 2024-04-20T02:22:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:48.648+0000 7f58c8d9e700 1 librados: shutdown 2024-04-20T02:22:51.663 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:22:51.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.806+0000 7ff42562d700 10 monclient: get_monmap_and_config 2024-04-20T02:22:51.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.806+0000 7ff42562d700 10 monclient: build_initial_monmap 2024-04-20T02:22:51.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.806+0000 7ff42562d700 10 monclient: monmap: 2024-04-20T02:22:51.805 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:51.805 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:51.805 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:51.806576+0000 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:51.806576+0000 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:51.806 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 Processor -- start 2024-04-20T02:22:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 -- start start 2024-04-20T02:22:51.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient: init 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient: _add_conns ranks=[9,6,7,5,10,2,0,4,8,1,3] 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): picked mon.noname-a con 0x7ff4200a4c80 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): picked mon.noname-d con 0x7ff4200a0d20 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): picked mon.noname-i con 0x7ff4200a55a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200a0000 con 0x7ff4200a4c80 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200a6a10 con 0x7ff4200a0d20 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:51.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff42009e440 con 0x7ff4200a55a0 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff42562d700 10 monclient(hunting): authenticate will time out at 2051.207708s 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38482/0 (socket says 172.21.15.47:38482) 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 learned_addr learned my addr 172.21.15.47:0/30131200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57462/0 (socket says 172.21.15.47:57462) 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff4200a4c80 auth_method 0 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff4200a0d20 auth_method 0 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:51.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.807+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff4200a55a0 auth_method 0 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:51.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:51.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:51.808+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff4200a0d20 auth_method 0 2024-04-20T02:22:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:52.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff4200a4c80 auth_method 0 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:52.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff4200a55a0 auth_method 0 2024-04-20T02:22:52.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:52.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:52.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:52.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:52.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:52.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:52.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.009+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff4200a4c80 auth_method 0 2024-04-20T02:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:52.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff4200a0d20 auth_method 0 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:52.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff4200a55a0 auth_method 0 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:52.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:52.410+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff4200a4c80 auth_method 0 2024-04-20T02:22:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:53.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff4200a0d20 auth_method 0 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.211+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff4200a55a0 auth_method 0 2024-04-20T02:22:53.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:53.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:53.212+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 10 monclient(hunting): tick 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 monclient(hunting): continuing hunt 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 msgr2=0x7ff420137a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff4200a55a0 0x7ff420137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 msgr2=0x7ff4200a5060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff4200a0d20 0x7ff4200a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:54.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 msgr2=0x7ff4200a07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.47:3300/0 conn(0x7ff4200a4c80 0x7ff4200a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 10 monclient: _add_conns ranks=[4,10,2,3,0,8,5,1] 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.807+0000 7ff41cff9700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): picked mon.noname-c con 0x7ff40c003860 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): picked mon.noname-f con 0x7ff40c0061f0 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): picked mon.noname-b con 0x7ff40c008ce0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 1 -- 172.21.15.47:0/30131200 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200a0000 con 0x7ff40c0061f0 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 1 -- 172.21.15.47:0/30131200 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200a6a10 con 0x7ff40c008ce0 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:54.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 1 -- 172.21.15.47:0/30131200 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff42009e440 con 0x7ff40c003860 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41cff9700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff40c003860 auth_method 0 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff40c008ce0 auth_method 0 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:54.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff40c0061f0 auth_method 0 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:54.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:54.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:54.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:54.808+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff40c008ce0 auth_method 0 2024-04-20T02:22:55.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff40c003860 auth_method 0 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:55.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff40c0061f0 auth_method 0 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:55.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.009+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:22:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff40c003860 auth_method 0 2024-04-20T02:22:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff40c008ce0 auth_method 0 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff40c0061f0 auth_method 0 2024-04-20T02:22:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:22:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:55.409+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff40c003860 auth_method 0 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff40c008ce0 auth_method 0 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:56.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff40c0061f0 auth_method 0 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.210+0000 7ff41e7fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 10 monclient(hunting): handle_auth_done global_id 54265 payload 274 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 msgr2=0x7ff40c00b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 msgr2=0x7ff40c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> v2:172.21.15.179:3300/0 conn(0x7ff40c0061f0 0x7ff40c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 10 monclient: _finish_hunting 0 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 monclient: found mon.noname-c 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 -- 172.21.15.47:0/30131200 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200751e0 con 0x7ff40c003860 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff40c008ce0 0x7ff40c00b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41f7fe700 1 -- 172.21.15.47:0/30131200 reap_dead start 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41effd700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff410009180 tx=0x7ff410018e60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=cfee991333505e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 1 -- 172.21.15.47:0/30131200 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff41001d070 con 0x7ff40c003860 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: dump: 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.222 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff42562d700 5 monclient: authenticate success, global_id 54265 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff42562d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 1 -- 172.21.15.47:0/30131200 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41000c040 con 0x7ff40c003860 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff42562d700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff42562d700 4 set_mon_vals no callback set 2024-04-20T02:22:56.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 1 -- 172.21.15.47:0/30131200 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff410016d30 con 0x7ff40c003860 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.211+0000 7ff41dffb700 10 monclient: dump: 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 10 monclient: shutdown 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 -- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 msgr2=0x7ff40c005c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff410009180 tx=0x7ff410018e60 comp rx=0 tx=0).stop 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 -- 172.21.15.47:0/30131200 shutdown_connections 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 --2- 172.21.15.47:0/30131200 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff40c003860 0x7ff40c005c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 -- 172.21.15.47:0/30131200 >> 172.21.15.47:0/30131200 conn(0x7ff4200aa280 msgr2=0x7ff4200aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 -- 172.21.15.47:0/30131200 shutdown_connections 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.212+0000 7ff42562d700 1 -- 172.21.15.47:0/30131200 wait complete. 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.213+0000 7ff42562d700 10 monclient: build_initial_monmap 2024-04-20T02:22:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.213+0000 7ff42562d700 10 monclient: monmap: 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:56.226 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 1 librados: starting msgr at 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 1 librados: starting objecter 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 1 Processor -- start 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 1 -- start start 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 1 librados: setting wanted keys 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 1 librados: calling monclient init 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.214+0000 7ff42562d700 10 monclient: init 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient: _add_conns ranks=[8,0,4,2,1,3,7,6,5] 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff40c003860 0x7ff4201cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): picked mon.noname-e con 0x7ff40c003860 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:56.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 0x7ff4201bcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): picked mon.noname-a con 0x7ff4201bc9c0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 0x7ff4201a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): picked mon.noname-c con 0x7ff4201bd320 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201aa380 con 0x7ff4201bc9c0 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201aa110 con 0x7ff4201bd320 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff42009ea00 con 0x7ff40c003860 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff42562d700 10 monclient(hunting): authenticate will time out at 2055.616164s 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 0x7ff4201a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 0x7ff4201bcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff40c003860 0x7ff4201cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 0x7ff4201a6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:38666/0 (socket says 172.21.15.47:38666) 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff40c003860 0x7ff4201cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:44198/0 (socket says 172.21.15.47:44198) 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 -- 172.21.15.47:0/1134021183 learned_addr learned my addr 172.21.15.47:0/1134021183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request con 0x7ff4201bd320 auth_method 0 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 0x7ff4201bcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34952/0 (socket says 172.21.15.47:34952) 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): get_auth_request con 0x7ff40c003860 auth_method 0 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request con 0x7ff4201bc9c0 auth_method 0 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient(hunting): handle_auth_done global_id 54274 payload 824 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 -- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff40c003860 msgr2=0x7ff4201cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff40c003860 0x7ff4201cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 -- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 msgr2=0x7ff4201bcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 0x7ff4201bcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 monclient: found mon.noname-c 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41f7fe700 1 -- 172.21.15.47:0/1134021183 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410018860 con 0x7ff4201bd320 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.215+0000 7ff41e7fc700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 0x7ff4201bcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff41f7fe700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 0x7ff4201a6420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff418009610 tx=0x7ff418018650 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 client.?.objecter ms_handle_connect 0x7ff4201bd320 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff41801e070 con 0x7ff4201bd320 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: dump: 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.232 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41800f040 con 0x7ff4201bd320 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 5 monclient: authenticate success, global_id 54274 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 10 monclient: _renew_subs 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42009dd20 con 0x7ff4201bd320 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:56.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff418022ca0 con 0x7ff4201bd320 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 10 client.54274.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff41d7fa700 4 set_mon_vals no callback set 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff4177fe700 10 monclient: dump: 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.234 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.235 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 10 monclient: _renew_subs 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4201a9ae0 con 0x7ff4201bd320 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.216+0000 7ff42562d700 1 librados: init done 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff418018cc0 con 0x7ff4201bd320 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff42562d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff42562d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff42562d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4201aa4c0 con 0x7ff4201bd320 2024-04-20T02:22:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff40c03b470 0x7ff40c03d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7ff41804c720 con 0x7ff4201bd320 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 10 client.54274.objecter ms_dispatch 0x7ff420001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 3 client.54274.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 3 client.54274.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.217+0000 7ff4177fe700 20 client.54274.objecter dump_active .. 0 homeless 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.218+0000 7ff41effd700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff40c03b470 0x7ff40c03d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.218+0000 7ff41effd700 10 monclient: get_auth_request con 0x7ff40c03b470 auth_method 0 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.218+0000 7ff41effd700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff40c03b470 0x7ff40c03d930 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7ff410018b50 tx=0x7ff410019000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.218+0000 7ff4177fe700 10 client.54274.objecter ms_handle_connect 0x7ff40c03b470 2024-04-20T02:22:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.223+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ff4201aa4c0 con 0x7ff4201bd320 2024-04-20T02:22:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.223+0000 7ff4177fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.223+0000 7ff4177fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:56.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.528+0000 7ff42562d700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:22:56.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.529+0000 7ff42562d700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:56.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.529+0000 7ff42562d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.529+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff4201aa4c0 con 0x7ff4201bd320 2024-04-20T02:22:56.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.529+0000 7ff4177fe700 1 -- 172.21.15.47:0/1134021183 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3169 (secure 0 0 0) 0x7ff4201aa4c0 con 0x7ff4201bd320 2024-04-20T02:22:56.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.529+0000 7ff4177fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:22:56.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.529+0000 7ff4177fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:56.530 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":60,"quorum":[0,1,2,3,4,6,7,8],"quorum_names":["a","f","b","g","c","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 10 librados: watch_flush enter 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 10 librados: watch_flush exit 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 20 client.54274.objecter shutdown clearing up homeless session... 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 10 client.54274.objecter successfully canceled tick 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff40c03b470 msgr2=0x7ff40c03d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff40c03b470 0x7ff40c03d930 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 crypto rx=0x7ff410018b50 tx=0x7ff410019000 comp rx=0 tx=0).stop 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 10 monclient: shutdown 2024-04-20T02:22:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 msgr2=0x7ff4201a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.531+0000 7ff42562d700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 0x7ff4201a6420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff418009610 tx=0x7ff418018650 comp rx=0 tx=0).stop 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 shutdown_connections 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff40c03b470 0x7ff40c03d930 unknown :-1 s=CLOSED pgs=3198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff4201bd320 0x7ff4201a6420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff40c003860 0x7ff4201cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 --2- 172.21.15.47:0/1134021183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4201bc9c0 0x7ff4201bcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 >> 172.21.15.47:0/1134021183 conn(0x7ff4200aa280 msgr2=0x7ff4201bfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 shutdown_connections 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 -- 172.21.15.47:0/1134021183 wait complete. 2024-04-20T02:22:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.532+0000 7ff42562d700 1 librados: shutdown 2024-04-20T02:22:56.545 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-04-20T02:22:56.545 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-04-20T02:22:56.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:22:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.682+0000 7faf12db2700 10 monclient: get_monmap_and_config 2024-04-20T02:22:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.682+0000 7faf12db2700 10 monclient: build_initial_monmap 2024-04-20T02:22:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.682+0000 7faf12db2700 10 monclient: monmap: 2024-04-20T02:22:56.682 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:56.683791+0000 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:56.683791+0000 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:56.683 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:56.684 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:56.684 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.683+0000 7faf12db2700 1 Processor -- start 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.683+0000 7faf12db2700 1 -- start start 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.683+0000 7faf12db2700 10 monclient: init 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient: _add_conns ranks=[8,7,4,1,10,6,3,9,0,5,2] 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 0x7faf0c135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): picked mon.noname-e con 0x7faf0c09fa80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf0c09fe60 0x7faf0c0a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): picked mon.noname-i con 0x7faf0c09fe60 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 0x7faf0c0a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): picked mon.noname-c con 0x7faf0c0a1f10 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0a0780 con 0x7faf0c0a1f10 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c09df90 con 0x7faf0c09fe60 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c09dcb0 con 0x7faf0c09fa80 2024-04-20T02:22:56.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf12db2700 10 monclient(hunting): authenticate will time out at 2056.085178s 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 0x7faf0c135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 0x7faf0c0a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 0x7faf0c135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:44202/0 (socket says 172.21.15.47:44202) 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 0x7faf0c0a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:38682/0 (socket says 172.21.15.47:38682) 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 -- 172.21.15.47:0/955473416 learned_addr learned my addr 172.21.15.47:0/955473416 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): get_auth_request con 0x7faf0c09fa80 auth_method 0 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): get_auth_request con 0x7faf0c0a1f10 auth_method 0 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf0bfff700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf0c09fe60 0x7faf0c0a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf0bfff700 10 monclient(hunting): get_auth_request con 0x7faf0c09fe60 auth_method 0 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf0bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf0bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient(hunting): handle_auth_done global_id 35171 payload 274 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 -- 172.21.15.47:0/955473416 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf0c09fe60 msgr2=0x7faf0c0a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf0c09fe60 0x7faf0c0a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 -- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 msgr2=0x7faf0c0a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 0x7faf0c0a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient: _finish_hunting 0 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 monclient: found mon.noname-e 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 -- 172.21.15.47:0/955473416 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0751e0 con 0x7faf0c09fa80 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf1134f700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 0x7faf0c0a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.684+0000 7faf10b4e700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 0x7faf0c135df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf00009920 tx=0x7faf00009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f00e677ec323b016 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 1 -- 172.21.15.47:0/955473416 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf00011aa0 con 0x7faf0c09fa80 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: dump: 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.699 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:56.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 1 -- 172.21.15.47:0/955473416 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf00011c00 con 0x7faf0c09fa80 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 5 monclient: authenticate success, global_id 35171 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 1 -- 172.21.15.47:0/955473416 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf00011d70 con 0x7faf0c09fa80 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 4 set_mon_vals no callback set 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf0b7fe700 10 monclient: dump: 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.701 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 10 monclient: shutdown 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:56.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 -- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 msgr2=0x7faf0c135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 0x7faf0c135df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faf00009920 tx=0x7faf00009c30 comp rx=0 tx=0).stop 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 -- 172.21.15.47:0/955473416 shutdown_connections 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faf0c0a1f10 0x7faf0c0a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7faf0c09fa80 0x7faf0c135df0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 --2- 172.21.15.47:0/955473416 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7faf0c09fe60 0x7faf0c0a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 -- 172.21.15.47:0/955473416 >> 172.21.15.47:0/955473416 conn(0x7faf0c0a3000 msgr2=0x7faf0c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 -- 172.21.15.47:0/955473416 shutdown_connections 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.685+0000 7faf12db2700 1 -- 172.21.15.47:0/955473416 wait complete. 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 10 monclient: build_initial_monmap 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 10 monclient: monmap: 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:56.703 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 1 librados: starting msgr at 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 1 librados: starting objecter 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 1 Processor -- start 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 1 -- start start 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 1 librados: setting wanted keys 2024-04-20T02:22:56.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 1 librados: calling monclient init 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.686+0000 7faf12db2700 10 monclient: init 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient: _add_conns ranks=[3,6,5,7,0,8,1,4,2] 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf0c09fa80 0x7faf0c09a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): picked mon.noname-g con 0x7faf0c09fa80 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf0c09fe60 0x7faf0c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): picked mon.noname-d con 0x7faf0c09fe60 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faf0c0a1f10 0x7faf0c09af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): picked mon.noname-h con 0x7faf0c0a1f10 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c134920 con 0x7faf0c09fa80 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c132a10 con 0x7faf0c0a1f10 2024-04-20T02:22:56.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c132c60 con 0x7faf0c09fe60 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf12db2700 10 monclient(hunting): authenticate will time out at 2056.088681s 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf1134f700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faf0c0a1f10 msgr2=0x7faf0c09af80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf1134f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faf0c0a1f10 0x7faf0c09af80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf0c09fe60 0x7faf0c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf0c09fa80 0x7faf0c09a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf0c09fe60 0x7faf0c09aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:36202/0 (socket says 172.21.15.47:36202) 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 1 -- 172.21.15.47:0/1647108159 learned_addr learned my addr 172.21.15.47:0/1647108159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:56.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): get_auth_request con 0x7faf0c09fe60 auth_method 0 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 10 monclient(hunting): get_auth_request con 0x7faf0c09fa80 auth_method 0 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf0bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.687+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 10 monclient(hunting): handle_auth_done global_id 54294 payload 824 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faf0c0a1f10 msgr2=0x7faf0c09af80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:56.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faf0c0a1f10 0x7faf0c09af80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf0c09fa80 msgr2=0x7faf0c09a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7faf0c09fa80 0x7faf0c09a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 monclient: found mon.noname-d 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009a70 con 0x7faf0c09fe60 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0bfff700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf0c09fe60 0x7faf0c09aa40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faef800e870 tx=0x7faef800eb80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 client.?.objecter ms_handle_connect 0x7faf0c09fe60 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faef8013880 con 0x7faf0c09fe60 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: dump: 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.708 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: _finish_auth 0 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: _check_auth_tickets 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8013a20 con 0x7faf0c09fe60 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 5 monclient: authenticate success, global_id 54294 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 10 monclient: _renew_subs 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf00009550 con 0x7faf0c09fe60 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faef8012370 con 0x7faf0c09fe60 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf097fa700 10 monclient: dump: 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.711 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf0affd700 4 set_mon_vals no callback set 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 10 client.54294.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 10 monclient: _renew_subs 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c09cb50 con 0x7faf0c09fe60 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.688+0000 7faf12db2700 1 librados: init done 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf12db2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf12db2700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 0x7faf0c09b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c068180 con 0x7faf0c06aac0 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 0x7faf0c09b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient: get_auth_request con 0x7faf0c06aac0 auth_method 0 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7faef8013b90 con 0x7faf0c09fe60 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 10 monclient(hunting): handle_auth_done global_id 35532 payload 824 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf10b4e700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf0c0a2680 con 0x7faf0c06aac0 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf097fa700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faef003a2f0 0x7faef003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7faef804a530 con 0x7faf0c09fe60 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.689+0000 7faf097fa700 10 client.54294.objecter ms_dispatch 0x7faf0c001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 3 client.54294.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 3 client.54294.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf10b4e700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 0x7faf0c09b4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf00009920 tx=0x7faf00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf1134f700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faef003a2f0 0x7faef003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf1134f700 10 monclient: get_auth_request con 0x7faef003a2f0 auth_method 0 2024-04-20T02:22:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 20 client.54294.objecter dump_active .. 0 homeless 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 client.54294.objecter ms_handle_connect 0x7faf0c06aac0 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 client.54294.objecter resend_mon_ops 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf00020070 con 0x7faf0c06aac0 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 monclient: dump: 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.714 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faf000044f0 con 0x7faf0c06aac0 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 msgr2=0x7faf0c09b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 0x7faf0c09b4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf00009920 tx=0x7faf00004750 comp rx=0 tx=0).stop 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 msgr2=0x7faf0c09b4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf1134f700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faef003a2f0 0x7faef003c7b0 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 crypto rx=0x7faf0c09bb40 tx=0x7faf0400d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06aac0 0x7faf0c09b4c0 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faf00009920 tx=0x7faf00004750 comp rx=0 tx=0).stop 2024-04-20T02:22:56.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.690+0000 7faf097fa700 10 client.54294.objecter ms_handle_connect 0x7faef003a2f0 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf12db2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf12db2700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 0x7faf0c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf12db2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c06a6c0 con 0x7faf0c06ff40 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 0x7faf0c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient: get_auth_request con 0x7faf0c06ff40 auth_method 0 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 10 monclient(hunting): handle_auth_done global_id 35541 payload 824 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 1 -- 172.21.15.47:0/1647108159 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faf0c0a0930 con 0x7faf0c06ff40 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf097fa700 10 client.54294.objecter ms_handle_connect 0x7faf0c06ff40 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf097fa700 10 client.54294.objecter resend_mon_ops 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.698+0000 7faf10b4e700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 0x7faf0c070350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c70 tx=0x7faf00011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faf0002bd00 con 0x7faf0c06ff40 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 10 monclient: dump: 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.719 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9512 (secure 0 0 0) 0x7faf0002be60 con 0x7faf0c06ff40 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 msgr2=0x7faf0c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 0x7faf0c070350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c70 tx=0x7faf00011000 comp rx=0 tx=0).stop 2024-04-20T02:22:56.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 msgr2=0x7faf0c070350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf097fa700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faf0c06ff40 0x7faf0c070350 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c70 tx=0x7faf00011000 comp rx=0 tx=0).stop 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:56.721 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:56.722 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:56.723 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:56.724 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:56.725 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:56.727 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:56.728 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.731 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.734 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:56.735 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.737 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:56.738 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.739 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:56.740 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:56.741 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 10 librados: watch_flush enter 2024-04-20T02:22:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 10 librados: watch_flush exit 2024-04-20T02:22:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 20 client.54294.objecter shutdown clearing up homeless session... 2024-04-20T02:22:56.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 10 client.54294.objecter successfully canceled tick 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faef003a2f0 msgr2=0x7faef003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faef003a2f0 0x7faef003c7b0 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 crypto rx=0x7faf0c09bb40 tx=0x7faf0400d040 comp rx=0 tx=0).stop 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 10 monclient: shutdown 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf0c09fe60 msgr2=0x7faf0c09aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf12db2700 1 --2- 172.21.15.47:0/1647108159 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faf0c09fe60 0x7faf0c09aa40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faef800e870 tx=0x7faef800eb80 comp rx=0 tx=0).stop 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf1134f700 1 -- 172.21.15.47:0/1647108159 reap_dead start 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.699+0000 7faf1134f700 1 -- 172.21.15.47:0/1647108159 reap_dead start 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.700+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 shutdown_connections 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.700+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 >> 172.21.15.47:0/1647108159 conn(0x7faf0c0a3000 msgr2=0x7faf0c1bd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.700+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 shutdown_connections 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.700+0000 7faf12db2700 1 -- 172.21.15.47:0/1647108159 wait complete. 2024-04-20T02:22:56.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.700+0000 7faf12db2700 1 librados: shutdown 2024-04-20T02:22:56.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:56.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-04-20T02:22:56.745 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.883+0000 7f397aa03700 10 monclient: get_monmap_and_config 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.883+0000 7f397aa03700 10 monclient: build_initial_monmap 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.883+0000 7f397aa03700 10 monclient: monmap: 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:56.884791+0000 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:56.884791+0000 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.898 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.884+0000 7f397aa03700 1 Processor -- start 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.884+0000 7f397aa03700 1 -- start start 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.884+0000 7f397aa03700 10 monclient: init 2024-04-20T02:22:56.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient: _add_conns ranks=[7,3,9,8,5,2,4,1,6,10,0] 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f397406cf40 0x7f39740a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): picked mon.noname-i con 0x7f397406cf40 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f39740a2db0 0x7f39740a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): picked mon.noname-g con 0x7f39740a2db0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3974134690 0x7f3974134af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): picked mon.noname-a con 0x7f3974134690 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39740a36f0 con 0x7f3974134690 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39740a0240 con 0x7f39740a2db0 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f397406d320 con 0x7f397406cf40 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:56.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f397aa03700 10 monclient(hunting): authenticate will time out at 2056.286149s 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3974134690 0x7f3974134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3974134690 0x7f3974134af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34992/0 (socket says 172.21.15.47:34992) 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 -- 172.21.15.47:0/3381050188 learned_addr learned my addr 172.21.15.47:0/3381050188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): get_auth_request con 0x7f3974134690 auth_method 0 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f39737fe700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f39740a2db0 0x7f39740a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3973fff700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f397406cf40 0x7f39740a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f39737fe700 10 monclient(hunting): get_auth_request con 0x7f39740a2db0 auth_method 0 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f39737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f39737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f39737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3973fff700 10 monclient(hunting): get_auth_request con 0x7f397406cf40 auth_method 0 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3973fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3973fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3973fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient(hunting): handle_auth_done global_id 35550 payload 274 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 -- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f397406cf40 msgr2=0x7f39740a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f397406cf40 0x7f39740a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 -- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f39740a2db0 msgr2=0x7f39740a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f39740a2db0 0x7f39740a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient: _finish_hunting 0 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 monclient: found mon.noname-a 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 -- 172.21.15.47:0/3381050188 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740751e0 con 0x7f3974134690 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.885+0000 7f3978fa0700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974134690 0x7f3974134af0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f396c00d150 tx=0x7f396c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6b4e53544d9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 1 -- 172.21.15.47:0/3381050188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f396c010070 con 0x7f3974134690 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: dump: 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.903 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 5 monclient: authenticate success, global_id 35550 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 1 -- 172.21.15.47:0/3381050188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c00ddc0 con 0x7f3974134690 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 1 -- 172.21.15.47:0/3381050188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f396c009a70 con 0x7f3974134690 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 4 set_mon_vals no callback set 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f3972ffd700 10 monclient: dump: 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 10 monclient: shutdown 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 -- 172.21.15.47:0/3381050188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974134690 msgr2=0x7f3974134af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974134690 0x7f3974134af0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f396c00d150 tx=0x7f396c00d460 comp rx=0 tx=0).stop 2024-04-20T02:22:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 -- 172.21.15.47:0/3381050188 shutdown_connections 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974134690 0x7f3974134af0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f397406cf40 0x7f39740a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 --2- 172.21.15.47:0/3381050188 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f39740a2db0 0x7f39740a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 -- 172.21.15.47:0/3381050188 >> 172.21.15.47:0/3381050188 conn(0x7f39740a4010 msgr2=0x7f397406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 -- 172.21.15.47:0/3381050188 shutdown_connections 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.886+0000 7f397aa03700 1 -- 172.21.15.47:0/3381050188 wait complete. 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.887+0000 7f397aa03700 10 monclient: build_initial_monmap 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.887+0000 7f397aa03700 10 monclient: monmap: 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:56.907 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.887+0000 7f397aa03700 1 librados: starting msgr at 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.887+0000 7f397aa03700 1 librados: starting objecter 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.887+0000 7f397aa03700 1 Processor -- start 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 -- start start 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 librados: setting wanted keys 2024-04-20T02:22:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 librados: calling monclient init 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient: init 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient: _add_conns ranks=[4,1,5,3,2,7,0,6,8] 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f397406cf40 0x7f39741c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): picked mon.noname-c con 0x7f397406cf40 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39740a2db0 0x7f397412e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): picked mon.noname-f con 0x7f39740a2db0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3974134690 0x7f397412eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): picked mon.noname-h con 0x7f3974134690 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39741c8bf0 con 0x7f39740a2db0 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f397411d6e0 con 0x7f397406cf40 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f397409f370 con 0x7f3974134690 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f397aa03700 10 monclient(hunting): authenticate will time out at 2056.289650s 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3978fa0700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3974134690 msgr2=0x7f397412eae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f397406cf40 0x7f39741c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3978fa0700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3974134690 0x7f397412eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f397406cf40 0x7f39741c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:38686/0 (socket says 172.21.15.47:38686) 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 1 -- 172.21.15.47:0/226053684 learned_addr learned my addr 172.21.15.47:0/226053684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): get_auth_request con 0x7f397406cf40 auth_method 0 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39740a2db0 0x7f397412e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): get_auth_request con 0x7f39740a2db0 auth_method 0 2024-04-20T02:22:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f3973fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.888+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 10 monclient(hunting): handle_auth_done global_id 54292 payload 824 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3974134690 msgr2=0x7f397412eae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3974134690 0x7f397412eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39740a2db0 msgr2=0x7f397412e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39740a2db0 0x7f397412e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 monclient: found mon.noname-c 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c00bd40 con 0x7f397406cf40 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3973fff700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f397406cf40 0x7f39741c7f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f396400e810 tx=0x7f396400eb20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 client.?.objecter ms_handle_connect 0x7f397406cf40 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3964013880 con 0x7f397406cf40 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: dump: 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: _finish_auth 0 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 5 monclient: authenticate success, global_id 54292 2024-04-20T02:22:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3964013a20 con 0x7f397406cf40 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 10 monclient: _renew_subs 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39740725a0 con 0x7f397406cf40 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3964012370 con 0x7f397406cf40 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f39727fc700 4 set_mon_vals no callback set 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f3970ff9700 10 monclient: dump: 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 10 client.54292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 10 monclient: _renew_subs 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397412f020 con 0x7f397406cf40 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.889+0000 7f397aa03700 1 librados: init done 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3964013b90 con 0x7f397406cf40 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f397aa03700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f397aa03700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 0x7f397406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3974068430 con 0x7f397406aac0 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f395403a320 0x7f395403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 0x7f397406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient: get_auth_request con 0x7f397406aac0 auth_method 0 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(303..303 src has 1..303) v4 ==== 7062+0+0 (secure 0 0 0) 0x7f396404a560 con 0x7f397406cf40 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 10 client.54292.objecter ms_dispatch 0x7f3974001550 osd_map(303..303 src has 1..303) v4 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 3 client.54292.objecter handle_osd_map got epochs [303,303] > 0 2024-04-20T02:22:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 3 client.54292.objecter handle_osd_map decoding full epoch 303 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3978fa0700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f395403a320 0x7f395403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3970ff9700 20 client.54292.objecter dump_active .. 0 homeless 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f3978fa0700 10 monclient: get_auth_request con 0x7f395403a320 auth_method 0 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 10 monclient(hunting): handle_auth_done global_id 23960 payload 824 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.890+0000 7f39737fe700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39740a11e0 con 0x7f397406aac0 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 client.54292.objecter ms_handle_connect 0x7f397406aac0 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 client.54292.objecter resend_mon_ops 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f39737fe700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 0x7f397406f070 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f397409d5e0 tx=0x7f395c011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f395c00c070 con 0x7f397406aac0 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 monclient: dump: 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 client.54292.objecter ms_handle_connect 0x7f395403a320 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3978fa0700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f395403a320 0x7f395403c7e0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7f396c00d4a0 tx=0x7f396c00b000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f395c015780 con 0x7f397406aac0 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 msgr2=0x7f397406f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 0x7f397406f070 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f397409d5e0 tx=0x7f395c011960 comp rx=0 tx=0).stop 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 msgr2=0x7f397406f070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.891+0000 7f3970ff9700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406aac0 0x7f397406f070 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f397aa03700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f397aa03700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 0x7f397406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f397aa03700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397406ba00 con 0x7f397406f5b0 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 0x7f397406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient: get_auth_request con 0x7f397406f5b0 auth_method 0 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 10 monclient(hunting): handle_auth_done global_id 23969 payload 824 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.899+0000 7f39737fe700 1 -- 172.21.15.47:0/226053684 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f397406ff00 con 0x7f397406f5b0 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 client.54292.objecter ms_handle_connect 0x7f397406f5b0 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f39737fe700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 0x7f397406f9c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f395c01b000 tx=0x7f395c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 client.54292.objecter resend_mon_ops 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f395c024070 con 0x7f397406f5b0 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 monclient: dump: 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9174 (secure 0 0 0) 0x7f395c01fdf0 con 0x7f397406f5b0 2024-04-20T02:22:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 msgr2=0x7f397406f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 0x7f397406f9c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f395c01b000 tx=0x7f395c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 msgr2=0x7f397406f9c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f3970ff9700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f397406f5b0 0x7f397406f9c0 secure :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f395c01b000 tx=0x7f395c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:22:56.924 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:56.925 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:56.926 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:56.927 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:56.928 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:56.929 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:56.930 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.932 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:56.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.935 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.936 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.938 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.940 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:56.941 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.942 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.943 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:56.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:56.945 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:56.945 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f397aa03700 10 librados: watch_flush enter 2024-04-20T02:22:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f397aa03700 10 librados: watch_flush exit 2024-04-20T02:22:56.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f397aa03700 20 client.54292.objecter shutdown clearing up homeless session... 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.900+0000 7f397aa03700 10 client.54292.objecter successfully canceled tick 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f395403a320 msgr2=0x7f395403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f395403a320 0x7f395403c7e0 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 crypto rx=0x7f396c00d4a0 tx=0x7f396c00b000 comp rx=0 tx=0).stop 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 10 monclient: shutdown 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f397406cf40 msgr2=0x7f39741c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 --2- 172.21.15.47:0/226053684 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f397406cf40 0x7f39741c7f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f396400e810 tx=0x7f396400eb20 comp rx=0 tx=0).stop 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f3978fa0700 1 -- 172.21.15.47:0/226053684 reap_dead start 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f3978fa0700 1 -- 172.21.15.47:0/226053684 reap_dead start 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 shutdown_connections 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 >> 172.21.15.47:0/226053684 conn(0x7f39740a4010 msgr2=0x7f39740a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 shutdown_connections 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 -- 172.21.15.47:0/226053684 wait complete. 2024-04-20T02:22:56.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:56.901+0000 7f397aa03700 1 librados: shutdown 2024-04-20T02:22:56.948 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:56.948 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-04-20T02:22:56.948 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.093+0000 7f10cd9f9700 10 monclient: get_monmap_and_config 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.093+0000 7f10cd9f9700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.093+0000 7f10cd9f9700 10 monclient: monmap: 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:57.094342+0000 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:57.094342+0000 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.093 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:57.094 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.093+0000 7f10cd9f9700 1 Processor -- start 2024-04-20T02:22:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.093+0000 7f10cd9f9700 1 -- start start 2024-04-20T02:22:57.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.093+0000 7f10cd9f9700 10 monclient: init 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient: _add_conns ranks=[5,1,7,6,0,4,9,10,8,2,3] 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f10c80a0bb0 0x7f10c80a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): picked mon.noname-h con 0x7f10c80a0bb0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f10c80a28e0 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a3450 0x7f10c813fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): picked mon.noname-i con 0x7f10c80a3450 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c80751e0 con 0x7f10c80a28e0 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c80678f0 con 0x7f10c80a0bb0 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f10c809ef20 con 0x7f10c80a3450 2024-04-20T02:22:57.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10cd9f9700 10 monclient(hunting): authenticate will time out at 2056.495478s 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c6ffd700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f10c80a0bb0 msgr2=0x7f10c80a0f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c6ffd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f10c80a0bb0 0x7f10c80a0f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c67fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f10c80a28e0 0x7f10c80a2d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:42376/0 (socket says 172.21.15.47:42376) 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c67fc700 1 -- 172.21.15.47:0/169725752 learned_addr learned my addr 172.21.15.47:0/169725752 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c77fe700 1 --2- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a3450 0x7f10c813fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c77fe700 10 monclient(hunting): get_auth_request con 0x7f10c80a3450 auth_method 0 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.094+0000 7f10c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 1 -- 172.21.15.47:0/169725752 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072143755 0 0) 0x7f10c80751e0 con 0x7f10c80a28e0 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient(hunting): my global_id is 54388 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 1 -- 172.21.15.47:0/169725752 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10b00035b0 con 0x7f10c80a28e0 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 10 monclient(hunting): handle_auth_done global_id 54304 payload 274 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 1 -- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f10c80a0bb0 msgr2=0x7f10c80a0f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 1 --2- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f10c80a0bb0 0x7f10c80a0f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 1 -- 172.21.15.47:0/169725752 >> v1:172.21.15.179:6789/0 conn(0x7f10c80a28e0 legacy=0x7f10c80a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 1 monclient: found mon.noname-i 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:22:57.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 1 -- 172.21.15.47:0/169725752 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c809e3d0 con 0x7f10c80a3450 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c77fe700 1 --2- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a3450 0x7f10c813fae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009dc0 tx=0x7f10c000b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=653230b76f71ab97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 1 -- 172.21.15.47:0/169725752 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10c000e070 con 0x7f10c80a3450 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: dump: 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.109 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10cd9f9700 5 monclient: authenticate success, global_id 54304 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 1 -- 172.21.15.47:0/169725752 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0012430 con 0x7f10c80a3450 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10cd9f9700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:57.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10cd9f9700 4 set_mon_vals no callback set 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 1 -- 172.21.15.47:0/169725752 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10c0012a10 con 0x7f10c80a3450 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.095+0000 7f10c5ffb700 10 monclient: dump: 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.111 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 10 monclient: shutdown 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 -- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a3450 msgr2=0x7f10c813fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a3450 0x7f10c813fae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009dc0 tx=0x7f10c000b330 comp rx=0 tx=0).stop 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 -- 172.21.15.47:0/169725752 shutdown_connections 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a3450 0x7f10c813fae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/169725752 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f10c80a0bb0 0x7f10c80a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 -- 172.21.15.47:0/169725752 >> 172.21.15.47:0/169725752 conn(0x7f10c80a39d0 msgr2=0x7f10c80a3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 -- 172.21.15.47:0/169725752 shutdown_connections 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.096+0000 7f10cd9f9700 1 -- 172.21.15.47:0/169725752 wait complete. 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 10 monclient: monmap: 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.113 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 1 librados: starting msgr at 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 1 librados: starting objecter 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 1 Processor -- start 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 1 -- start start 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 1 librados: setting wanted keys 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 1 librados: calling monclient init 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.097+0000 7f10cd9f9700 10 monclient: init 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient: _add_conns ranks=[7,8,2,6,5,3,4,0,1] 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a0bb0 0x7f10c81bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): picked mon.noname-i con 0x7f10c80a0bb0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:57.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f10c80a28e0 0x7f10c81bd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): picked mon.noname-e con 0x7f10c80a28e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10c80a3450 0x7f10c81cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): picked mon.noname-b con 0x7f10c80a3450 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c81aa5c0 con 0x7f10c80a3450 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f10c81a7e00 con 0x7f10c80a0bb0 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f10c81aa2e0 con 0x7f10c80a28e0 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10cd9f9700 10 monclient(hunting): authenticate will time out at 2056.499303s 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10c80a3450 0x7f10c81cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f10c80a28e0 0x7f10c81bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10c80a3450 0x7f10c81cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58116/0 (socket says 172.21.15.47:58116) 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 -- 172.21.15.47:0/4179604039 learned_addr learned my addr 172.21.15.47:0/4179604039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): get_auth_request con 0x7f10c80a3450 auth_method 0 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): get_auth_request con 0x7f10c80a28e0 auth_method 0 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a0bb0 0x7f10c81bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): get_auth_request con 0x7f10c80a0bb0 auth_method 0 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient(hunting): handle_auth_done global_id 23978 payload 824 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f10c80a28e0 msgr2=0x7f10c81bd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f10c80a28e0 0x7f10c81bd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a0bb0 msgr2=0x7f10c81bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f10c80a0bb0 0x7f10c81bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 monclient: found mon.noname-b 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0009a70 con 0x7f10c80a3450 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10c77fe700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10c80a3450 0x7f10c81cc2a0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b370 tx=0x7f10c0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.098+0000 7f10affff700 10 client.?.objecter ms_handle_connect 0x7f10c80a3450 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10c0012c20 con 0x7f10c80a3450 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:57.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: dump: 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.119 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 5 monclient: authenticate success, global_id 23978 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0012de0 con 0x7f10c80a3450 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10c00044b0 con 0x7f10c80a3450 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10c4ff9700 4 set_mon_vals no callback set 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10affff700 10 monclient: dump: 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.120 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 10 monclient: _renew_subs 2024-04-20T02:22:57.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c81bd680 con 0x7f10c80a3450 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 10 client.23978.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 10 monclient: _renew_subs 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c81aa000 con 0x7f10c80a3450 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.099+0000 7f10cd9f9700 1 librados: init done 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10cd9f9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 0x7f10c81cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c8068430 con 0x7f10c806aac0 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 0x7f10c81cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient: get_auth_request con 0x7f10c806aac0 auth_method 0 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f10c0004040 con 0x7f10c80a3450 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_done global_id 54301 payload 824 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10c8072a10 con 0x7f10c806aac0 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f10b403a2f0 0x7f10b403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(304..304 src has 1..304) v4 ==== 6645+0+0 (secure 0 0 0) 0x7f10c004c580 con 0x7f10c80a3450 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 10 client.23978.objecter ms_dispatch 0x7f10c8001550 osd_map(304..304 src has 1..304) v4 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 3 client.23978.objecter handle_osd_map got epochs [304,304] > 0 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 3 client.23978.objecter handle_osd_map decoding full epoch 304 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c6ffd700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 0x7f10c81cc7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096d0 tx=0x7f10b8009a90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 20 client.23978.objecter dump_active .. 0 homeless 2024-04-20T02:22:57.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c67fc700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f10b403a2f0 0x7f10b403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 10 client.23978.objecter ms_handle_connect 0x7f10c806aac0 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 10 client.23978.objecter resend_mon_ops 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10c67fc700 10 monclient: get_auth_request con 0x7f10b403a2f0 auth_method 0 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10b8012bd0 con 0x7f10c806aac0 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.100+0000 7f10affff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 10 monclient: dump: 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.124 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f10b8012d30 con 0x7f10c806aac0 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 msgr2=0x7f10c81cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 0x7f10c81cc7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096d0 tx=0x7f10b8009a90 comp rx=0 tx=0).stop 2024-04-20T02:22:57.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 msgr2=0x7f10c81cc7e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c806aac0 0x7f10c81cc7e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10affff700 10 client.23978.objecter ms_handle_connect 0x7f10b403a2f0 2024-04-20T02:22:57.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.101+0000 7f10c67fc700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f10b403a2f0 0x7f10b403c7b0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f10c809db40 tx=0x7f10bc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10cd9f9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 0x7f10c806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10cd9f9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10c806a400 con 0x7f10c81ccd20 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 0x7f10c806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient: get_auth_request con 0x7f10c81ccd20 auth_method 0 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 10 monclient(hunting): handle_auth_done global_id 54310 payload 824 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 1 -- 172.21.15.47:0/4179604039 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f10c8072070 con 0x7f10c81ccd20 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10affff700 10 client.23978.objecter ms_handle_connect 0x7f10c81ccd20 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10affff700 10 client.23978.objecter resend_mon_ops 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10c6ffd700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 0x7f10c806f090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096d0 tx=0x7f10b8005a60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f10b8022070 con 0x7f10c81ccd20 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10affff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10affff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.109+0000 7f10affff700 10 monclient: dump: 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7f10b801a820 con 0x7f10c81ccd20 2024-04-20T02:22:57.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 msgr2=0x7f10c806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 0x7f10c806f090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096d0 tx=0x7f10b8005a60 comp rx=0 tx=0).stop 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 msgr2=0x7f10c806f090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10affff700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f10c81ccd20 0x7f10c806f090 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10b80096d0 tx=0x7f10b8005a60 comp rx=0 tx=0).stop 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:57.131 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:57.132 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.133 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:57.134 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:57.135 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:57.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.137 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.139 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.140 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:57.141 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.142 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.145 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.147 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:57.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.150 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.151 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:57.152 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:57.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 10 librados: watch_flush enter 2024-04-20T02:22:57.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 10 librados: watch_flush exit 2024-04-20T02:22:57.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 20 client.23978.objecter shutdown clearing up homeless session... 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 10 client.23978.objecter successfully canceled tick 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f10b403a2f0 msgr2=0x7f10b403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f10b403a2f0 0x7f10b403c7b0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f10c809db40 tx=0x7f10bc009040 comp rx=0 tx=0).stop 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 10 monclient: shutdown 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10c80a3450 msgr2=0x7f10c81cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 --2- 172.21.15.47:0/4179604039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f10c80a3450 0x7f10c81cc2a0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f10c000b370 tx=0x7f10c0004ad0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10c77fe700 1 -- 172.21.15.47:0/4179604039 reap_dead start 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10c77fe700 1 -- 172.21.15.47:0/4179604039 reap_dead start 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 shutdown_connections 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 >> 172.21.15.47:0/4179604039 conn(0x7f10c80a39d0 msgr2=0x7f10c80a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 shutdown_connections 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 -- 172.21.15.47:0/4179604039 wait complete. 2024-04-20T02:22:57.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.110+0000 7f10cd9f9700 1 librados: shutdown 2024-04-20T02:22:57.156 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:57.156 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-04-20T02:22:57.156 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:22:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.297+0000 7fe1ea661700 10 monclient: get_monmap_and_config 2024-04-20T02:22:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.297+0000 7fe1ea661700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.297+0000 7fe1ea661700 10 monclient: monmap: 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:57.298308+0000 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:57.298308+0000 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.297+0000 7fe1ea661700 1 Processor -- start 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 1 -- start start 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient: init 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient: _add_conns ranks=[9,4,1,0,7,2,3,5,6,10,8] 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 0x7fe1e40a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): picked mon.noname-a con 0x7fe1e409cec0 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:57.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 0x7fe1e40a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): picked mon.noname-c con 0x7fe1e4099d50 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe1e409c2a0 addr v1:172.21.15.179:6789/0 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e40751e0 con 0x7fe1e409c2a0 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e40684d0 con 0x7fe1e409cec0 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e409a640 con 0x7fe1e4099d50 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1ea661700 10 monclient(hunting): authenticate will time out at 2056.699666s 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 0x7fe1e40a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 0x7fe1e40a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 0x7fe1e40a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35004/0 (socket says 172.21.15.47:35004) 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 0x7fe1e40a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:38708/0 (socket says 172.21.15.47:38708) 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 1 -- 172.21.15.47:0/901105528 learned_addr learned my addr 172.21.15.47:0/901105528 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): get_auth_request con 0x7fe1e409cec0 auth_method 0 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): get_auth_request con 0x7fe1e4099d50 auth_method 0 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.298+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_done global_id 54319 payload 274 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 1 -- 172.21.15.47:0/901105528 >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 msgr2=0x7fe1e40a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 1 --2- 172.21.15.47:0/901105528 >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 0x7fe1e40a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 1 -- 172.21.15.47:0/901105528 >> v1:172.21.15.179:6789/0 conn(0x7fe1e409c2a0 legacy=0x7fe1e413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 1 monclient: found mon.noname-c 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 1 -- 172.21.15.47:0/901105528 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e409bba0 con 0x7fe1e4099d50 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/901105528 >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 0x7fe1e40a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e37fe700 1 --2- 172.21.15.47:0/901105528 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 0x7fe1e40a2450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009840 tx=0x7fe1cc009b50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=632e6d1fed445795 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 1 -- 172.21.15.47:0/901105528 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1cc011aa0 con 0x7fe1e4099d50 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: dump: 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.314 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1ea661700 5 monclient: authenticate success, global_id 54319 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 1 -- 172.21.15.47:0/901105528 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc011c00 con 0x7fe1e4099d50 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1ea661700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 1 -- 172.21.15.47:0/901105528 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1cc011d70 con 0x7fe1e4099d50 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1ea661700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1ea661700 4 set_mon_vals no callback set 2024-04-20T02:22:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.299+0000 7fe1e2ffd700 10 monclient: dump: 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.316 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 10 monclient: shutdown 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 -- 172.21.15.47:0/901105528 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 msgr2=0x7fe1e40a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 --2- 172.21.15.47:0/901105528 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 0x7fe1e40a2450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009840 tx=0x7fe1cc009b50 comp rx=0 tx=0).stop 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 -- 172.21.15.47:0/901105528 shutdown_connections 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 --2- 172.21.15.47:0/901105528 >> v2:172.21.15.47:3300/0 conn(0x7fe1e409cec0 0x7fe1e40a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 --2- 172.21.15.47:0/901105528 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe1e4099d50 0x7fe1e40a2450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 -- 172.21.15.47:0/901105528 >> 172.21.15.47:0/901105528 conn(0x7fe1e40a3000 msgr2=0x7fe1e40a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 -- 172.21.15.47:0/901105528 shutdown_connections 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.300+0000 7fe1ea661700 1 -- 172.21.15.47:0/901105528 wait complete. 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 10 monclient: monmap: 2024-04-20T02:22:57.317 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 1 librados: starting msgr at 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 1 librados: starting objecter 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 1 Processor -- start 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 1 -- start start 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 1 librados: setting wanted keys 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 1 librados: calling monclient init 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.301+0000 7fe1ea661700 10 monclient: init 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient: _add_conns ranks=[1,2,6,0,5,7,8,4,3] 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4099d50 0x7fe1e40902d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): picked mon.noname-f con 0x7fe1e4099d50 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1e409c2a0 0x7fe1e4090810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): picked mon.noname-b con 0x7fe1e409c2a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e409cec0 0x7fe1e4090d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): picked mon.noname-d con 0x7fe1e409cec0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e409abb0 con 0x7fe1e4099d50 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4093500 con 0x7fe1e409c2a0 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e409a7f0 con 0x7fe1e409cec0 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1ea661700 10 monclient(hunting): authenticate will time out at 2056.703270s 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1e409c2a0 0x7fe1e4090810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1e409c2a0 0x7fe1e4090810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58128/0 (socket says 172.21.15.47:58128) 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 -- 172.21.15.47:0/3817980127 learned_addr learned my addr 172.21.15.47:0/3817980127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): get_auth_request con 0x7fe1e409c2a0 auth_method 0 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e409cec0 0x7fe1e4090d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): get_auth_request con 0x7fe1e409cec0 auth_method 0 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4099d50 0x7fe1e40902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e8bfe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e3fff700 10 monclient(hunting): get_auth_request con 0x7fe1e4099d50 auth_method 0 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient(hunting): handle_auth_done global_id 23987 payload 824 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e409cec0 msgr2=0x7fe1e4090d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e409cec0 0x7fe1e4090d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4099d50 msgr2=0x7fe1e40902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4099d50 0x7fe1e40902d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 monclient: found mon.noname-b 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4009550 con 0x7fe1e409c2a0 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4099d50 0x7fe1e40902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e37fe700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1e409c2a0 0x7fe1e4090810 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009b90 tx=0x7fe1cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e0ff9700 10 client.?.objecter ms_handle_connect 0x7fe1e409c2a0 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.302+0000 7fe1e0ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1cc004310 con 0x7fe1e409c2a0 2024-04-20T02:22:57.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: dump: 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.323 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 5 monclient: authenticate success, global_id 23987 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 10 monclient: _renew_subs 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc01f070 con 0x7fe1e409c2a0 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0094a0 con 0x7fe1e409c2a0 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1cc0044b0 con 0x7fe1e409c2a0 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e27fc700 4 set_mon_vals no callback set 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 10 client.23987.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:57.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1e0ff9700 10 monclient: dump: 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.325 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 10 monclient: _renew_subs 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e40937e0 con 0x7fe1e409c2a0 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.303+0000 7fe1ea661700 1 librados: init done 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe1cc021030 con 0x7fe1e409c2a0 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1ea661700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1ea661700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 0x7fe1e406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4068230 con 0x7fe1e406aac0 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe1c403a2f0 0x7fe1c403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(304..304 src has 1..304) v4 ==== 6645+0+0 (secure 0 0 0) 0x7fe1cc04baf0 con 0x7fe1e409c2a0 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 10 client.23987.objecter ms_dispatch 0x7fe1e4001550 osd_map(304..304 src has 1..304) v4 2024-04-20T02:22:57.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 3 client.23987.objecter handle_osd_map got epochs [304,304] > 0 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 3 client.23987.objecter handle_osd_map decoding full epoch 304 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 0x7fe1e406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient: get_auth_request con 0x7fe1e406aac0 auth_method 0 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 20 client.23987.objecter dump_active .. 0 homeless 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_done global_id 54312 payload 824 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1e409b8c0 con 0x7fe1e406aac0 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e8bfe700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe1c403a2f0 0x7fe1c403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e8bfe700 10 monclient: get_auth_request con 0x7fe1c403a2f0 auth_method 0 2024-04-20T02:22:57.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 10 client.23987.objecter ms_handle_connect 0x7fe1e406aac0 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e0ff9700 10 client.23987.objecter resend_mon_ops 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.304+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 0x7fe1e406ef60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4011830 tx=0x7fe1d4011bf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1d4015ae0 con 0x7fe1e406aac0 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 10 monclient: dump: 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.328 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 10 client.23987.objecter ms_handle_connect 0x7fe1c403a2f0 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e8bfe700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe1c403a2f0 0x7fe1c403c7b0 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7fe1e406c210 tx=0x7fe1dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe1d4015c40 con 0x7fe1e406aac0 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 msgr2=0x7fe1e406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 0x7fe1e406ef60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4011830 tx=0x7fe1d4011bf0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 msgr2=0x7fe1e406ef60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.305+0000 7fe1e0ff9700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406aac0 0x7fe1e406ef60 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4011830 tx=0x7fe1d4011bf0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1ea661700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:57.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1ea661700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 0x7fe1e406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1ea661700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e406a400 con 0x7fe1e406f4a0 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 0x7fe1e406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient: get_auth_request con 0x7fe1e406f4a0 auth_method 0 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 10 monclient(hunting): handle_auth_done global_id 54321 payload 824 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 1 -- 172.21.15.47:0/3817980127 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe1e409d2a0 con 0x7fe1e406f4a0 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e0ff9700 10 client.23987.objecter ms_handle_connect 0x7fe1e406f4a0 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e0ff9700 10 client.23987.objecter resend_mon_ops 2024-04-20T02:22:57.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e3fff700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 0x7fe1e406f8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe1d401b000 tx=0x7fe1d4004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1d402b070 con 0x7fe1e406f4a0 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.313+0000 7fe1e0ff9700 10 monclient: dump: 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.333 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7fe1d4023040 con 0x7fe1e406f4a0 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 msgr2=0x7fe1e406f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 0x7fe1e406f8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe1d401b000 tx=0x7fe1d4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 msgr2=0x7fe1e406f8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e0ff9700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe1e406f4a0 0x7fe1e406f8b0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe1d401b000 tx=0x7fe1d4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.334 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:57.335 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.336 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:57.337 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:57.338 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.339 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.340 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.343 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.344 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.345 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:57.347 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.348 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:57.350 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.351 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.353 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.354 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:57.355 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 10 librados: watch_flush enter 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 10 librados: watch_flush exit 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 20 client.23987.objecter shutdown clearing up homeless session... 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 10 client.23987.objecter successfully canceled tick 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe1c403a2f0 msgr2=0x7fe1c403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe1c403a2f0 0x7fe1c403c7b0 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7fe1e406c210 tx=0x7fe1dc009040 comp rx=0 tx=0).stop 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 10 monclient: shutdown 2024-04-20T02:22:57.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1e409c2a0 msgr2=0x7fe1e4090810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 --2- 172.21.15.47:0/3817980127 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe1e409c2a0 0x7fe1e4090810 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009b90 tx=0x7fe1cc004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e8bfe700 1 -- 172.21.15.47:0/3817980127 reap_dead start 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1e8bfe700 1 -- 172.21.15.47:0/3817980127 reap_dead start 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 shutdown_connections 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 >> 172.21.15.47:0/3817980127 conn(0x7fe1e40a3000 msgr2=0x7fe1e41bf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 shutdown_connections 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 -- 172.21.15.47:0/3817980127 wait complete. 2024-04-20T02:22:57.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.314+0000 7fe1ea661700 1 librados: shutdown 2024-04-20T02:22:57.358 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:57.358 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-04-20T02:22:57.359 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:22:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.498+0000 7fcb3a762700 10 monclient: get_monmap_and_config 2024-04-20T02:22:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.498+0000 7fcb3a762700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.498+0000 7fcb3a762700 10 monclient: monmap: 2024-04-20T02:22:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.498 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:57.499458+0000 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:57.499458+0000 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:57.499 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:57.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 Processor -- start 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 -- start start 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient: init 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient: _add_conns ranks=[10,7,6,9,2,1,3,8,4,5,0] 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb34071730 0x7fcb34071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): picked mon.noname-f con 0x7fcb34071730 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb340a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): picked mon.noname-i con 0x7fcb3406f020 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 0x7fcb340a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): picked mon.noname-d con 0x7fcb340a2db0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3409fea0 con 0x7fcb34071730 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3409e7b0 con 0x7fcb340a2db0 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3406d190 con 0x7fcb3406f020 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb3a762700 10 monclient(hunting): authenticate will time out at 2056.900723s 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 0x7fcb340a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 0x7fcb340a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:36230/0 (socket says 172.21.15.47:36230) 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 1 -- 172.21.15.47:0/2675939577 learned_addr learned my addr 172.21.15.47:0/2675939577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 10 monclient(hunting): get_auth_request con 0x7fcb340a2db0 auth_method 0 2024-04-20T02:22:57.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb38cff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.499+0000 7fcb33fff700 1 --2- 172.21.15.47:0/2675939577 >> v2:172.21.15.179:3300/0 conn(0x7fcb34071730 0x7fcb34071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): get_auth_request con 0x7fcb34071730 auth_method 0 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 1 --2- 172.21.15.47:0/2675939577 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb340a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): get_auth_request con 0x7fcb3406f020 auth_method 0 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb337fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 10 monclient(hunting): handle_auth_done global_id 54330 payload 274 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 -- 172.21.15.47:0/2675939577 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 msgr2=0x7fcb340a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 --2- 172.21.15.47:0/2675939577 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb340a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 -- 172.21.15.47:0/2675939577 >> v2:172.21.15.179:3300/0 conn(0x7fcb34071730 msgr2=0x7fcb34071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 --2- 172.21.15.47:0/2675939577 >> v2:172.21.15.179:3300/0 conn(0x7fcb34071730 0x7fcb34071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 monclient: found mon.noname-d 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 -- 172.21.15.47:0/2675939577 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb340751e0 con 0x7fcb340a2db0 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb38cff700 1 --2- 172.21.15.47:0/2675939577 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 0x7fcb340a3240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009e50 tx=0x7fcb2c00b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ed9553b817052106 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 1 -- 172.21.15.47:0/2675939577 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2c00e070 con 0x7fcb340a2db0 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: dump: 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.516 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 1 -- 172.21.15.47:0/2675939577 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0099b0 con 0x7fcb340a2db0 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb3a762700 5 monclient: authenticate success, global_id 54330 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb3a762700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:57.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 1 -- 172.21.15.47:0/2675939577 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2c00bd80 con 0x7fcb340a2db0 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb3a762700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb3a762700 4 set_mon_vals no callback set 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.500+0000 7fcb32ffd700 10 monclient: dump: 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.518 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 10 monclient: shutdown 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 -- 172.21.15.47:0/2675939577 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 msgr2=0x7fcb340a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 --2- 172.21.15.47:0/2675939577 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 0x7fcb340a3240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009e50 tx=0x7fcb2c00b330 comp rx=0 tx=0).stop 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 -- 172.21.15.47:0/2675939577 shutdown_connections 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 --2- 172.21.15.47:0/2675939577 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcb340a2db0 0x7fcb340a3240 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 --2- 172.21.15.47:0/2675939577 >> v2:172.21.15.179:3300/0 conn(0x7fcb34071730 0x7fcb34071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 --2- 172.21.15.47:0/2675939577 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb340a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 -- 172.21.15.47:0/2675939577 >> 172.21.15.47:0/2675939577 conn(0x7fcb340a3f90 msgr2=0x7fcb3406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 -- 172.21.15.47:0/2675939577 shutdown_connections 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.501+0000 7fcb3a762700 1 -- 172.21.15.47:0/2675939577 wait complete. 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 10 monclient: monmap: 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.520 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 1 librados: starting msgr at 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 1 librados: starting objecter 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 1 Processor -- start 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 1 -- start start 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 1 librados: setting wanted keys 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 1 librados: calling monclient init 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.502+0000 7fcb3a762700 10 monclient: init 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient: _add_conns ranks=[7,1,4,2,0,6,5,8,3] 2024-04-20T02:22:57.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb341cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): picked mon.noname-i con 0x7fcb3406f020 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:57.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb34071730 0x7fcb341cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): picked mon.noname-f con 0x7fcb34071730 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb340a2db0 0x7fcb34124100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): picked mon.noname-c con 0x7fcb340a2db0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34127a00 con 0x7fcb34071730 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3409f1b0 con 0x7fcb340a2db0 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34072410 con 0x7fcb3406f020 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb3a762700 10 monclient(hunting): authenticate will time out at 2056.904181s 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb340a2db0 0x7fcb34124100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb340a2db0 0x7fcb34124100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:38722/0 (socket says 172.21.15.47:38722) 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 -- 172.21.15.47:0/1497811239 learned_addr learned my addr 172.21.15.47:0/1497811239 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): get_auth_request con 0x7fcb340a2db0 auth_method 0 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb34071730 0x7fcb341cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): get_auth_request con 0x7fcb34071730 auth_method 0 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb341cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): get_auth_request con 0x7fcb3406f020 auth_method 0 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb337fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient(hunting): handle_auth_done global_id 54328 payload 824 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 msgr2=0x7fcb341cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcb3406f020 0x7fcb341cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb34071730 msgr2=0x7fcb341cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb34071730 0x7fcb341cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 monclient: found mon.noname-c 2024-04-20T02:22:57.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c009b30 con 0x7fcb340a2db0 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb38cff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb340a2db0 0x7fcb34124100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b370 tx=0x7fcb2c004ad0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb30ff9700 10 client.?.objecter ms_handle_connect 0x7fcb340a2db0 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.503+0000 7fcb30ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2c0092e0 con 0x7fcb340a2db0 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: dump: 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.527 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c003e10 con 0x7fcb340a2db0 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 5 monclient: authenticate success, global_id 54328 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2c0046c0 con 0x7fcb340a2db0 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb30ff9700 10 monclient: dump: 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.529 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb31ffb700 4 set_mon_vals no callback set 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 10 monclient: _renew_subs 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb34127fc0 con 0x7fcb340a2db0 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 10 client.54328.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 10 monclient: _renew_subs 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:22:57.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb34125bf0 con 0x7fcb340a2db0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.504+0000 7fcb3a762700 1 librados: init done 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcb2c004820 con 0x7fcb340a2db0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb3a762700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb3a762700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 0x7fcb34124640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3406cd50 con 0x7fcb3406aac0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb2003a2f0 0x7fcb2003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 0x7fcb34124640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(304..304 src has 1..304) v4 ==== 6645+0+0 (secure 0 0 0) 0x7fcb2c04bb60 con 0x7fcb340a2db0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 10 client.54328.objecter ms_dispatch 0x7fcb34001550 osd_map(304..304 src has 1..304) v4 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 3 client.54328.objecter handle_osd_map got epochs [304,304] > 0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 3 client.54328.objecter handle_osd_map decoding full epoch 304 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient: get_auth_request con 0x7fcb3406aac0 auth_method 0 2024-04-20T02:22:57.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 20 client.54328.objecter dump_active .. 0 homeless 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 10 monclient(hunting): handle_auth_done global_id 35189 payload 824 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb34127b70 con 0x7fcb3406aac0 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb337fe700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb2003a2f0 0x7fcb2003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb337fe700 10 monclient: get_auth_request con 0x7fcb2003a2f0 auth_method 0 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 10 client.54328.objecter ms_handle_connect 0x7fcb3406aac0 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb33fff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 0x7fcb34124640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb240119a0 tx=0x7fcb24011d60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 10 client.54328.objecter resend_mon_ops 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb240036a0 con 0x7fcb3406aac0 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.505+0000 7fcb30ff9700 10 monclient: dump: 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.533 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcb24004240 con 0x7fcb3406aac0 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 msgr2=0x7fcb34124640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 0x7fcb34124640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb240119a0 tx=0x7fcb24011d60 comp rx=0 tx=0).stop 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 msgr2=0x7fcb34124640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb3406aac0 0x7fcb34124640 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb240119a0 tx=0x7fcb24011d60 comp rx=0 tx=0).stop 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb30ff9700 10 client.54328.objecter ms_handle_connect 0x7fcb2003a2f0 2024-04-20T02:22:57.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.506+0000 7fcb337fe700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb2003a2f0 0x7fcb2003c7b0 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 crypto rx=0x7fcb3406d340 tx=0x7fcb28009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb3a762700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:57.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb3a762700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 0x7fcb34124b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb3a762700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34068280 con 0x7fcb341283e0 2024-04-20T02:22:57.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 0x7fcb34124b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient: get_auth_request con 0x7fcb341283e0 auth_method 0 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 10 monclient(hunting): handle_auth_done global_id 35198 payload 824 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 1 -- 172.21.15.47:0/1497811239 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcb34066610 con 0x7fcb341283e0 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb33fff700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 0x7fcb34124b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb240091e0 tx=0x7fcb24004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb30ff9700 10 client.54328.objecter ms_handle_connect 0x7fcb341283e0 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb30ff9700 10 client.54328.objecter resend_mon_ops 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb2401e450 con 0x7fcb341283e0 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb30ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb30ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.514+0000 7fcb30ff9700 10 monclient: dump: 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.538 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7fcb2400fba0 con 0x7fcb341283e0 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 msgr2=0x7fcb34124b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 0x7fcb34124b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb240091e0 tx=0x7fcb24004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 msgr2=0x7fcb34124b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb30ff9700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcb341283e0 0x7fcb34124b80 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb240091e0 tx=0x7fcb24004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:57.539 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.540 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:57.541 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:57.542 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.543 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:57.544 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.545 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:57.547 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.548 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.551 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.554 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.557 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.558 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:57.559 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 10 librados: watch_flush enter 2024-04-20T02:22:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 10 librados: watch_flush exit 2024-04-20T02:22:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 20 client.54328.objecter shutdown clearing up homeless session... 2024-04-20T02:22:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 10 client.54328.objecter successfully canceled tick 2024-04-20T02:22:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb2003a2f0 msgr2=0x7fcb2003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcb2003a2f0 0x7fcb2003c7b0 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 crypto rx=0x7fcb3406d340 tx=0x7fcb28009210 comp rx=0 tx=0).stop 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 10 monclient: shutdown 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb340a2db0 msgr2=0x7fcb34124100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 --2- 172.21.15.47:0/1497811239 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcb340a2db0 0x7fcb34124100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b370 tx=0x7fcb2c004ad0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb38cff700 1 -- 172.21.15.47:0/1497811239 reap_dead start 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb38cff700 1 -- 172.21.15.47:0/1497811239 reap_dead start 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 shutdown_connections 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 >> 172.21.15.47:0/1497811239 conn(0x7fcb340a3f90 msgr2=0x7fcb340a6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 shutdown_connections 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 -- 172.21.15.47:0/1497811239 wait complete. 2024-04-20T02:22:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.515+0000 7fcb3a762700 1 librados: shutdown 2024-04-20T02:22:57.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:57.563 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-04-20T02:22:57.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.701+0000 7fdc30087700 10 monclient: get_monmap_and_config 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.701+0000 7fdc30087700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.701+0000 7fdc30087700 10 monclient: monmap: 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:57.702974+0000 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:57.702974+0000 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:57.702 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.702+0000 7fdc30087700 1 Processor -- start 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.702+0000 7fdc30087700 1 -- start start 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.702+0000 7fdc30087700 10 monclient: init 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient: _add_conns ranks=[2,5,10,8,9,4,3,0,1,7,6] 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc28071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): picked mon.noname-b con 0x7fdc28071730 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdc2806f020 0x7fdc280a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): picked mon.noname-h con 0x7fdc2806f020 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc280a2db0 0x7fdc280a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): picked mon.noname-f con 0x7fdc280a2db0 addr v2:172.21.15.179:3300/0 2024-04-20T02:22:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2809fea0 con 0x7fdc280a2db0 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2809e7b0 con 0x7fdc28071730 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2806d190 con 0x7fdc2806f020 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc30087700 10 monclient(hunting): authenticate will time out at 2057.104261s 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2d622700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdc2806f020 msgr2=0x7fdc280a2870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2d622700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdc2806f020 0x7fdc280a2870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc28071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc28071b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58130/0 (socket says 172.21.15.47:58130) 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 -- 172.21.15.47:0/3352908161 learned_addr learned my addr 172.21.15.47:0/3352908161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): get_auth_request con 0x7fdc28071730 auth_method 0 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2e624700 1 --2- 172.21.15.47:0/3352908161 >> v2:172.21.15.179:3300/0 conn(0x7fdc280a2db0 0x7fdc280a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2e624700 10 monclient(hunting): get_auth_request con 0x7fdc280a2db0 auth_method 0 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2e624700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2e624700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2e624700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient(hunting): handle_auth_done global_id 23996 payload 274 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 -- 172.21.15.47:0/3352908161 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdc2806f020 msgr2=0x7fdc280a2870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 --2- 172.21.15.47:0/3352908161 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdc2806f020 0x7fdc280a2870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 -- 172.21.15.47:0/3352908161 >> v2:172.21.15.179:3300/0 conn(0x7fdc280a2db0 msgr2=0x7fdc280a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 --2- 172.21.15.47:0/3352908161 >> v2:172.21.15.179:3300/0 conn(0x7fdc280a2db0 0x7fdc280a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 monclient: found mon.noname-b 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 -- 172.21.15.47:0/3352908161 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280751e0 con 0x7fdc28071730 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.703+0000 7fdc2de23700 1 --2- 172.21.15.47:0/3352908161 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc28071b10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098f0 tx=0x7fdc24009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2cdc6cefc63b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 1 -- 172.21.15.47:0/3352908161 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc2400bb10 con 0x7fdc28071730 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: dump: 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 1 -- 172.21.15.47:0/3352908161 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2400bc70 con 0x7fdc28071730 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 5 monclient: authenticate success, global_id 23996 2024-04-20T02:22:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 1 -- 172.21.15.47:0/3352908161 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc2400bde0 con 0x7fdc28071730 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 4 set_mon_vals no callback set 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc2ce21700 10 monclient: dump: 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 10 monclient: shutdown 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 -- 172.21.15.47:0/3352908161 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 msgr2=0x7fdc28071b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 --2- 172.21.15.47:0/3352908161 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc28071b10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098f0 tx=0x7fdc24009c00 comp rx=0 tx=0).stop 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 -- 172.21.15.47:0/3352908161 shutdown_connections 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 --2- 172.21.15.47:0/3352908161 >> v2:172.21.15.179:3300/0 conn(0x7fdc280a2db0 0x7fdc280a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 --2- 172.21.15.47:0/3352908161 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc28071b10 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 --2- 172.21.15.47:0/3352908161 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdc2806f020 0x7fdc280a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 -- 172.21.15.47:0/3352908161 >> 172.21.15.47:0/3352908161 conn(0x7fdc280a3f90 msgr2=0x7fdc2806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 -- 172.21.15.47:0/3352908161 shutdown_connections 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.704+0000 7fdc30087700 1 -- 172.21.15.47:0/3352908161 wait complete. 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 10 monclient: monmap: 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 1 librados: starting msgr at 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 1 librados: starting objecter 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 1 Processor -- start 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 1 -- start start 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 1 librados: setting wanted keys 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 1 librados: calling monclient init 2024-04-20T02:22:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.705+0000 7fdc30087700 10 monclient: init 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient: _add_conns ranks=[1,2,3,5,8,7,6,0,4] 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806f020 0x7fdc281ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): picked mon.noname-f con 0x7fdc2806f020 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc281243c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): picked mon.noname-b con 0x7fdc28071730 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc280a2db0 0x7fdc28124900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): picked mon.noname-g con 0x7fdc280a2db0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28072410 con 0x7fdc2806f020 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2809f1b0 con 0x7fdc28071730 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28126a80 con 0x7fdc280a2db0 2024-04-20T02:22:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc30087700 10 monclient(hunting): authenticate will time out at 2057.107382s 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc281243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc281243c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58146/0 (socket says 172.21.15.47:58146) 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 -- 172.21.15.47:0/4281060378 learned_addr learned my addr 172.21.15.47:0/4281060378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): get_auth_request con 0x7fdc28071730 auth_method 0 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2de23700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806f020 0x7fdc281ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2de23700 10 monclient(hunting): get_auth_request con 0x7fdc2806f020 auth_method 0 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2de23700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2de23700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2de23700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2e624700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc280a2db0 0x7fdc28124900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2e624700 10 monclient(hunting): get_auth_request con 0x7fdc280a2db0 auth_method 0 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2e624700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2e624700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2e624700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient(hunting): handle_auth_done global_id 24005 payload 824 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc280a2db0 msgr2=0x7fdc28124900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdc280a2db0 0x7fdc28124900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806f020 msgr2=0x7fdc281ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806f020 0x7fdc281ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 monclient: found mon.noname-b 2024-04-20T02:22:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009550 con 0x7fdc28071730 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.706+0000 7fdc2d622700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc281243c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800b570 tx=0x7fdc1800b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 client.?.objecter ms_handle_connect 0x7fdc28071730 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc18003c40 con 0x7fdc28071730 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: dump: 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.727 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18003de0 con 0x7fdc28071730 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 5 monclient: authenticate success, global_id 24005 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 10 monclient: _renew_subs 2024-04-20T02:22:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc28127fa0 con 0x7fdc28071730 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc18011400 con 0x7fdc28071730 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1e7fc700 10 monclient: dump: 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc1ffff700 4 set_mon_vals no callback set 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 10 client.24005.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 10 monclient: _renew_subs 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281267c0 con 0x7fdc28071730 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.707+0000 7fdc30087700 1 librados: init done 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc30087700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc30087700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 0x7fdc28124e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28068280 con 0x7fdc28128bf0 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 0x7fdc28124e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient: get_auth_request con 0x7fdc28128bf0 auth_method 0 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fdc18005070 con 0x7fdc28071730 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc0c03a2f0 0x7fdc0c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(304..304 src has 1..304) v4 ==== 6645+0+0 (secure 0 0 0) 0x7fdc1804a2b0 con 0x7fdc28071730 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 10 client.24005.objecter ms_dispatch 0x7fdc28001550 osd_map(304..304 src has 1..304) v4 2024-04-20T02:22:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 3 client.24005.objecter handle_osd_map got epochs [304,304] > 0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 3 client.24005.objecter handle_osd_map decoding full epoch 304 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 10 monclient(hunting): handle_auth_done global_id 54451 payload 824 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc28127c20 con 0x7fdc28128bf0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 20 client.24005.objecter dump_active .. 0 homeless 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2e624700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc0c03a2f0 0x7fdc0c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2e624700 10 monclient: get_auth_request con 0x7fdc0c03a2f0 auth_method 0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 10 client.24005.objecter ms_handle_connect 0x7fdc28128bf0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc1e7fc700 10 client.24005.objecter resend_mon_ops 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.708+0000 7fdc2de23700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 0x7fdc28124e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098f0 tx=0x7fdc24004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc240036a0 con 0x7fdc28128bf0 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 10 monclient: dump: 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 10 client.24005.objecter ms_handle_connect 0x7fdc0c03a2f0 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc2e624700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc0c03a2f0 0x7fdc0c03c7b0 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fdc28125b20 tx=0x7fdc2000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdc24004250 con 0x7fdc28128bf0 2024-04-20T02:22:57.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:57.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 msgr2=0x7fdc28124e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 0x7fdc28124e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098f0 tx=0x7fdc24004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 msgr2=0x7fdc28124e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.709+0000 7fdc1e7fc700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc28128bf0 0x7fdc28124e40 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098f0 tx=0x7fdc24004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc30087700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc30087700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 0x7fdc28125380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc30087700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2806a400 con 0x7fdc2806aac0 2024-04-20T02:22:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 0x7fdc28125380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient: get_auth_request con 0x7fdc2806aac0 auth_method 0 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 10 monclient(hunting): handle_auth_done global_id 54460 payload 824 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 1 -- 172.21.15.47:0/4281060378 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdc2806a620 con 0x7fdc2806aac0 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.717+0000 7fdc2de23700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 0x7fdc28125380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009c40 tx=0x7fdc24011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 client.24005.objecter ms_handle_connect 0x7fdc2806aac0 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 client.24005.objecter resend_mon_ops 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc2402bca0 con 0x7fdc2806aac0 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 monclient: dump: 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7fdc2402be00 con 0x7fdc2806aac0 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 msgr2=0x7fdc28125380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 0x7fdc28125380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009c40 tx=0x7fdc24011000 comp rx=0 tx=0).stop 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 msgr2=0x7fdc28125380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc1e7fc700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc2806aac0 0x7fdc28125380 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009c40 tx=0x7fdc24011000 comp rx=0 tx=0).stop 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:57.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.756 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.758 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.759 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.760 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:22:57.760 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.760 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.760 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:57.760 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc30087700 10 librados: watch_flush enter 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc30087700 10 librados: watch_flush exit 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc30087700 20 client.24005.objecter shutdown clearing up homeless session... 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc30087700 10 client.24005.objecter successfully canceled tick 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.718+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc0c03a2f0 msgr2=0x7fdc0c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc0c03a2f0 0x7fdc0c03c7b0 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 crypto rx=0x7fdc28125b20 tx=0x7fdc2000b040 comp rx=0 tx=0).stop 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 10 monclient: shutdown 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 msgr2=0x7fdc281243c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc281243c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800b570 tx=0x7fdc1800b930 comp rx=0 tx=0).stop 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc2e624700 1 -- 172.21.15.47:0/4281060378 reap_dead start 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 shutdown_connections 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 --2- 172.21.15.47:0/4281060378 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc28071730 0x7fdc281243c0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 >> 172.21.15.47:0/4281060378 conn(0x7fdc280a3f90 msgr2=0x7fdc28066ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 shutdown_connections 2024-04-20T02:22:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 -- 172.21.15.47:0/4281060378 wait complete. 2024-04-20T02:22:57.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.719+0000 7fdc30087700 1 librados: shutdown 2024-04-20T02:22:57.763 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:57.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-04-20T02:22:57.763 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:22:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.900+0000 7fdddad7f700 10 monclient: get_monmap_and_config 2024-04-20T02:22:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.900+0000 7fdddad7f700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.900+0000 7fdddad7f700 10 monclient: monmap: 2024-04-20T02:22:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.900 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:57.901354+0000 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:57.901354+0000 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.901 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:57.902 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:57.902 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 Processor -- start 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 -- start start 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient: init 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient: _add_conns ranks=[3,7,2,1,5,0,10,9,4,8,6] 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd40a2950 0x7fddd40a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): picked mon.noname-g con 0x7fddd40a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd40a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): picked mon.noname-i con 0x7fddd4067b90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 0x7fddd4137f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): picked mon.noname-b con 0x7fddd4137b00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4072810 con 0x7fddd4137b00 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40a37b0 con 0x7fddd40a2950 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fddd409f710 con 0x7fddd4067b90 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fdddad7f700 10 monclient(hunting): authenticate will time out at 2057.302757s 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 0x7fddd4137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 0x7fddd4137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58148/0 (socket says 172.21.15.47:58148) 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 1 -- 172.21.15.47:0/1248191063 learned_addr learned my addr 172.21.15.47:0/1248191063 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 10 monclient(hunting): get_auth_request con 0x7fddd4137b00 auth_method 0 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.901+0000 7fddd931c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd8b1b700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd40a2950 0x7fddd40a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd3fff700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd40a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd8b1b700 10 monclient(hunting): get_auth_request con 0x7fddd40a2950 auth_method 0 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd8b1b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd8b1b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd8b1b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd3fff700 10 monclient(hunting): get_auth_request con 0x7fddd4067b90 auth_method 0 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 10 monclient(hunting): handle_auth_done global_id 24014 payload 274 2024-04-20T02:22:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 -- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 msgr2=0x7fddd40a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd40a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 -- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd40a2950 msgr2=0x7fddd40a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd40a2950 0x7fddd40a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 monclient: found mon.noname-b 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 -- 172.21.15.47:0/1248191063 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd40751e0 con 0x7fddd4137b00 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd8b1b700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd40a2950 0x7fddd40a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd931c700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 0x7fddd4137f60 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009e80 tx=0x7fddcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75bf94d4128b424e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 1 -- 172.21.15.47:0/1248191063 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddcc00e070 con 0x7fddd4137b00 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:22:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: dump: 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 1 -- 172.21.15.47:0/1248191063 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc0099b0 con 0x7fddd4137b00 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fdddad7f700 5 monclient: authenticate success, global_id 24014 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fdddad7f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fdddad7f700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fdddad7f700 4 set_mon_vals no callback set 2024-04-20T02:22:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 1 -- 172.21.15.47:0/1248191063 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddcc00be30 con 0x7fddd4137b00 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.902+0000 7fddd37fe700 10 monclient: dump: 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 10 monclient: shutdown 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 -- 172.21.15.47:0/1248191063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 msgr2=0x7fddd4137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 0x7fddd4137f60 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009e80 tx=0x7fddcc00b330 comp rx=0 tx=0).stop 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 -- 172.21.15.47:0/1248191063 shutdown_connections 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fddd4137b00 0x7fddd4137f60 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd40a2950 0x7fddd40a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 --2- 172.21.15.47:0/1248191063 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd40a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 -- 172.21.15.47:0/1248191063 >> 172.21.15.47:0/1248191063 conn(0x7fddd40a7570 msgr2=0x7fddd40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 -- 172.21.15.47:0/1248191063 shutdown_connections 2024-04-20T02:22:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.903+0000 7fdddad7f700 1 -- 172.21.15.47:0/1248191063 wait complete. 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 10 monclient: build_initial_monmap 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 10 monclient: monmap: 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 1 librados: starting msgr at 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 1 librados: starting objecter 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 1 Processor -- start 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 1 -- start start 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 1 librados: setting wanted keys 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 1 librados: calling monclient init 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.904+0000 7fdddad7f700 10 monclient: init 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient: _add_conns ranks=[7,1,5,8,2,0,3,4,6] 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd41cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): picked mon.noname-i con 0x7fddd4067b90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd40a2950 0x7fddd4136e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): picked mon.noname-f con 0x7fddd40a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fddd4137b00 0x7fddd4137340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): picked mon.noname-h con 0x7fddd4137b00 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4124be0 con 0x7fddd40a2950 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4125d10 con 0x7fddd4137b00 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fddd41cd080 con 0x7fddd4067b90 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fdddad7f700 10 monclient(hunting): authenticate will time out at 2057.306190s 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd931c700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fddd4137b00 msgr2=0x7fddd4137340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd931c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fddd4137b00 0x7fddd4137340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd41cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd40a2950 0x7fddd4136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd40a2950 0x7fddd4136e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:39624/0 (socket says 172.21.15.47:39624) 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd41cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:42688/0 (socket says 172.21.15.47:42688) 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 -- 172.21.15.47:0/3834211985 learned_addr learned my addr 172.21.15.47:0/3834211985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): get_auth_request con 0x7fddd40a2950 auth_method 0 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): get_auth_request con 0x7fddd4067b90 auth_method 0 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient(hunting): handle_auth_done global_id 54469 payload 824 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 msgr2=0x7fddd41cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fddd4067b90 0x7fddd41cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fddd4137b00 msgr2=0x7fddd4137340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fddd4137b00 0x7fddd4137340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 monclient: found mon.noname-f 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.905+0000 7fddd3fff700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc009b30 con 0x7fddd40a2950 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd3fff700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd40a2950 0x7fddd4136e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddc800e810 tx=0x7fddc800eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 client.?.objecter ms_handle_connect 0x7fddd40a2950 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddc800c070 con 0x7fddd40a2950 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: dump: 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: _finish_auth 0 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: _check_auth_tickets 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 5 monclient: authenticate success, global_id 54469 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc8013880 con 0x7fddd40a2950 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 10 monclient: _renew_subs 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd409dd40 con 0x7fddd40a2950 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddc8013e60 con 0x7fddd40a2950 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd2ffd700 4 set_mon_vals no callback set 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fddd17fa700 10 monclient: dump: 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 10 client.54469.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 10 monclient: _renew_subs 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd41265b0 con 0x7fddd40a2950 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.906+0000 7fdddad7f700 1 librados: init done 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fdddad7f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fdddad7f700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 0x7fddd4122760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4068310 con 0x7fddd406aac0 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fddc80139f0 con 0x7fddd40a2950 2024-04-20T02:22:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 0x7fddd4122760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient: get_auth_request con 0x7fddd406aac0 auth_method 0 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fddc003a320 0x7fddc003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(304..304 src has 1..304) v4 ==== 6645+0+0 (secure 0 0 0) 0x7fddc804a4d0 con 0x7fddd40a2950 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 10 client.54469.objecter ms_dispatch 0x7fddd4001550 osd_map(304..304 src has 1..304) v4 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 3 client.54469.objecter handle_osd_map got epochs [304,304] > 0 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 3 client.54469.objecter handle_osd_map decoding full epoch 304 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_done global_id 22134 payload 824 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddd4067560 con 0x7fddd406aac0 2024-04-20T02:22:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd931c700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fddc003a320 0x7fddc003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd931c700 10 monclient: get_auth_request con 0x7fddc003a320 auth_method 0 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 20 client.54469.objecter dump_active .. 0 homeless 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 10 client.54469.objecter ms_handle_connect 0x7fddd406aac0 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd17fa700 10 client.54469.objecter resend_mon_ops 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.907+0000 7fddd8b1b700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 0x7fddd4122760 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fddc4010820 tx=0x7fddc4010be0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddc400ec60 con 0x7fddd406aac0 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd931c700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fddc003a320 0x7fddc003c7e0 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009e80 tx=0x7fddcc009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 10 monclient: dump: 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 10 client.54469.objecter ms_handle_connect 0x7fddc003a320 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fddc400edc0 con 0x7fddd406aac0 2024-04-20T02:22:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 msgr2=0x7fddd4122760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 0x7fddd4122760 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fddc4010820 tx=0x7fddc4010be0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 msgr2=0x7fddd4122760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.908+0000 7fddd17fa700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd406aac0 0x7fddd4122760 secure :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fddc4010820 tx=0x7fddc4010be0 comp rx=0 tx=0).stop 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fdddad7f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fdddad7f700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 0x7fddd41230b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fdddad7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40a0cc0 con 0x7fddd4122ca0 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 0x7fddd41230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient: get_auth_request con 0x7fddd4122ca0 auth_method 0 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.916+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd8b1b700 10 monclient(hunting): handle_auth_done global_id 22143 payload 824 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd8b1b700 1 -- 172.21.15.47:0/3834211985 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fddd409ee70 con 0x7fddd4122ca0 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 client.54469.objecter ms_handle_connect 0x7fddd4122ca0 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 client.54469.objecter resend_mon_ops 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd8b1b700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 0x7fddd41230b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fddc4010820 tx=0x7fddc4004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fddc4021da0 con 0x7fddd4122ca0 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 monclient: dump: 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7fddc4021930 con 0x7fddd4122ca0 2024-04-20T02:22:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 msgr2=0x7fddd41230b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 0x7fddd41230b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fddc4010820 tx=0x7fddc4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 msgr2=0x7fddd41230b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.917+0000 7fddd17fa700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fddd4122ca0 0x7fddd41230b0 secure :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fddc4010820 tx=0x7fddc4004750 comp rx=0 tx=0).stop 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:57.938 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:57.961 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:57.961 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 10 librados: watch_flush enter 2024-04-20T02:22:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 10 librados: watch_flush exit 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 20 client.54469.objecter shutdown clearing up homeless session... 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 10 client.54469.objecter successfully canceled tick 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fddc003a320 msgr2=0x7fddc003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fddc003a320 0x7fddc003c7e0 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009e80 tx=0x7fddcc009000 comp rx=0 tx=0).stop 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 10 monclient: shutdown 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd40a2950 msgr2=0x7fddd4136e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 --2- 172.21.15.47:0/3834211985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fddd40a2950 0x7fddd4136e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddc800e810 tx=0x7fddc800eb20 comp rx=0 tx=0).stop 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fddd931c700 1 -- 172.21.15.47:0/3834211985 reap_dead start 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fddd931c700 1 -- 172.21.15.47:0/3834211985 reap_dead start 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 shutdown_connections 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 >> 172.21.15.47:0/3834211985 conn(0x7fddd40a7570 msgr2=0x7fddd41bcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 shutdown_connections 2024-04-20T02:22:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 -- 172.21.15.47:0/3834211985 wait complete. 2024-04-20T02:22:57.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:57.918+0000 7fdddad7f700 1 librados: shutdown 2024-04-20T02:22:57.964 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:57.964 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-04-20T02:22:57.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.112+0000 7f9566ae8700 10 monclient: get_monmap_and_config 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.112+0000 7f9566ae8700 10 monclient: build_initial_monmap 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.112+0000 7f9566ae8700 10 monclient: monmap: 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:22:58.113309+0000 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:22:58.113309+0000 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:22:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:22:58.114 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.112+0000 7f9566ae8700 1 Processor -- start 2024-04-20T02:22:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 -- start start 2024-04-20T02:22:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient: init 2024-04-20T02:22:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient: _add_conns ranks=[2,3,9,8,5,4,10,7,0,6,1] 2024-04-20T02:22:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 0x7f95600a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): picked mon.noname-b con 0x7f95600a0780 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95600a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): picked mon.noname-g con 0x7f95600a0b60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95600a2950 0x7f95600a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): picked mon.noname-a con 0x7f95600a2950 addr v2:172.21.15.47:3300/0 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9560067770 con 0x7f95600a2950 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956009eee0 con 0x7f95600a0780 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956009f1c0 con 0x7f95600a0b60 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9566ae8700 10 monclient(hunting): authenticate will time out at 2057.514627s 2024-04-20T02:22:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95600a2950 0x7f95600a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 0x7f95600a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f95600a2950 0x7f95600a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35010/0 (socket says 172.21.15.47:35010) 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 0x7f95600a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:58160/0 (socket says 172.21.15.47:58160) 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 1 -- 172.21.15.47:0/443684901 learned_addr learned my addr 172.21.15.47:0/443684901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f955ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95600a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): get_auth_request con 0x7f95600a2950 auth_method 0 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 10 monclient(hunting): get_auth_request con 0x7f95600a0780 auth_method 0 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f955ffff700 10 monclient(hunting): get_auth_request con 0x7f95600a0b60 auth_method 0 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f955ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f955ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f955ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9565085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.113+0000 7f9564884700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9564884700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 10 monclient(hunting): handle_auth_done global_id 35568 payload 274 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 -- 172.21.15.47:0/443684901 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 msgr2=0x7f95600a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95600a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 -- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 msgr2=0x7f95600a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 0x7f95600a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 10 monclient: _finish_hunting 0 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 monclient: found mon.noname-a 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 -- 172.21.15.47:0/443684901 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600751e0 con 0x7f95600a2950 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9564884700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 0x7f95600a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9565085700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600a2950 0x7f95600a2e00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9558009e50 tx=0x7f955800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2744b69a034e4218 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 1 -- 172.21.15.47:0/443684901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f955800e070 con 0x7f95600a2950 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: dump: 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9566ae8700 5 monclient: authenticate success, global_id 35568 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9566ae8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 1 -- 172.21.15.47:0/443684901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955800bd80 con 0x7f95600a2950 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9566ae8700 10 monclient: get_monmap_and_config success 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 1 -- 172.21.15.47:0/443684901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9558012860 con 0x7f95600a2950 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f9566ae8700 4 set_mon_vals no callback set 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.114+0000 7f955f7fe700 10 monclient: dump: 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 10 monclient: shutdown 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 -- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600a2950 msgr2=0x7f95600a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600a2950 0x7f95600a2e00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9558009e50 tx=0x7f955800b330 comp rx=0 tx=0).stop 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 -- 172.21.15.47:0/443684901 shutdown_connections 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95600a2950 0x7f95600a2e00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f95600a0780 0x7f95600a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 --2- 172.21.15.47:0/443684901 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95600a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 -- 172.21.15.47:0/443684901 >> 172.21.15.47:0/443684901 conn(0x7f95600a7570 msgr2=0x7f95600a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 -- 172.21.15.47:0/443684901 shutdown_connections 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.115+0000 7f9566ae8700 1 -- 172.21.15.47:0/443684901 wait complete. 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 10 monclient: build_initial_monmap 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 10 monclient: monmap: 2024-04-20T02:22:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:22:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 1 librados: starting msgr at 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 1 librados: starting objecter 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 1 Processor -- start 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 1 -- start start 2024-04-20T02:22:58.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 1 librados: setting wanted keys 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 1 librados: calling monclient init 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.116+0000 7f9566ae8700 10 monclient: init 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient: _reopen_session rank -1 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient: _add_conns ranks=[8,3,5,7,4,6,1,0,2] 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95600a0780 0x7f95601cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): picked mon.noname-e con 0x7f95600a0780 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95601bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): picked mon.noname-g con 0x7f95600a0b60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f95600a2950 0x7f95601be050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): picked mon.noname-h con 0x7f95600a2950 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95601acc70 con 0x7f95600a0b60 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f956009e440 con 0x7f95600a2950 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f956009e900 con 0x7f95600a0780 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): _renew_subs 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9566ae8700 10 monclient(hunting): authenticate will time out at 2057.518490s 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9565085700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f95600a2950 msgr2=0x7f95601be050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9565085700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f95600a2950 0x7f95601be050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95600a0780 0x7f95601cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95601bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95600a0780 0x7f95601cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:44226/0 (socket says 172.21.15.47:44226) 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95601bdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:33974/0 (socket says 172.21.15.47:33974) 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 1 -- 172.21.15.47:0/4283589743 learned_addr learned my addr 172.21.15.47:0/4283589743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): get_auth_request con 0x7f95600a0780 auth_method 0 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): get_auth_request con 0x7f95600a0b60 auth_method 0 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.117+0000 7f9564884700 10 monclient(hunting): handle_auth_done global_id 35207 payload 824 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f95600a2950 msgr2=0x7f95601be050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f95600a2950 0x7f95601be050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 msgr2=0x7f95601bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95601bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 10 monclient: _finish_hunting 0 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 monclient: found mon.noname-e 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:22:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540094a0 con 0x7f95600a0780 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955ffff700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95600a0b60 0x7f95601bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9564884700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95600a0780 0x7f95601cc210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f955400ebe0 tx=0x7f955400eef0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 client.?.objecter ms_handle_connect 0x7f95600a0780 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9554009800 con 0x7f95600a0780 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: dump: 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400cd80 con 0x7f95600a0780 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 5 monclient: authenticate success, global_id 35207 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 10 monclient: _renew_subs 2024-04-20T02:22:58.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9558009b30 con 0x7f95600a0780 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95540138e0 con 0x7f95600a0780 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955effd700 4 set_mon_vals no callback set 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f955d7fa700 10 monclient: dump: 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:58.144 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 10 client.35207.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 10 monclient: _renew_subs 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95601aba90 con 0x7f95600a0780 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.118+0000 7f9566ae8700 1 librados: init done 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f9566ae8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:58.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f9566ae8700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 0x7f95601a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9560068310 con 0x7f956006aaa0 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9554013470 con 0x7f95600a0780 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 0x7f95601a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient: get_auth_request con 0x7f956006aaa0 auth_method 0 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f954403a2f0 0x7f954403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(305..305 src has 1..305) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f955404aa20 con 0x7f95600a0780 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 10 client.35207.objecter ms_dispatch 0x7f9560001550 osd_map(305..305 src has 1..305) v4 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 3 client.35207.objecter handle_osd_map got epochs [305,305] > 0 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 3 client.35207.objecter handle_osd_map decoding full epoch 305 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955d7fa700 20 client.35207.objecter dump_active .. 0 homeless 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f9565085700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f954403a2f0 0x7f954403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f9565085700 10 monclient: get_auth_request con 0x7f954403a2f0 auth_method 0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 10 monclient(hunting): handle_auth_done global_id 54358 payload 824 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.119+0000 7f955ffff700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95600a4450 con 0x7f956006aaa0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 client.35207.objecter ms_handle_connect 0x7f956006aaa0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 client.35207.objecter resend_mon_ops 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955ffff700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 0x7f95601a95f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f95601aad10 tx=0x7f954c00f520 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 client.35207.objecter ms_handle_connect 0x7f954403a2f0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f9565085700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f954403a2f0 0x7f954403c7b0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f95601ab530 tx=0x7f9558009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954c0044a0 con 0x7f956006aaa0 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 monclient: dump: 2024-04-20T02:22:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f954c00fc10 con 0x7f956006aaa0 2024-04-20T02:22:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:22:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:22:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 msgr2=0x7f95601a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 0x7f95601a95f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f95601aad10 tx=0x7f954c00f520 comp rx=0 tx=0).stop 2024-04-20T02:22:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 msgr2=0x7f95601a95f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:58.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.120+0000 7f955d7fa700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f956006aaa0 0x7f95601a95f0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f95601aad10 tx=0x7f954c00f520 comp rx=0 tx=0).stop 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f9566ae8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f9566ae8700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 0x7f95601a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f9566ae8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f956006a400 con 0x7f95601a9b30 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 0x7f95601a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient: get_auth_request con 0x7f95601a9b30 auth_method 0 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:22:58.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.128+0000 7f955ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955ffff700 10 monclient(hunting): handle_auth_done global_id 54367 payload 824 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955ffff700 1 -- 172.21.15.47:0/4283589743 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9560067c90 con 0x7f95601a9b30 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955ffff700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 0x7f95601a9f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f954c0091e0 tx=0x7f954c004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 client.35207.objecter ms_handle_connect 0x7f95601a9b30 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 client.35207.objecter resend_mon_ops 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954c01e8c0 con 0x7f95601a9b30 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 monclient: dump: 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:22:58.152 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7f954c01e450 con 0x7f95601a9b30 2024-04-20T02:22:58.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 msgr2=0x7f95601a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 0x7f95601a9f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f954c0091e0 tx=0x7f954c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 msgr2=0x7f95601a9f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f955d7fa700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95601a9b30 0x7f95601a9f40 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f954c0091e0 tx=0x7f954c004750 comp rx=0 tx=0).stop 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 60, 2024-04-20T02:22:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:22:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:22:58.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:22:58.173 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:22:58.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:22:58.174 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:22:58.174 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:22:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f9566ae8700 10 librados: watch_flush enter 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f9566ae8700 10 librados: watch_flush exit 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f9566ae8700 20 client.35207.objecter shutdown clearing up homeless session... 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.129+0000 7f9566ae8700 10 client.35207.objecter successfully canceled tick 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f954403a2f0 msgr2=0x7f954403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f954403a2f0 0x7f954403c7b0 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 crypto rx=0x7f95601ab530 tx=0x7f9558009000 comp rx=0 tx=0).stop 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 10 monclient: shutdown 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95600a0780 msgr2=0x7f95601cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 --2- 172.21.15.47:0/4283589743 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95600a0780 0x7f95601cc210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f955400ebe0 tx=0x7f955400eef0 comp rx=0 tx=0).stop 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9565085700 1 -- 172.21.15.47:0/4283589743 reap_dead start 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9565085700 1 -- 172.21.15.47:0/4283589743 reap_dead start 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 shutdown_connections 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 >> 172.21.15.47:0/4283589743 conn(0x7f95600a7570 msgr2=0x7f956006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:22:58.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 shutdown_connections 2024-04-20T02:22:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 -- 172.21.15.47:0/4283589743 wait complete. 2024-04-20T02:22:58.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:22:58.130+0000 7f9566ae8700 1 librados: shutdown 2024-04-20T02:22:58.177 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:22:58.177 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:23:13.139 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ PP2 2024-04-20T02:23:13.139 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (6035 ms) 2024-04-20T02:23:13.139 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-04-20T02:23:13.139 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (6892 ms) 2024-04-20T02:23:13.139 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-04-20T02:23:13.139 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (16044 ms) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (12349 ms) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (21683 ms) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (6949 ms) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7076 ms) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (214989 ms total) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (391101 ms total) 2024-04-20T02:23:13.140 INFO:tasks.workunit.client.0.smithi179.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46822 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46822 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47286 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47286 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46995 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46995 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46953 2024-04-20T02:23:13.141 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46953 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46846 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46846 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47137 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47137 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46814 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46814 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47192 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47192 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47043 2024-04-20T02:23:13.142 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47043 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47313 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47313 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47173 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47173 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47364 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47364 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46798 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46798 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47338 2024-04-20T02:23:13.143 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47338 2024-04-20T02:23:13.144 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.144 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=47205 2024-04-20T02:23:13.144 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 47205 2024-04-20T02:23:13.144 INFO:tasks.workunit.client.0.smithi179.stderr:+ for t in "${!pids[@]}" 2024-04-20T02:23:13.144 INFO:tasks.workunit.client.0.smithi179.stderr:+ pid=46980 2024-04-20T02:23:13.144 INFO:tasks.workunit.client.0.smithi179.stderr:+ wait 46980 2024-04-20T02:24:28.178 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:24:28.179 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-04-20T02:24:28.179 INFO:tasks.ceph.mon.h:Restarting daemon 2024-04-20T02:24:28.179 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-04-20T02:24:28.183 INFO:tasks.ceph.mon.h:Started 2024-04-20T02:24:28.183 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:24:28.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:24:28.292 INFO:tasks.ceph.mon.h.smithi179.stderr:2024-04-20T02:24:28.291+0000 7f789f614cc0 -1 mon.h@-1(???).log v0 log_external_backlog rewinding external_log_to from 293 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:24:28.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.327+0000 7fa5fa450700 10 monclient: get_monmap_and_config 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.327+0000 7fa5fa450700 10 monclient: build_initial_monmap 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.327+0000 7fa5fa450700 10 monclient: monmap: 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:28.328962+0000 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:28.328962+0000 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:28.328 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:28.329 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.328+0000 7fa5fa450700 1 Processor -- start 2024-04-20T02:24:28.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.328+0000 7fa5fa450700 1 -- start start 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.328+0000 7fa5fa450700 10 monclient: init 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient: _add_conns ranks=[4,5,9,3,6,10,2,0,8,7,1] 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 0x7fa5f40a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): picked mon.noname-c con 0x7fa5f40a5370 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 0x7fa5f40a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): picked mon.noname-h con 0x7fa5f40a40b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa5f4135920 0x7fa5f4135da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): picked mon.noname-a con 0x7fa5f4135920 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:28.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40a49d0 con 0x7fa5f4135920 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f40a5750 con 0x7fa5f40a5370 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f409f160 con 0x7fa5f40a40b0 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5fa450700 10 monclient(hunting): authenticate will time out at 2147.730344s 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa5f4135920 0x7fa5f4135da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 0x7fa5f40a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa5f4135920 0x7fa5f4135da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58766/0 (socket says 172.21.15.47:58766) 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 0x7fa5f40a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:51626/0 (socket says 172.21.15.47:51626) 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 -- 172.21.15.47:0/748608180 learned_addr learned my addr 172.21.15.47:0/748608180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): get_auth_request con 0x7fa5f40a5370 auth_method 0 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): get_auth_request con 0x7fa5f4135920 auth_method 0 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 0x7fa5f40a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 10 monclient(hunting): get_auth_request con 0x7fa5f40a40b0 auth_method 0 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 1 -- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 msgr2=0x7fa5f40a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 1 -- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 msgr2=0x7fa5f40a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 0x7fa5f40a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f37fe700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 0x7fa5f40a4490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_done global_id 35586 payload 274 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 -- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 msgr2=0x7fa5f40a4490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 0x7fa5f40a4490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 -- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 msgr2=0x7fa5f40a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 0x7fa5f40a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient: _finish_hunting 0 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 monclient: found mon.noname-a 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f89ed700 1 -- 172.21.15.47:0/748608180 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40751e0 con 0x7fa5f4135920 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.329+0000 7fa5f3fff700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 0x7fa5f40a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f89ed700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f4135920 0x7fa5f4135da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009e50 tx=0x7fa5ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e56444429f765e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 1 -- 172.21.15.47:0/748608180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5ec00e070 con 0x7fa5f4135920 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: dump: 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:28.343 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 1 -- 172.21.15.47:0/748608180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec00bd80 con 0x7fa5f4135920 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 5 monclient: authenticate success, global_id 35586 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 1 -- 172.21.15.47:0/748608180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5ec012860 con 0x7fa5f4135920 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 4 set_mon_vals no callback set 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5f2ffd700 10 monclient: dump: 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 10 monclient: shutdown 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 1 -- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f4135920 msgr2=0x7fa5f4135da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f4135920 0x7fa5f4135da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009e50 tx=0x7fa5ec00b330 comp rx=0 tx=0).stop 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 1 -- 172.21.15.47:0/748608180 shutdown_connections 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa5f4135920 0x7fa5f4135da0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f40a5370 0x7fa5f40a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.330+0000 7fa5fa450700 1 --2- 172.21.15.47:0/748608180 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5f40a40b0 0x7fa5f40a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 1 -- 172.21.15.47:0/748608180 >> 172.21.15.47:0/748608180 conn(0x7fa5f40a8790 msgr2=0x7fa5f40a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 1 -- 172.21.15.47:0/748608180 shutdown_connections 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 1 -- 172.21.15.47:0/748608180 wait complete. 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 10 monclient: build_initial_monmap 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 10 monclient: monmap: 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:28.347 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 1 librados: starting msgr at 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.331+0000 7fa5fa450700 1 librados: starting objecter 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 Processor -- start 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 -- start start 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 librados: setting wanted keys 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 librados: calling monclient init 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient: init 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient: _add_conns ranks=[7,6,4,5,8,1,2,3,0] 2024-04-20T02:24:28.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5f40a40b0 0x7fa5f4131f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): picked mon.noname-i con 0x7fa5f40a40b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa5f40a5370 0x7fa5f41324d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): picked mon.noname-d con 0x7fa5f40a5370 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f4135920 0x7fa5f41c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): picked mon.noname-c con 0x7fa5f4135920 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f41216c0 con 0x7fa5f4135920 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f409eda0 con 0x7fa5f40a5370 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f41230f0 con 0x7fa5f40a40b0 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5fa450700 10 monclient(hunting): authenticate will time out at 2147.733760s 2024-04-20T02:24:28.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa5f40a5370 0x7fa5f41324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f89ed700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f4135920 0x7fa5f41c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa5f40a5370 0x7fa5f41324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45034/0 (socket says 172.21.15.47:45034) 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 1 -- 172.21.15.47:0/2260610481 learned_addr learned my addr 172.21.15.47:0/2260610481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 10 monclient(hunting): get_auth_request con 0x7fa5f40a5370 auth_method 0 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f89ed700 10 monclient(hunting): get_auth_request con 0x7fa5f4135920 auth_method 0 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f89ed700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f89ed700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.332+0000 7fa5f89ed700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5f40a40b0 0x7fa5f4131f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:28.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): get_auth_request con 0x7fa5f40a40b0 auth_method 0 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 10 monclient(hunting): handle_auth_done global_id 54373 payload 824 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 -- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5f40a40b0 msgr2=0x7fa5f4131f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5f40a40b0 0x7fa5f4131f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 -- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa5f40a5370 msgr2=0x7fa5f41324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa5f40a5370 0x7fa5f41324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 10 monclient: _finish_hunting 0 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 monclient: found mon.noname-c 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 -- 172.21.15.47:0/2260610481 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40094a0 con 0x7fa5f4135920 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f89ed700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f4135920 0x7fa5f41c6bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec015000 tx=0x7fa5ec004ad0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 client.?.objecter ms_handle_connect 0x7fa5f4135920 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5ec0092e0 con 0x7fa5f4135920 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: dump: 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:28.352 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: _finish_auth 0 2024-04-20T02:24:28.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec0046c0 con 0x7fa5f4135920 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 5 monclient: authenticate success, global_id 54373 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 10 monclient: _renew_subs 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec009b30 con 0x7fa5f4135920 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5ec004830 con 0x7fa5f4135920 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f0ff9700 10 monclient: dump: 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.354 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5f27fc700 4 set_mon_vals no callback set 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 10 client.54373.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 10 monclient: _renew_subs 2024-04-20T02:24:28.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f41236b0 con 0x7fa5f4135920 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.333+0000 7fa5fa450700 1 librados: init done 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5fa450700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5fa450700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5fa450700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f40a1f50 con 0x7fa5f4135920 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa5ec004250 con 0x7fa5f4135920 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5f0ff9700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5e003a2f0 0x7fa5e003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fa5ec04be40 con 0x7fa5f4135920 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5f0ff9700 10 client.54373.objecter ms_dispatch 0x7fa5f4001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5f0ff9700 3 client.54373.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.334+0000 7fa5f0ff9700 3 client.54373.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.335+0000 7fa5f0ff9700 20 client.54373.objecter dump_active .. 0 homeless 2024-04-20T02:24:28.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.335+0000 7fa5f3fff700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5e003a2f0 0x7fa5e003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.335+0000 7fa5f3fff700 10 monclient: get_auth_request con 0x7fa5e003a2f0 auth_method 0 2024-04-20T02:24:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.335+0000 7fa5f3fff700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5e003a2f0 0x7fa5e003c7b0 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009cf0 tx=0x7fa5e4009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.335+0000 7fa5f0ff9700 10 client.54373.objecter ms_handle_connect 0x7fa5e003a2f0 2024-04-20T02:24:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.339+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa5ec004500 con 0x7fa5f4135920 2024-04-20T02:24:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.339+0000 7fa5f0ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:28.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.339+0000 7fa5f0ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.640+0000 7fa5fa450700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.640+0000 7fa5fa450700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.640+0000 7fa5fa450700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.640+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa5f41239c0 con 0x7fa5f4135920 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.641+0000 7fa5f0ff9700 1 -- 172.21.15.47:0/2260610481 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3170 (secure 0 0 0) 0x7fa5ec0237d0 con 0x7fa5f4135920 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.641+0000 7fa5f0ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.641+0000 7fa5f0ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:28.641 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":60,"quorum":[0,1,2,3,4,6,7,8],"quorum_names":["a","f","b","g","c","d","i","e"],"quorum_leader_name":"b","quorum_age":92,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:24:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 10 librados: watch_flush enter 2024-04-20T02:24:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 10 librados: watch_flush exit 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 20 client.54373.objecter shutdown clearing up homeless session... 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 10 client.54373.objecter successfully canceled tick 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5e003a2f0 msgr2=0x7fa5e003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5e003a2f0 0x7fa5e003c7b0 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009cf0 tx=0x7fa5e4009210 comp rx=0 tx=0).stop 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 10 monclient: shutdown 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f4135920 msgr2=0x7fa5f41c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f4135920 0x7fa5f41c6bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec015000 tx=0x7fa5ec004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 shutdown_connections 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5e003a2f0 0x7fa5e003c7b0 unknown :-1 s=CLOSED pgs=3209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa5f4135920 0x7fa5f41c6bb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5f40a40b0 0x7fa5f4131f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 --2- 172.21.15.47:0/2260610481 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa5f40a5370 0x7fa5f41324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 >> 172.21.15.47:0/2260610481 conn(0x7fa5f40a8790 msgr2=0x7fa5f40a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 shutdown_connections 2024-04-20T02:24:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 -- 172.21.15.47:0/2260610481 wait complete. 2024-04-20T02:24:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:28.643+0000 7fa5fa450700 1 librados: shutdown 2024-04-20T02:24:31.658 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:24:31.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.802+0000 7f1388faf700 10 monclient: get_monmap_and_config 2024-04-20T02:24:31.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.802+0000 7f1388faf700 10 monclient: build_initial_monmap 2024-04-20T02:24:31.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.802+0000 7f1388faf700 10 monclient: monmap: 2024-04-20T02:24:31.802 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:31.802 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:31.803289+0000 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:31.803289+0000 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:31.803 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:31.804 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:31.804 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:31.804 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:31.804 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.802+0000 7f1388faf700 1 Processor -- start 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.802+0000 7f1388faf700 1 -- start start 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.802+0000 7f1388faf700 10 monclient: init 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient: _add_conns ranks=[3,8,1,7,4,0,5,2,10,9,6] 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f138409f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): picked mon.noname-g con 0x7f1384066610 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 0x7f13840a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): picked mon.noname-e con 0x7f13840a11e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f13840a1ba0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13840751e0 con 0x7f13840a1ba0 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138409a000 con 0x7f1384066610 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f13840a2010 con 0x7f13840a11e0 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1388faf700 10 monclient(hunting): authenticate will time out at 2151.204664s 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 0x7f13840a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 0x7f13840a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37726/0 (socket says 172.21.15.47:37726) 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 1 -- 172.21.15.47:0/3356271047 learned_addr learned my addr 172.21.15.47:0/3356271047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): get_auth_request con 0x7f13840a11e0 auth_method 0 2024-04-20T02:24:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.803+0000 7f1381d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 10 monclient(hunting): handle_auth_done global_id 35225 payload 274 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 1 -- 172.21.15.47:0/3356271047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 msgr2=0x7f138409f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 1 --2- 172.21.15.47:0/3356271047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f138409f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 1 -- 172.21.15.47:0/3356271047 >> v1:172.21.15.179:6789/0 conn(0x7f13840a1ba0 legacy=0x7f138413e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 1 monclient: found mon.noname-e 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 1 -- 172.21.15.47:0/3356271047 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13840669f0 con 0x7f13840a11e0 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1381d9b700 1 --2- 172.21.15.47:0/3356271047 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 0x7f13840a1660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1378009920 tx=0x7f1378009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1ca1c6d90f852665 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 1 -- 172.21.15.47:0/3356271047 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1378009e70 con 0x7f13840a11e0 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: dump: 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:31.816 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: _finish_auth 0 2024-04-20T02:24:31.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: _check_auth_tickets 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 1 -- 172.21.15.47:0/3356271047 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13780044a0 con 0x7f13840a11e0 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 1 -- 172.21.15.47:0/3356271047 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f137800bc90 con 0x7f13840a11e0 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1388faf700 5 monclient: authenticate success, global_id 35225 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1388faf700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f1388faf700 4 set_mon_vals no callback set 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.804+0000 7f138159a700 10 monclient: dump: 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.818 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 10 monclient: shutdown 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:31.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 -- 172.21.15.47:0/3356271047 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 msgr2=0x7f13840a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 --2- 172.21.15.47:0/3356271047 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 0x7f13840a1660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1378009920 tx=0x7f1378009c30 comp rx=0 tx=0).stop 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 -- 172.21.15.47:0/3356271047 shutdown_connections 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 --2- 172.21.15.47:0/3356271047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f138409f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 --2- 172.21.15.47:0/3356271047 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f13840a11e0 0x7f13840a1660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 -- 172.21.15.47:0/3356271047 >> 172.21.15.47:0/3356271047 conn(0x7f13840a6720 msgr2=0x7f13840a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 -- 172.21.15.47:0/3356271047 shutdown_connections 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.805+0000 7f1388faf700 1 -- 172.21.15.47:0/3356271047 wait complete. 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 10 monclient: build_initial_monmap 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 10 monclient: monmap: 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:31.820 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 1 librados: starting msgr at 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 1 librados: starting objecter 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 1 Processor -- start 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 1 -- start start 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 1 librados: setting wanted keys 2024-04-20T02:24:31.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 1 librados: calling monclient init 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.806+0000 7f1388faf700 10 monclient: init 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient: _add_conns ranks=[3,2,1,5,6,8,4,7,0] 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f1384072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): picked mon.noname-g con 0x7f1384066610 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 0x7f1384072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): picked mon.noname-b con 0x7f13840a11e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13840a1ba0 0x7f13841cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): picked mon.noname-f con 0x7f13840a1ba0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13841b8d30 con 0x7f13840a1ba0 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13840a0100 con 0x7f13840a11e0 2024-04-20T02:24:31.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138409e140 con 0x7f1384066610 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient(hunting): authenticate will time out at 2151.208182s 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 0x7f1384072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 0x7f1384072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57154/0 (socket says 172.21.15.47:57154) 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 -- 172.21.15.47:0/3811853573 learned_addr learned my addr 172.21.15.47:0/3811853573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): get_auth_request con 0x7f13840a11e0 auth_method 0 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f138259c700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f1384072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:31.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1382d9d700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13840a1ba0 0x7f13841cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f138259c700 10 monclient(hunting): get_auth_request con 0x7f1384066610 auth_method 0 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f138259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f138259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f138259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1382d9d700 10 monclient(hunting): get_auth_request con 0x7f13840a1ba0 auth_method 0 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1382d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1382d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1382d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient(hunting): handle_auth_done global_id 24032 payload 824 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 -- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 msgr2=0x7f1384072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f1384072570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 -- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13840a1ba0 msgr2=0x7f13841cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:31.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13840a1ba0 0x7f13841cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 monclient: found mon.noname-b 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 -- 172.21.15.47:0/3811853573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1378009550 con 0x7f13840a11e0 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1381d9b700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 0x7f1384072ab0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f1378009130 tx=0x7f1378004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 client.?.objecter ms_handle_connect 0x7f13840a11e0 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1378004330 con 0x7f13840a11e0 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 monclient: dump: 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:31.825 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:31.826 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378003b10 con 0x7f13840a11e0 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 5 monclient: authenticate success, global_id 24032 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient: _renew_subs 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13841ba040 con 0x7f13840a11e0 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.807+0000 7f1372ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1378003c80 con 0x7f13840a11e0 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1372ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1372ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:31.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1372ffd700 10 monclient: dump: 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:31.828 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1380d99700 4 set_mon_vals no callback set 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1388faf700 10 client.24032.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1388faf700 10 monclient: _renew_subs 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1388faf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138406fa70 con 0x7f13840a11e0 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.808+0000 7f1388faf700 1 librados: init done 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1378021070 con 0x7f13840a11e0 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1388faf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1388faf700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1388faf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:31.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138409d9e0 con 0x7f13840a11e0 2024-04-20T02:24:31.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f136403a320 0x7f136403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:31.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f137804bf00 con 0x7f13840a11e0 2024-04-20T02:24:31.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 10 client.24032.objecter ms_dispatch 0x7f1384001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 3 client.24032.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 3 client.24032.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f1372ffd700 20 client.24032.objecter dump_active .. 0 homeless 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f138259c700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f136403a320 0x7f136403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.809+0000 7f138259c700 10 monclient: get_auth_request con 0x7f136403a320 auth_method 0 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.810+0000 7f1372ffd700 10 client.24032.objecter ms_handle_connect 0x7f136403a320 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.810+0000 7f138259c700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f136403a320 0x7f136403c7e0 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 crypto rx=0x7f1374006fd0 tx=0x7f1374008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.814+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f137801adf0 con 0x7f13840a11e0 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.814+0000 7f1372ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:31.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:31.814+0000 7f1372ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:32.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1388faf700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1388faf700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1388faf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f13841b9250 con 0x7f13840a11e0 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1372ffd700 1 -- 172.21.15.47:0/3811853573 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f137801ac10 con 0x7f13840a11e0 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1372ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.129+0000 7f1372ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:32.130 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":62,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 10 librados: watch_flush enter 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 10 librados: watch_flush exit 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 20 client.24032.objecter shutdown clearing up homeless session... 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 10 client.24032.objecter successfully canceled tick 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f136403a320 msgr2=0x7f136403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f136403a320 0x7f136403c7e0 secure :-1 s=READY pgs=3211 cs=0 l=1 rev1=1 crypto rx=0x7f1374006fd0 tx=0x7f1374008040 comp rx=0 tx=0).stop 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 10 monclient: shutdown 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 msgr2=0x7f1384072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 0x7f1384072ab0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f1378009130 tx=0x7f1378004750 comp rx=0 tx=0).stop 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 shutdown_connections 2024-04-20T02:24:32.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f136403a320 0x7f136403c7e0 unknown :-1 s=CLOSED pgs=3211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13840a1ba0 0x7f13841cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1384066610 0x7f1384072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 --2- 172.21.15.47:0/3811853573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f13840a11e0 0x7f1384072ab0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 >> 172.21.15.47:0/3811853573 conn(0x7f13840a6720 msgr2=0x7f13841bfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 shutdown_connections 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 -- 172.21.15.47:0/3811853573 wait complete. 2024-04-20T02:24:32.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.132+0000 7f1388faf700 1 librados: shutdown 2024-04-20T02:24:32.145 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:24:32.146 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:24:32.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.282+0000 7f22c28e6700 10 monclient: get_monmap_and_config 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.282+0000 7f22c28e6700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.282+0000 7f22c28e6700 10 monclient: monmap: 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:32.283397+0000 2024-04-20T02:24:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:32.283397+0000 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:32.283 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 1 Processor -- start 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 1 -- start start 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient: init 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient: _add_conns ranks=[1,0,3,2,4,5,7,8,6,10,9] 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f22bc0a07f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f22bc0a0010 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc13e380 0x7f22bc140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient(hunting): picked mon.noname-g con 0x7f22bc13e380 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:32.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0751e0 con 0x7f22bc0a0010 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0672f0 con 0x7f22bc0a07f0 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc067800 con 0x7f22bc13e380 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c28e6700 10 monclient(hunting): authenticate will time out at 2151.684770s 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22bb7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22bc0a0010 0x7f22bc13cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54340/0 (socket says 172.21.15.47:54340) 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22bb7fe700 1 -- 172.21.15.47:0/1316120820 learned_addr learned my addr 172.21.15.47:0/1316120820 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c0e83700 1 --2- 172.21.15.47:0/1316120820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc13e380 0x7f22bc140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c0e83700 10 monclient(hunting): get_auth_request con 0x7f22bc13e380 auth_method 0 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c0e83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c0e83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.283+0000 7f22c0e83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932926403 0 0) 0x7f22bc0751e0 con 0x7f22bc0a0010 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): _negotiate 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): my global_id is 35595 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a40035b0 con 0x7f22bc0a0010 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754356659 0 0) 0x7f22bc0672f0 con 0x7f22bc0a07f0 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): _negotiate 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient(hunting): my global_id is 54505 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22bc0751e0 con 0x7f22bc0a07f0 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 10 monclient(hunting): handle_auth_done global_id 22188 payload 274 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 1 -- 172.21.15.47:0/1316120820 >> v1:172.21.15.179:6789/0 conn(0x7f22bc0a07f0 legacy=0x7f22bc0a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 1 -- 172.21.15.47:0/1316120820 >> v1:172.21.15.47:6789/0 conn(0x7f22bc0a0010 legacy=0x7f22bc13cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 1 monclient: found mon.noname-g 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 1 -- 172.21.15.47:0/1316120820 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0a12e0 con 0x7f22bc13e380 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c0e83700 1 --2- 172.21.15.47:0/1316120820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc13e380 0x7f22bc140770 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009dc0 tx=0x7f22b400b330 comp rx=0 tx=0).ready entity=mon.3 client_cookie=be2a3345107232d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22b400e070 con 0x7f22bc13e380 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: dump: 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c28e6700 5 monclient: authenticate success, global_id 22188 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40098f0 con 0x7f22bc13e380 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c28e6700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 1 -- 172.21.15.47:0/1316120820 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22b40128a0 con 0x7f22bc13e380 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22c28e6700 4 set_mon_vals no callback set 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.284+0000 7f22baffd700 10 monclient: dump: 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 10 monclient: shutdown 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 -- 172.21.15.47:0/1316120820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc13e380 msgr2=0x7f22bc140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 --2- 172.21.15.47:0/1316120820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc13e380 0x7f22bc140770 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009dc0 tx=0x7f22b400b330 comp rx=0 tx=0).stop 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 -- 172.21.15.47:0/1316120820 shutdown_connections 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 --2- 172.21.15.47:0/1316120820 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc13e380 0x7f22bc140770 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 -- 172.21.15.47:0/1316120820 >> 172.21.15.47:0/1316120820 conn(0x7f22bc0a3f90 msgr2=0x7f22bc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 -- 172.21.15.47:0/1316120820 shutdown_connections 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.285+0000 7f22c28e6700 1 -- 172.21.15.47:0/1316120820 wait complete. 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 10 monclient: monmap: 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.302 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 1 librados: starting msgr at 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 1 librados: starting objecter 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 1 Processor -- start 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 1 -- start start 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 1 librados: setting wanted keys 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 1 librados: calling monclient init 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.286+0000 7f22c28e6700 10 monclient: init 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient: _add_conns ranks=[1,3,5,8,6,0,2,7,4] 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22bc0a0010 0x7f22bc1cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): picked mon.noname-f con 0x7f22bc0a0010 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc0a07f0 0x7f22bc1ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): picked mon.noname-g con 0x7f22bc0a07f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 0x7f22bc13c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): picked mon.noname-h con 0x7f22bc13e380 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc122560 con 0x7f22bc0a0010 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc09e800 con 0x7f22bc0a07f0 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc09e360 con 0x7f22bc13e380 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c28e6700 10 monclient(hunting): authenticate will time out at 2151.688215s 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22bc0a0010 0x7f22bc1cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 0x7f22bc13c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc0a07f0 0x7f22bc1ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22bc0a0010 0x7f22bc1cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:58900/0 (socket says 172.21.15.47:58900) 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 1 -- 172.21.15.47:0/3624175256 learned_addr learned my addr 172.21.15.47:0/3624175256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:32.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): get_auth_request con 0x7f22bc0a0010 auth_method 0 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 10 monclient(hunting): get_auth_request con 0x7f22bc13e380 auth_method 0 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): get_auth_request con 0x7f22bc0a07f0 auth_method 0 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 msgr2=0x7f22bc13c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 msgr2=0x7f22bc13c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:24:32.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 0x7f22bc13c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22c0e83700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 0x7f22bc13c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient(hunting): handle_auth_done global_id 22197 payload 824 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 msgr2=0x7f22bc13c5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22bc13e380 0x7f22bc13c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22bc0a0010 msgr2=0x7f22bc1cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22bc0a0010 0x7f22bc1cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 monclient: found mon.noname-g 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bb7fe700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b4009a70 con 0x7f22bc0a07f0 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.287+0000 7f22bbfff700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22bc0a0010 0x7f22bc1cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22bb7fe700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc0a07f0 0x7f22bc1ccd10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f22b000c500 tx=0x7f22b000c810 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 client.?.objecter ms_handle_connect 0x7f22bc0a07f0 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22b00070d0 con 0x7f22bc0a07f0 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: dump: 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 5 monclient: authenticate success, global_id 22197 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b0007250 con 0x7f22bc0a07f0 2024-04-20T02:24:32.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22b0010300 con 0x7f22bc0a07f0 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 10 monclient: _renew_subs 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc123f30 con 0x7f22bc0a07f0 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22ba7fc700 4 set_mon_vals no callback set 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 10 client.22197.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22b8ff9700 10 monclient: dump: 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.309 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 10 monclient: _renew_subs 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc121b60 con 0x7f22bc0a07f0 2024-04-20T02:24:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.288+0000 7f22c28e6700 1 librados: init done 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22c28e6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22c28e6700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 0x7f22bc13cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22c28e6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc068280 con 0x7f22bc06aac0 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f22b0005b70 con 0x7f22bc0a07f0 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 0x7f22bc13cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient: get_auth_request con 0x7f22bc06aac0 auth_method 0 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22a803a2f0 0x7f22a803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f22b00496d0 con 0x7f22bc0a07f0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 10 client.22197.objecter ms_dispatch 0x7f22bc001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 10 monclient(hunting): handle_auth_done global_id 35604 payload 824 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 3 client.22197.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 3 client.22197.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22bc09f070 con 0x7f22bc06aac0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 20 client.22197.objecter dump_active .. 0 homeless 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22c0e83700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22a803a2f0 0x7f22a803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 10 client.22197.objecter ms_handle_connect 0x7f22bc06aac0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 10 client.22197.objecter resend_mon_ops 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22bbfff700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 0x7f22bc13cb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00fae0 tx=0x7f22ac00fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22c0e83700 10 monclient: get_auth_request con 0x7f22a803a2f0 auth_method 0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22ac0052e0 con 0x7f22bc06aac0 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.289+0000 7f22b8ff9700 10 monclient: dump: 2024-04-20T02:24:32.312 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.313 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 10 client.22197.objecter ms_handle_connect 0x7f22a803a2f0 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22c0e83700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22a803a2f0 0x7f22a803c7b0 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7f22b400b370 tx=0x7f22b4011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f22ac005440 con 0x7f22bc06aac0 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 msgr2=0x7f22bc13cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 0x7f22bc13cb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00fae0 tx=0x7f22ac00fea0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 msgr2=0x7f22bc13cb30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.290+0000 7f22b8ff9700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc06aac0 0x7f22bc13cb30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22c28e6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22c28e6700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 0x7f22bc13d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22c28e6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0756c0 con 0x7f22bc124350 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 0x7f22bc13d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient: get_auth_request con 0x7f22bc124350 auth_method 0 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 10 monclient(hunting): handle_auth_done global_id 35613 payload 824 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 1 -- 172.21.15.47:0/3624175256 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22bc09f350 con 0x7f22bc124350 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22b8ff9700 10 client.22197.objecter ms_handle_connect 0x7f22bc124350 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22b8ff9700 10 client.22197.objecter resend_mon_ops 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22bbfff700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 0x7f22bc13d070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22ac017000 tx=0x7f22ac001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22ac023070 con 0x7f22bc124350 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22b8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22b8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.298+0000 7f22b8ff9700 10 monclient: dump: 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.318 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7f22ac00de70 con 0x7f22bc124350 2024-04-20T02:24:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 msgr2=0x7f22bc13d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 0x7f22bc13d070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22ac017000 tx=0x7f22ac001150 comp rx=0 tx=0).stop 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 msgr2=0x7f22bc13d070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22b8ff9700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22bc124350 0x7f22bc13d070 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22ac017000 tx=0x7f22ac001150 comp rx=0 tx=0).stop 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:32.322 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:32.323 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.324 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:32.325 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.326 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:32.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.328 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.330 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:32.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.336 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.340 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.341 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.342 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:32.343 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 10 librados: watch_flush enter 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 10 librados: watch_flush exit 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 20 client.22197.objecter shutdown clearing up homeless session... 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 10 client.22197.objecter successfully canceled tick 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22a803a2f0 msgr2=0x7f22a803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22a803a2f0 0x7f22a803c7b0 secure :-1 s=READY pgs=3212 cs=0 l=1 rev1=1 crypto rx=0x7f22b400b370 tx=0x7f22b4011000 comp rx=0 tx=0).stop 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 10 monclient: shutdown 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc0a07f0 msgr2=0x7f22bc1ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 --2- 172.21.15.47:0/3624175256 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22bc0a07f0 0x7f22bc1ccd10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f22b000c500 tx=0x7f22b000c810 comp rx=0 tx=0).stop 2024-04-20T02:24:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c0e83700 1 -- 172.21.15.47:0/3624175256 reap_dead start 2024-04-20T02:24:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c0e83700 1 -- 172.21.15.47:0/3624175256 reap_dead start 2024-04-20T02:24:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 shutdown_connections 2024-04-20T02:24:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 >> 172.21.15.47:0/3624175256 conn(0x7f22bc0a3f90 msgr2=0x7f22bc0717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 shutdown_connections 2024-04-20T02:24:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 -- 172.21.15.47:0/3624175256 wait complete. 2024-04-20T02:24:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.299+0000 7f22c28e6700 1 librados: shutdown 2024-04-20T02:24:32.346 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:32.346 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:24:32.347 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:24:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.485+0000 7f826e43e700 10 monclient: get_monmap_and_config 2024-04-20T02:24:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.485+0000 7f826e43e700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.485+0000 7f826e43e700 10 monclient: monmap: 2024-04-20T02:24:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.485 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:32.486416+0000 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:32.486416+0000 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:32.486 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:32.487 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 Processor -- start 2024-04-20T02:24:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 -- start start 2024-04-20T02:24:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient: init 2024-04-20T02:24:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient: _add_conns ranks=[9,8,6,3,2,10,4,5,0,7,1] 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f826809fa80 0x7f8268135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): picked mon.noname-a con 0x7f826809fa80 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 0x7f82680a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): picked mon.noname-e con 0x7f826809fe60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f82680a1f10 0x7f82680a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): picked mon.noname-d con 0x7f82680a1f10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82680a0780 con 0x7f826809fa80 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f826809df90 con 0x7f82680a1f10 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f826809dcb0 con 0x7f826809fe60 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826e43e700 10 monclient(hunting): authenticate will time out at 2151.887738s 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 0x7f82680a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826c9db700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f82680a1f10 0x7f82680a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f8267fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f826809fa80 0x7f8268135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 0x7f82680a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:37740/0 (socket says 172.21.15.47:37740) 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 1 -- 172.21.15.47:0/2970328618 learned_addr learned my addr 172.21.15.47:0/2970328618 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 10 monclient(hunting): get_auth_request con 0x7f826809fe60 auth_method 0 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f82677fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f8267fff700 10 monclient(hunting): get_auth_request con 0x7f826809fa80 auth_method 0 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f8267fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f8267fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f8267fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826c9db700 10 monclient(hunting): get_auth_request con 0x7f82680a1f10 auth_method 0 2024-04-20T02:24:32.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826c9db700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826c9db700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.486+0000 7f826c9db700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f82677fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f82677fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f82677fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 10 monclient(hunting): handle_auth_done global_id 35622 payload 274 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 -- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 msgr2=0x7f82680a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 0x7f82680a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 -- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f82680a1f10 msgr2=0x7f82680a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f82680a1f10 0x7f82680a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 monclient: found mon.noname-a 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 -- 172.21.15.47:0/2970328618 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680751e0 con 0x7f826809fa80 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f82677fe700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 0x7f82680a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826c9db700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f82680a1f10 0x7f82680a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8267fff700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268135df0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8258009840 tx=0x7f8258009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a3ef6292e4e978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 1 -- 172.21.15.47:0/2970328618 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8258011aa0 con 0x7f826809fa80 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: dump: 2024-04-20T02:24:32.501 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.502 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826e43e700 5 monclient: authenticate success, global_id 35622 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 1 -- 172.21.15.47:0/2970328618 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258011c00 con 0x7f826809fa80 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826e43e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826e43e700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f826e43e700 4 set_mon_vals no callback set 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 1 -- 172.21.15.47:0/2970328618 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8258011d70 con 0x7f826809fa80 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.487+0000 7f8266ffd700 10 monclient: dump: 2024-04-20T02:24:32.503 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.504 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 10 monclient: shutdown 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 -- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 msgr2=0x7f8268135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268135df0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8258009840 tx=0x7f8258009b50 comp rx=0 tx=0).stop 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 -- 172.21.15.47:0/2970328618 shutdown_connections 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f82680a1f10 0x7f82680a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268135df0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 --2- 172.21.15.47:0/2970328618 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f826809fe60 0x7f82680a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 -- 172.21.15.47:0/2970328618 >> 172.21.15.47:0/2970328618 conn(0x7f82680a3000 msgr2=0x7f82680a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 -- 172.21.15.47:0/2970328618 shutdown_connections 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.488+0000 7f826e43e700 1 -- 172.21.15.47:0/2970328618 wait complete. 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 10 monclient: monmap: 2024-04-20T02:24:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.506 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 1 librados: starting msgr at 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 1 librados: starting objecter 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 1 Processor -- start 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 1 -- start start 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 1 librados: setting wanted keys 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 1 librados: calling monclient init 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.489+0000 7f826e43e700 10 monclient: init 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient: _add_conns ranks=[0,4,2,7,5,1,6,8,3] 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): picked mon.noname-a con 0x7f826809fa80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f826809fe60 0x7f826809a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): picked mon.noname-c con 0x7f826809fe60 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:32.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f82680a1f10 0x7f826809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): picked mon.noname-b con 0x7f82680a1f10 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82681305a0 con 0x7f826809fa80 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826809b7e0 con 0x7f82680a1f10 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f826809bd20 con 0x7f826809fe60 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826e43e700 10 monclient(hunting): authenticate will time out at 2151.891389s 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f826809fe60 0x7f826809a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58816/0 (socket says 172.21.15.47:58816) 2024-04-20T02:24:32.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 -- 172.21.15.47:0/2495809379 learned_addr learned my addr 172.21.15.47:0/2495809379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f82680a1f10 0x7f826809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): get_auth_request con 0x7f826809fa80 auth_method 0 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): get_auth_request con 0x7f826809fe60 auth_method 0 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): get_auth_request con 0x7f82680a1f10 auth_method 0 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f82677fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient(hunting): handle_auth_done global_id 35631 payload 824 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f826809fe60 msgr2=0x7f826809a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f826809fe60 0x7f826809a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f82680a1f10 msgr2=0x7f826809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f82680a1f10 0x7f826809aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 monclient: found mon.noname-a 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.490+0000 7f8267fff700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c009550 con 0x7f826809fa80 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8267fff700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268072810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 client.?.objecter ms_handle_connect 0x7f826809fa80 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f82580036a0 con 0x7f826809fa80 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: dump: 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.511 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825801f070 con 0x7f826809fa80 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8258003c70 con 0x7f826809fa80 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 5 monclient: authenticate success, global_id 35631 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f82667fc700 4 set_mon_vals no callback set 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f8264ff9700 10 monclient: dump: 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.513 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 10 monclient: _renew_subs 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260009a70 con 0x7f826809fa80 2024-04-20T02:24:32.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 10 client.35631.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 10 monclient: _renew_subs 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82580094a0 con 0x7f826809fa80 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.491+0000 7f826e43e700 1 librados: init done 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8258021030 con 0x7f826809fa80 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826e43e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f825403a2f0 0x7f825403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826e43e700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 0x7f826809af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8268068180 con 0x7f826809c930 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f825804b8e0 con 0x7f826809fa80 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 10 client.35631.objecter ms_dispatch 0x7f8268001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 3 client.35631.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 3 client.35631.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:32.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 0x7f826809af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f8264ff9700 20 client.35631.objecter dump_active .. 0 homeless 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient: get_auth_request con 0x7f826809c930 auth_method 0 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f82677fe700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f825403a2f0 0x7f825403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f82677fe700 10 monclient: get_auth_request con 0x7f825403a2f0 auth_method 0 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 10 monclient(hunting): handle_auth_done global_id 24050 payload 824 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.492+0000 7f826c9db700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f826809e550 con 0x7f826809c930 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f826c9db700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 0x7f826809af80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f20 tx=0x7f826000f490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 client.35631.objecter ms_handle_connect 0x7f826809c930 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 client.35631.objecter resend_mon_ops 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f826000e040 con 0x7f826809c930 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 monclient: dump: 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.517 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 client.35631.objecter ms_handle_connect 0x7f825403a2f0 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f82677fe700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f825403a2f0 0x7f825403c7b0 secure :-1 s=READY pgs=3213 cs=0 l=1 rev1=1 crypto rx=0x7f8268072e60 tx=0x7f825c0092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f826000f040 con 0x7f826809c930 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 msgr2=0x7f826809af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 0x7f826809af80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f20 tx=0x7f826000f490 comp rx=0 tx=0).stop 2024-04-20T02:24:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 msgr2=0x7f826809af80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.493+0000 7f8264ff9700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826809c930 0x7f826809af80 secure :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f20 tx=0x7f826000f490 comp rx=0 tx=0).stop 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826e43e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826e43e700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 0x7f8268130a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826e43e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826806a400 con 0x7f826806aac0 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 0x7f8268130a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient: get_auth_request con 0x7f826806aac0 auth_method 0 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.501+0000 7f826c9db700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f826c9db700 10 monclient(hunting): handle_auth_done global_id 24059 payload 824 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f826c9db700 1 -- 172.21.15.47:0/2495809379 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f82680668e0 con 0x7f826806aac0 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f826c9db700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 0x7f8268130a50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f20 tx=0x7f8260004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 client.35631.objecter ms_handle_connect 0x7f826806aac0 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 client.35631.objecter resend_mon_ops 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f82600229f0 con 0x7f826806aac0 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 monclient: dump: 2024-04-20T02:24:32.522 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.523 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f8260022580 con 0x7f826806aac0 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 msgr2=0x7f8268130a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 0x7f8268130a50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f20 tx=0x7f8260004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 msgr2=0x7f8268130a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f8264ff9700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f826806aac0 0x7f8268130a50 secure :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f20 tx=0x7f8260004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:32.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.525 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.526 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.530 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.530 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.530 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:32.530 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.531 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:32.532 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:32.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.534 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.535 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.536 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.538 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.539 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.541 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:32.544 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.545 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.546 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:32.547 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.548 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.549 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.550 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:32.550 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f826e43e700 10 librados: watch_flush enter 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.502+0000 7f826e43e700 10 librados: watch_flush exit 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 20 client.35631.objecter shutdown clearing up homeless session... 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 10 client.35631.objecter successfully canceled tick 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f825403a2f0 msgr2=0x7f825403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f825403a2f0 0x7f825403c7b0 secure :-1 s=READY pgs=3213 cs=0 l=1 rev1=1 crypto rx=0x7f8268072e60 tx=0x7f825c0092c0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 10 monclient: shutdown 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 msgr2=0x7f8268072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268072810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8258000c00 tx=0x7f8258004750 comp rx=0 tx=0).stop 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826c9db700 1 -- 172.21.15.47:0/2495809379 reap_dead start 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 shutdown_connections 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 --2- 172.21.15.47:0/2495809379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f826809fa80 0x7f8268072810 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 >> 172.21.15.47:0/2495809379 conn(0x7f82680a3000 msgr2=0x7f82681bd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 shutdown_connections 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 -- 172.21.15.47:0/2495809379 wait complete. 2024-04-20T02:24:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.503+0000 7f826e43e700 1 librados: shutdown 2024-04-20T02:24:32.553 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:32.553 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:24:32.554 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:24:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.692+0000 7f1e173f6700 10 monclient: get_monmap_and_config 2024-04-20T02:24:32.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.692+0000 7f1e173f6700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.692+0000 7f1e173f6700 10 monclient: monmap: 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:32.693778+0000 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:32.693778+0000 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.693 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.694 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.694 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.694 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.694 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:32.694 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:32.694 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 1 Processor -- start 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 1 -- start start 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 10 monclient: init 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 10 monclient: _add_conns ranks=[6,10,9,8,7,4,2,1,5,3,0] 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 0x7f1e1013c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 10 monclient(hunting): picked mon.noname-d con 0x7f1e1013c350 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1e100710c0 0x7f1e1006c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.693+0000 7f1e173f6700 10 monclient(hunting): picked mon.noname-f con 0x7f1e100710c0 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 0x7f1e1006cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 10 monclient(hunting): picked mon.noname-a con 0x7f1e100714e0 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100a4090 con 0x7f1e100714e0 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1006e610 con 0x7f1e100710c0 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100a3c00 con 0x7f1e1013c350 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 10 monclient(hunting): authenticate will time out at 2152.095032s 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 0x7f1e1006cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 0x7f1e1013c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 0x7f1e1006cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58828/0 (socket says 172.21.15.47:58828) 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 0x7f1e1013c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45050/0 (socket says 172.21.15.47:45050) 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 1 -- 172.21.15.47:0/2081113369 learned_addr learned my addr 172.21.15.47:0/2081113369 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): get_auth_request con 0x7f1e100714e0 auth_method 0 2024-04-20T02:24:32.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): get_auth_request con 0x7f1e1013c350 auth_method 0 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e14991700 1 --2- 172.21.15.47:0/2081113369 >> v2:172.21.15.179:3300/0 conn(0x7f1e100710c0 0x7f1e1006c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e14991700 10 monclient(hunting): get_auth_request con 0x7f1e100710c0 auth_method 0 2024-04-20T02:24:32.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e14991700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e14991700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e14991700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient(hunting): handle_auth_done global_id 54375 payload 274 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 -- 172.21.15.47:0/2081113369 >> v2:172.21.15.179:3300/0 conn(0x7f1e100710c0 msgr2=0x7f1e1006c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 --2- 172.21.15.47:0/2081113369 >> v2:172.21.15.179:3300/0 conn(0x7f1e100710c0 0x7f1e1006c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 -- 172.21.15.47:0/2081113369 >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 msgr2=0x7f1e1006cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 --2- 172.21.15.47:0/2081113369 >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 0x7f1e1006cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 monclient: found mon.noname-d 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 -- 172.21.15.47:0/2081113369 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100751e0 con 0x7f1e1013c350 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15993700 1 --2- 172.21.15.47:0/2081113369 >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 0x7f1e1006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e15192700 1 --2- 172.21.15.47:0/2081113369 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 0x7f1e1013c730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e100676d0 tx=0x7f1e0c009b20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7a73ad70173279df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 1 -- 172.21.15.47:0/2081113369 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e0c011aa0 con 0x7f1e1013c350 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 10 monclient: dump: 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e07fff700 1 -- 172.21.15.47:0/2081113369 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c011c00 con 0x7f1e1013c350 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.694+0000 7f1e173f6700 5 monclient: authenticate success, global_id 54375 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:32.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e07fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e07fff700 1 -- 172.21.15.47:0/2081113369 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e0c011d70 con 0x7f1e1013c350 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e07fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e07fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e07fff700 10 monclient: dump: 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.712 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 4 set_mon_vals no callback set 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 10 monclient: shutdown 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 -- 172.21.15.47:0/2081113369 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 msgr2=0x7f1e1013c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 --2- 172.21.15.47:0/2081113369 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 0x7f1e1013c730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e100676d0 tx=0x7f1e0c009b20 comp rx=0 tx=0).stop 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 -- 172.21.15.47:0/2081113369 shutdown_connections 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 --2- 172.21.15.47:0/2081113369 >> v2:172.21.15.47:3300/0 conn(0x7f1e100714e0 0x7f1e1006cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 --2- 172.21.15.47:0/2081113369 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1e1013c350 0x7f1e1013c730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 --2- 172.21.15.47:0/2081113369 >> v2:172.21.15.179:3300/0 conn(0x7f1e100710c0 0x7f1e1006c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 -- 172.21.15.47:0/2081113369 >> 172.21.15.47:0/2081113369 conn(0x7f1e100a4a50 msgr2=0x7f1e100a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 -- 172.21.15.47:0/2081113369 shutdown_connections 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.695+0000 7f1e173f6700 1 -- 172.21.15.47:0/2081113369 wait complete. 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 10 monclient: monmap: 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 1 librados: starting msgr at 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 1 librados: starting objecter 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 1 Processor -- start 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 1 -- start start 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 1 librados: setting wanted keys 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 1 librados: calling monclient init 2024-04-20T02:24:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.696+0000 7f1e173f6700 10 monclient: init 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient: _add_conns ranks=[2,8,4,5,6,0,1,7,3] 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 0x7f1e101d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): picked mon.noname-b con 0x7f1e100710c0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1e100714e0 0x7f1e1013b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): picked mon.noname-e con 0x7f1e100714e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013c350 0x7f1e1009c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): picked mon.noname-c con 0x7f1e1013c350 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10127220 con 0x7f1e100710c0 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1009ed30 con 0x7f1e1013c350 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1009e3e0 con 0x7f1e100714e0 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e173f6700 10 monclient(hunting): authenticate will time out at 2152.098346s 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 0x7f1e101d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1e100714e0 0x7f1e1013b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013c350 0x7f1e1009c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 0x7f1e101d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:57190/0 (socket says 172.21.15.47:57190) 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 -- 172.21.15.47:0/3094137630 learned_addr learned my addr 172.21.15.47:0/3094137630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): get_auth_request con 0x7f1e100710c0 auth_method 0 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): get_auth_request con 0x7f1e100714e0 auth_method 0 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): get_auth_request con 0x7f1e1013c350 auth_method 0 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient(hunting): handle_auth_done global_id 24068 payload 824 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1e100714e0 msgr2=0x7f1e1013b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1e100714e0 0x7f1e1013b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013c350 msgr2=0x7f1e1009c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013c350 0x7f1e1009c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 monclient: found mon.noname-b 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:32.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15192700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08009b30 con 0x7f1e100710c0 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e14991700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1e100714e0 0x7f1e1013b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.697+0000 7f1e15993700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013c350 0x7f1e1009c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e15192700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 0x7f1e101d05d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 client.?.objecter ms_handle_connect 0x7f1e100710c0 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e0c003b10 con 0x7f1e100710c0 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: dump: 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.720 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 5 monclient: authenticate success, global_id 24068 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 10 monclient: _renew_subs 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c0094a0 con 0x7f1e100710c0 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c01e070 con 0x7f1e100710c0 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e0c003c70 con 0x7f1e100710c0 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 10 client.24068.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e077fe700 4 set_mon_vals no callback set 2024-04-20T02:24:32.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e05ffb700 10 monclient: dump: 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.723 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 10 monclient: _renew_subs 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e100a3d70 con 0x7f1e100710c0 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.698+0000 7f1e173f6700 1 librados: init done 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e173f6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e173f6700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 0x7f1e1009cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10068430 con 0x7f1e1013b770 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1e0c0036a0 con 0x7f1e100710c0 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 0x7f1e1009cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient: get_auth_request con 0x7f1e1013b770 auth_method 0 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1df403a2f0 0x7f1df403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 10 monclient(hunting): handle_auth_done global_id 54400 payload 824 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1e0c020070 con 0x7f1e100710c0 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 10 client.24068.objecter ms_dispatch 0x7f1e10001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 3 client.24068.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 3 client.24068.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e14991700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e100a43b0 con 0x7f1e1013b770 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.699+0000 7f1e05ffb700 20 client.24068.objecter dump_active .. 0 homeless 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 client.24068.objecter ms_handle_connect 0x7f1e1013b770 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 client.24068.objecter resend_mon_ops 2024-04-20T02:24:32.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e14991700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 0x7f1e1009cd80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e1013cb60 tx=0x7f1e00011e10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e0000c070 con 0x7f1e1013b770 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 monclient: dump: 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.726 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e15993700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1df403a2f0 0x7f1df403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e15993700 10 monclient: get_auth_request con 0x7f1df403a2f0 auth_method 0 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1e0000fa50 con 0x7f1e1013b770 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 msgr2=0x7f1e1009cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 0x7f1e1009cd80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e1013cb60 tx=0x7f1e00011e10 comp rx=0 tx=0).stop 2024-04-20T02:24:32.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 msgr2=0x7f1e1009cd80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1013b770 0x7f1e1009cd80 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e1013cb60 tx=0x7f1e00011e10 comp rx=0 tx=0).stop 2024-04-20T02:24:32.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e05ffb700 10 client.24068.objecter ms_handle_connect 0x7f1df403a2f0 2024-04-20T02:24:32.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.700+0000 7f1e15993700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1df403a2f0 0x7f1df403c7b0 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7f1e080072e0 tx=0x7f1e0800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e173f6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e173f6700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 0x7f1e1009d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e173f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1006a400 con 0x7f1e1006aac0 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 0x7f1e1009d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient: get_auth_request con 0x7f1e1006aac0 auth_method 0 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 10 monclient(hunting): handle_auth_done global_id 54409 payload 824 2024-04-20T02:24:32.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 1 -- 172.21.15.47:0/3094137630 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1e10128870 con 0x7f1e1006aac0 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e05ffb700 10 client.24068.objecter ms_handle_connect 0x7f1e1006aac0 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e14991700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 0x7f1e1009d2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e1013cb60 tx=0x7f1e00004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.708+0000 7f1e05ffb700 10 client.24068.objecter resend_mon_ops 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e0001e8c0 con 0x7f1e1006aac0 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 10 monclient: dump: 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.731 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f1e00024040 con 0x7f1e1006aac0 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 msgr2=0x7f1e1009d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 0x7f1e1009d2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e1013cb60 tx=0x7f1e00004750 comp rx=0 tx=0).stop 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 msgr2=0x7f1e1009d2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e05ffb700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e1006aac0 0x7f1e1009d2c0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e1013cb60 tx=0x7f1e00004750 comp rx=0 tx=0).stop 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:32.733 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.734 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.735 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.738 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:32.739 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.740 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:32.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.743 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.745 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:32.747 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.750 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.751 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.752 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.753 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:32.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.757 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:32.757 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 10 librados: watch_flush enter 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 10 librados: watch_flush exit 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 20 client.24068.objecter shutdown clearing up homeless session... 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 10 client.24068.objecter successfully canceled tick 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1df403a2f0 msgr2=0x7f1df403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1df403a2f0 0x7f1df403c7b0 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7f1e080072e0 tx=0x7f1e0800b040 comp rx=0 tx=0).stop 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 10 monclient: shutdown 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 msgr2=0x7f1e101d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 0x7f1e101d05d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c004750 comp rx=0 tx=0).stop 2024-04-20T02:24:32.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e15993700 1 -- 172.21.15.47:0/3094137630 reap_dead start 2024-04-20T02:24:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 shutdown_connections 2024-04-20T02:24:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 --2- 172.21.15.47:0/3094137630 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e100710c0 0x7f1e101d05d0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.709+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 >> 172.21.15.47:0/3094137630 conn(0x7f1e100a4a50 msgr2=0x7f1e1006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.710+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 shutdown_connections 2024-04-20T02:24:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.710+0000 7f1e173f6700 1 -- 172.21.15.47:0/3094137630 wait complete. 2024-04-20T02:24:32.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.710+0000 7f1e173f6700 1 librados: shutdown 2024-04-20T02:24:32.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:32.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:24:32.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.899+0000 7fc81f375700 10 monclient: get_monmap_and_config 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.899+0000 7fc81f375700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.899+0000 7fc81f375700 10 monclient: monmap: 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:32.900377+0000 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:32.900377+0000 2024-04-20T02:24:32.899 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:32.900 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.899+0000 7fc81f375700 1 Processor -- start 2024-04-20T02:24:32.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 1 -- start start 2024-04-20T02:24:32.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient: init 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient: _add_conns ranks=[0,9,10,3,8,6,7,4,5,2,1] 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc81809c2d0 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc818134c40 0x7fc81809cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): picked mon.noname-a con 0x7fc818134c40 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc818139aa0 0x7fc81813be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): picked mon.noname-f con 0x7fc818139aa0 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180751e0 con 0x7fc81809c2d0 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8180a0510 con 0x7fc818134c40 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81809d300 con 0x7fc818139aa0 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81f375700 10 monclient(hunting): authenticate will time out at 2152.301621s 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc818134c40 0x7fc81809cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d111700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc81809c2d0 0x7fc81809c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54350/0 (socket says 172.21.15.47:54350) 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d111700 1 -- 172.21.15.47:0/389220097 learned_addr learned my addr 172.21.15.47:0/389220097 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): get_auth_request con 0x7fc818134c40 auth_method 0 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d912700 1 --2- 172.21.15.47:0/389220097 >> v2:172.21.15.179:3300/0 conn(0x7fc818139aa0 0x7fc81813be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81c910700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d912700 10 monclient(hunting): get_auth_request con 0x7fc818139aa0 auth_method 0 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d912700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d912700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.900+0000 7fc81d912700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81d912700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81d912700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81d912700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 10 monclient(hunting): handle_auth_done global_id 35649 payload 274 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 1 -- 172.21.15.47:0/389220097 >> v2:172.21.15.179:3300/0 conn(0x7fc818139aa0 msgr2=0x7fc81813be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 1 --2- 172.21.15.47:0/389220097 >> v2:172.21.15.179:3300/0 conn(0x7fc818139aa0 0x7fc81813be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 1 -- 172.21.15.47:0/389220097 >> v1:172.21.15.47:6789/0 conn(0x7fc81809c2d0 legacy=0x7fc81809c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 1 monclient: found mon.noname-a 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 1 -- 172.21.15.47:0/389220097 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81806a620 con 0x7fc818134c40 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 1 -- 172.21.15.47:0/389220097 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580982342 0 0) 0x7fc8180751e0 con 0x7fc81809c2d0 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81c910700 1 --2- 172.21.15.47:0/389220097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc818134c40 0x7fc81809cdc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc81806dff0 tx=0x7fc808009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3404218b26869a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 1 -- 172.21.15.47:0/389220097 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc808011aa0 con 0x7fc818134c40 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: dump: 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.915 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81f375700 5 monclient: authenticate success, global_id 35649 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81f375700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 1 -- 172.21.15.47:0/389220097 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808011c00 con 0x7fc818134c40 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81f375700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc81f375700 4 set_mon_vals no callback set 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 1 -- 172.21.15.47:0/389220097 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc808011d70 con 0x7fc818134c40 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:32.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.901+0000 7fc80ffff700 10 monclient: dump: 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.917 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 10 monclient: shutdown 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 -- 172.21.15.47:0/389220097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc818134c40 msgr2=0x7fc81809cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 --2- 172.21.15.47:0/389220097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc818134c40 0x7fc81809cdc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc81806dff0 tx=0x7fc808009b20 comp rx=0 tx=0).stop 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 -- 172.21.15.47:0/389220097 shutdown_connections 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 --2- 172.21.15.47:0/389220097 >> v2:172.21.15.179:3300/0 conn(0x7fc818139aa0 0x7fc81813be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 --2- 172.21.15.47:0/389220097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc818134c40 0x7fc81809cdc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 -- 172.21.15.47:0/389220097 >> 172.21.15.47:0/389220097 conn(0x7fc8180a2db0 msgr2=0x7fc8180a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 -- 172.21.15.47:0/389220097 shutdown_connections 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 1 -- 172.21.15.47:0/389220097 wait complete. 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 10 monclient: build_initial_monmap 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.902+0000 7fc81f375700 10 monclient: monmap: 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:32.918 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 librados: starting msgr at 2024-04-20T02:24:32.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 librados: starting objecter 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 Processor -- start 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 -- start start 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 librados: setting wanted keys 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 librados: calling monclient init 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient: init 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient: _add_conns ranks=[1,3,4,2,6,5,8,7,0] 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81809c2d0 0x7fc818071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): picked mon.noname-f con 0x7fc81809c2d0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc818134c40 0x7fc818071760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): picked mon.noname-g con 0x7fc818134c40 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc818139aa0 0x7fc818071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): picked mon.noname-c con 0x7fc818139aa0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818133820 con 0x7fc81809c2d0 2024-04-20T02:24:32.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818131340 con 0x7fc818134c40 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181315b0 con 0x7fc818139aa0 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81f375700 10 monclient(hunting): authenticate will time out at 2152.304872s 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.903+0000 7fc81d912700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc818139aa0 0x7fc818071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc818139aa0 0x7fc818071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:51672/0 (socket says 172.21.15.47:51672) 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 -- 172.21.15.47:0/43895595 learned_addr learned my addr 172.21.15.47:0/43895595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): get_auth_request con 0x7fc818139aa0 auth_method 0 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81809c2d0 0x7fc818071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): get_auth_request con 0x7fc81809c2d0 auth_method 0 2024-04-20T02:24:32.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc818134c40 0x7fc818071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): get_auth_request con 0x7fc818134c40 auth_method 0 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81c910700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient(hunting): handle_auth_done global_id 54418 payload 824 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc818134c40 msgr2=0x7fc818071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc818134c40 0x7fc818071760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81809c2d0 msgr2=0x7fc818071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81809c2d0 0x7fc818071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient: _finish_hunting 0 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 monclient: found mon.noname-c 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080094a0 con 0x7fc818139aa0 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81d912700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc818139aa0 0x7fc818071ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc810009d30 tx=0x7fc81000c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 client.?.objecter ms_handle_connect 0x7fc818139aa0 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:32.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc810009e80 con 0x7fc818139aa0 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: dump: 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.924 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81000f040 con 0x7fc818139aa0 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 5 monclient: authenticate success, global_id 54418 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 10 monclient: _renew_subs 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81809f2b0 con 0x7fc818139aa0 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8100148f0 con 0x7fc818139aa0 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:32.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80dffb700 10 monclient: dump: 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.926 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 10 client.54418.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc80f7fe700 4 set_mon_vals no callback set 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 10 monclient: _renew_subs 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818131890 con 0x7fc818139aa0 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.904+0000 7fc81f375700 1 librados: init done 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81f375700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81f375700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 0x7fc818137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc818067740 con 0x7fc81806aac0 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc810014480 con 0x7fc818139aa0 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 0x7fc818137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient: get_auth_request con 0x7fc81806aac0 auth_method 0 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.905+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc7fc03a2f0 0x7fc7fc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc81004ae10 con 0x7fc818139aa0 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc81d111700 10 monclient(hunting): handle_auth_done global_id 54384 payload 824 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 client.54418.objecter ms_dispatch 0x7fc818001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 3 client.54418.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 3 client.54418.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc81d111700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc81809b490 con 0x7fc81806aac0 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 20 client.54418.objecter dump_active .. 0 homeless 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc81c910700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc7fc03a2f0 0x7fc7fc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 client.54418.objecter ms_handle_connect 0x7fc81806aac0 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc81d111700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 0x7fc818137880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc814000f60 tx=0x7fc8140018f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 client.54418.objecter resend_mon_ops 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc81c910700 10 monclient: get_auth_request con 0x7fc7fc03a2f0 auth_method 0 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc814017070 con 0x7fc81806aac0 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 monclient: dump: 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.929 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 client.54418.objecter ms_handle_connect 0x7fc7fc03a2f0 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc81c910700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc7fc03a2f0 0x7fc7fc03c7b0 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fc808009b60 tx=0x7fc808008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc814012560 con 0x7fc81806aac0 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 msgr2=0x7fc818137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 0x7fc818137880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc814000f60 tx=0x7fc8140018f0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 msgr2=0x7fc818137880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.906+0000 7fc80dffb700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc81806aac0 0x7fc818137880 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81f375700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81f375700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 0x7fc8181381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81f375700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806a400 con 0x7fc818137dc0 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 0x7fc8181381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient: get_auth_request con 0x7fc818137dc0 auth_method 0 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.914+0000 7fc81d111700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc81d111700 10 monclient(hunting): handle_auth_done global_id 54393 payload 824 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc81d111700 1 -- 172.21.15.47:0/43895595 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc8180a0130 con 0x7fc818137dc0 2024-04-20T02:24:32.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc81d111700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 0x7fc8181381d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc814015040 tx=0x7fc814000ed0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 client.54418.objecter ms_handle_connect 0x7fc818137dc0 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 client.54418.objecter resend_mon_ops 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc814023070 con 0x7fc818137dc0 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 monclient: dump: 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:32.934 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fc814012e70 con 0x7fc818137dc0 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:32.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 msgr2=0x7fc8181381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 0x7fc8181381d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc814015040 tx=0x7fc814000ed0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 msgr2=0x7fc8181381d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.915+0000 7fc80dffb700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc818137dc0 0x7fc8181381d0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc814015040 tx=0x7fc814000ed0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:32.939 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.940 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:32.942 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:32.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.945 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:32.946 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.948 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.949 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.952 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:32.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:32.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.958 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:32.959 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 10 librados: watch_flush enter 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 10 librados: watch_flush exit 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 20 client.54418.objecter shutdown clearing up homeless session... 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 10 client.54418.objecter successfully canceled tick 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc7fc03a2f0 msgr2=0x7fc7fc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc7fc03a2f0 0x7fc7fc03c7b0 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fc808009b60 tx=0x7fc808008030 comp rx=0 tx=0).stop 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 10 monclient: shutdown 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc818139aa0 msgr2=0x7fc818071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:32.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 --2- 172.21.15.47:0/43895595 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc818139aa0 0x7fc818071ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc810009d30 tx=0x7fc81000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81d912700 1 -- 172.21.15.47:0/43895595 reap_dead start 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81d912700 1 -- 172.21.15.47:0/43895595 reap_dead start 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 shutdown_connections 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 >> 172.21.15.47:0/43895595 conn(0x7fc8180a2db0 msgr2=0x7fc8180a4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 shutdown_connections 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 -- 172.21.15.47:0/43895595 wait complete. 2024-04-20T02:24:32.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:32.916+0000 7fc81f375700 1 librados: shutdown 2024-04-20T02:24:32.962 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:32.962 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:24:32.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.100+0000 7f6f11065700 10 monclient: get_monmap_and_config 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.100+0000 7f6f11065700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.100+0000 7f6f11065700 10 monclient: monmap: 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:33.101941+0000 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:33.101941+0000 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:33.101 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:33.102 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.101+0000 7f6f11065700 1 Processor -- start 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.101+0000 7f6f11065700 1 -- start start 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.101+0000 7f6f11065700 10 monclient: init 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient: _add_conns ranks=[5,9,3,2,4,10,8,6,1,7,0] 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 0x7f6f0c134d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): picked mon.noname-h con 0x7f6f0c09c750 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6f0c09cb30 0x7f6f0c0a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): picked mon.noname-a con 0x7f6f0c09cb30 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cf30 0x7f6f0c0a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): picked mon.noname-g con 0x7f6f0c09cf30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c1357b0 con 0x7f6f0c09cb30 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c09fd40 con 0x7f6f0c09cf30 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0a3df0 con 0x7f6f0c09c750 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f11065700 10 monclient(hunting): authenticate will time out at 2152.503600s 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6f0c09cb30 0x7f6f0c0a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6f0c09cb30 0x7f6f0c0a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58856/0 (socket says 172.21.15.47:58856) 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 1 -- 172.21.15.47:0/744861780 learned_addr learned my addr 172.21.15.47:0/744861780 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): get_auth_request con 0x7f6f0c09cb30 auth_method 0 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 0x7f6f0c134d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0ad9d700 10 monclient(hunting): get_auth_request con 0x7f6f0c09c750 auth_method 0 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.102+0000 7f6f0a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0ad9d700 1 -- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 msgr2=0x7f6f0c134d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0ad9d700 1 -- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 msgr2=0x7f6f0c134d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 0x7f6f0c134d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 0x7f6f0c134d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 10 monclient(hunting): handle_auth_done global_id 35667 payload 274 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 -- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 msgr2=0x7f6f0c134d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:33.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 0x7f6f0c134d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 -- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cf30 msgr2=0x7f6f0c0a2450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cf30 0x7f6f0c0a2450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 monclient: found mon.noname-a 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 -- 172.21.15.47:0/744861780 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0751e0 con 0x7f6f0c09cb30 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.103+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cb30 0x7f6f0c0a1f10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009950 tx=0x7f6f00009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20901add279ee3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 1 -- 172.21.15.47:0/744861780 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f00009ea0 con 0x7f6f0c09cb30 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: dump: 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.110 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.111 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 1 -- 172.21.15.47:0/744861780 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f000044a0 con 0x7f6f0c09cb30 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f11065700 5 monclient: authenticate success, global_id 35667 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f11065700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f11065700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:33.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 1 -- 172.21.15.47:0/744861780 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f0000bc50 con 0x7f6f0c09cb30 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f11065700 4 set_mon_vals no callback set 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.104+0000 7f6f09d9b700 10 monclient: dump: 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.113 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 10 monclient: shutdown 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 -- 172.21.15.47:0/744861780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cb30 msgr2=0x7f6f0c0a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cb30 0x7f6f0c0a1f10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009950 tx=0x7f6f00009c60 comp rx=0 tx=0).stop 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 -- 172.21.15.47:0/744861780 shutdown_connections 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cf30 0x7f6f0c0a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6f0c09c750 0x7f6f0c134d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 --2- 172.21.15.47:0/744861780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cb30 0x7f6f0c0a1f10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 -- 172.21.15.47:0/744861780 >> 172.21.15.47:0/744861780 conn(0x7f6f0c0a3000 msgr2=0x7f6f0c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 -- 172.21.15.47:0/744861780 shutdown_connections 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.105+0000 7f6f11065700 1 -- 172.21.15.47:0/744861780 wait complete. 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.106+0000 7f6f11065700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.106+0000 7f6f11065700 10 monclient: monmap: 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.115 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.106+0000 7f6f11065700 1 librados: starting msgr at 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.106+0000 7f6f11065700 1 librados: starting objecter 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.107+0000 7f6f11065700 1 Processor -- start 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.107+0000 7f6f11065700 1 -- start start 2024-04-20T02:24:33.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.107+0000 7f6f11065700 1 librados: setting wanted keys 2024-04-20T02:24:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.107+0000 7f6f11065700 1 librados: calling monclient init 2024-04-20T02:24:33.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.107+0000 7f6f11065700 10 monclient: init 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient: _add_conns ranks=[7,3,0,6,8,2,1,5,4] 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6f0c09c750 0x7f6f0c12fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): picked mon.noname-i con 0x7f6f0c09c750 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cb30 0x7f6f0c130320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): picked mon.noname-g con 0x7f6f0c09cb30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 0x7f6f0c130860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): picked mon.noname-a con 0x7f6f0c09cf30 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:33.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1312b0 con 0x7f6f0c09cf30 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1337b0 con 0x7f6f0c09cb30 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c131770 con 0x7f6f0c09c750 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f11065700 10 monclient(hunting): authenticate will time out at 2152.509177s 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 0x7f6f0c130860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 0x7f6f0c130860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58858/0 (socket says 172.21.15.47:58858) 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 -- 172.21.15.47:0/1480027370 learned_addr learned my addr 172.21.15.47:0/1480027370 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): get_auth_request con 0x7f6f0c09cf30 auth_method 0 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6f0c09c750 0x7f6f0c12fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): get_auth_request con 0x7f6f0c09c750 auth_method 0 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cb30 0x7f6f0c130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0a59c700 10 monclient(hunting): get_auth_request con 0x7f6f0c09cb30 auth_method 0 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient(hunting): handle_auth_done global_id 35676 payload 824 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6f0c09c750 msgr2=0x7f6f0c12fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6f0c09c750 0x7f6f0c12fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cb30 msgr2=0x7f6f0c130320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cb30 0x7f6f0c130320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 monclient: found mon.noname-a 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0b59e700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00009550 con 0x7f6f0c09cf30 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.108+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6f0c09cb30 0x7f6f0c130320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.109+0000 7f6f0b59e700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 0x7f6f0c130860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400b9e0 tx=0x7f6f0400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.109+0000 7f6ef37fe700 10 client.?.objecter ms_handle_connect 0x7f6f0c09cf30 2024-04-20T02:24:33.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.109+0000 7f6ef37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f0400cba0 con 0x7f6f0c09cf30 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: dump: 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.128 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400cd60 con 0x7f6f0c09cf30 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6f040127b0 con 0x7f6f0c09cf30 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6ef37fe700 10 monclient: dump: 2024-04-20T02:24:33.129 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 5 monclient: authenticate success, global_id 35676 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 10 monclient: _renew_subs 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c099e00 con 0x7f6f0c09cf30 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f0959a700 4 set_mon_vals no callback set 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 10 client.35676.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 10 monclient: _renew_subs 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c09ef50 con 0x7f6f0c09cf30 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.111+0000 7f6f11065700 1 librados: init done 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f11065700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f11065700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 0x7f6f0c1cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c068180 con 0x7f6f0c06aac0 2024-04-20T02:24:33.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6f04012340 con 0x7f6f0c09cf30 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 0x7f6f0c1cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient: get_auth_request con 0x7f6f0c06aac0 auth_method 0 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6ef803a2f0 0x7f6ef803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6f0404e170 con 0x7f6f0c09cf30 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 10 client.35676.objecter ms_dispatch 0x7f6f0c001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 3 client.35676.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 3 client.35676.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_done global_id 35252 payload 824 2024-04-20T02:24:33.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0ad9d700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f0c0a0300 con 0x7f6f0c06aac0 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6ef37fe700 20 client.35676.objecter dump_active .. 0 homeless 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6ef803a2f0 0x7f6ef803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.112+0000 7f6f0a59c700 10 monclient: get_auth_request con 0x7f6ef803a2f0 auth_method 0 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 client.35676.objecter ms_handle_connect 0x7f6f0c06aac0 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 client.35676.objecter resend_mon_ops 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 0x7f6f0c1cc9d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009820 tx=0x7f6efc009be0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6efc0149e0 con 0x7f6f0c06aac0 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 monclient: dump: 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.133 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.134 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6efc014570 con 0x7f6f0c06aac0 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 msgr2=0x7f6f0c1cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6f0a59c700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6ef803a2f0 0x7f6ef803c7b0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009950 tx=0x7f6f00008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 0x7f6f0c1cc9d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009820 tx=0x7f6efc009be0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 msgr2=0x7f6f0c1cc9d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c06aac0 0x7f6f0c1cc9d0 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009820 tx=0x7f6efc009be0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.113+0000 7f6ef37fe700 10 client.35676.objecter ms_handle_connect 0x7f6ef803a2f0 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f11065700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f11065700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 0x7f6f0c1cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f11065700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c06a400 con 0x7f6f0c1ccf10 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 0x7f6f0c1cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient: get_auth_request con 0x7f6f0c1ccf10 auth_method 0 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 10 monclient(hunting): handle_auth_done global_id 35261 payload 824 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 1 -- 172.21.15.47:0/1480027370 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6f0c066910 con 0x7f6f0c1ccf10 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6f0ad9d700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 0x7f6f0c1cd320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0046e0 tx=0x7f6efc004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6ef37fe700 10 client.35676.objecter ms_handle_connect 0x7f6f0c1ccf10 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.121+0000 7f6ef37fe700 10 client.35676.objecter resend_mon_ops 2024-04-20T02:24:33.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6efc024070 con 0x7f6f0c1ccf10 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 10 monclient: dump: 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.137 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f6efc029470 con 0x7f6f0c1ccf10 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 msgr2=0x7f6f0c1cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 0x7f6f0c1cd320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0046e0 tx=0x7f6efc004750 comp rx=0 tx=0).stop 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 msgr2=0x7f6f0c1cd320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6ef37fe700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6f0c1ccf10 0x7f6f0c1cd320 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0046e0 tx=0x7f6efc004750 comp rx=0 tx=0).stop 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:33.139 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.140 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.141 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.143 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.144 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:33.146 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.148 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:33.149 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.150 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.151 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.154 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:33.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.158 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.159 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.160 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:33.160 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:33.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 10 librados: watch_flush enter 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 10 librados: watch_flush exit 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 20 client.35676.objecter shutdown clearing up homeless session... 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 10 client.35676.objecter successfully canceled tick 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6ef803a2f0 msgr2=0x7f6ef803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6ef803a2f0 0x7f6ef803c7b0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009950 tx=0x7f6f00008030 comp rx=0 tx=0).stop 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 10 monclient: shutdown 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 msgr2=0x7f6f0c130860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 0x7f6f0c130860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400b9e0 tx=0x7f6f0400bcf0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f0b59e700 1 -- 172.21.15.47:0/1480027370 reap_dead start 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 shutdown_connections 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 --2- 172.21.15.47:0/1480027370 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f0c09cf30 0x7f6f0c130860 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.122+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 >> 172.21.15.47:0/1480027370 conn(0x7f6f0c0a3000 msgr2=0x7f6f0c0a0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.123+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 shutdown_connections 2024-04-20T02:24:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.123+0000 7f6f11065700 1 -- 172.21.15.47:0/1480027370 wait complete. 2024-04-20T02:24:33.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.123+0000 7f6f11065700 1 librados: shutdown 2024-04-20T02:24:33.163 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:33.163 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:24:33.163 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.302+0000 7f3cad22c700 10 monclient: get_monmap_and_config 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.303+0000 7f3cad22c700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.303+0000 7f3cad22c700 10 monclient: monmap: 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:33.304181+0000 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:33.304181+0000 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.303 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:33.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.303+0000 7f3cad22c700 1 Processor -- start 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.303+0000 7f3cad22c700 1 -- start start 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.303+0000 7f3cad22c700 10 monclient: init 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient: _add_conns ranks=[5,10,1,6,0,3,4,9,8,7,2] 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ca80a19d0 0x7f3ca80a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): picked mon.noname-h con 0x7f3ca80a19d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3ca80a42d0 0x7f3ca80701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): picked mon.noname-f con 0x7f3ca80a42d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:33.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3ca8135db0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca80751e0 con 0x7f3ca8135db0 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca8136810 con 0x7f3ca80a42d0 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8072020 con 0x7f3ca80a19d0 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3cad22c700 10 monclient(hunting): authenticate will time out at 2152.705295s 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3ca80a42d0 0x7f3ca80701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ca80a19d0 0x7f3ca80a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca759e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3ca8135db0 0x7f3ca80706f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:44784/0 (socket says 172.21.15.47:44784) 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca759e700 1 -- 172.21.15.47:0/3615116995 learned_addr learned my addr 172.21.15.47:0/3615116995 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:33.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): get_auth_request con 0x7f3ca80a42d0 auth_method 0 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): get_auth_request con 0x7f3ca80a19d0 auth_method 0 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient(hunting): handle_auth_done global_id 54550 payload 274 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 1 -- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ca80a19d0 msgr2=0x7f3ca80a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 1 --2- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ca80a19d0 0x7f3ca80a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 1 -- 172.21.15.47:0/3615116995 >> v1:172.21.15.179:6789/0 conn(0x7f3ca8135db0 legacy=0x7f3ca80706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 1 monclient: found mon.noname-f 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca659c700 1 -- 172.21.15.47:0/3615116995 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8072300 con 0x7f3ca80a42d0 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.304+0000 7f3ca6d9d700 1 --2- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ca80a19d0 0x7f3ca80a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca659c700 1 --2- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca80a42d0 0x7f3ca80701b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009d90 tx=0x7f3c9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3f67e9728342d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 1 -- 172.21.15.47:0/3615116995 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9c00e070 con 0x7f3ca80a42d0 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: dump: 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.319 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 1 -- 172.21.15.47:0/3615116995 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c0098f0 con 0x7f3ca80a42d0 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 5 monclient: authenticate success, global_id 54550 2024-04-20T02:24:33.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 1 -- 172.21.15.47:0/3615116995 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9c0128a0 con 0x7f3ca80a42d0 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 4 set_mon_vals no callback set 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3ca5d9b700 10 monclient: dump: 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.321 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 10 monclient: shutdown 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 -- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca80a42d0 msgr2=0x7f3ca80701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 --2- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca80a42d0 0x7f3ca80701b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009d90 tx=0x7f3c9c00b330 comp rx=0 tx=0).stop 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 -- 172.21.15.47:0/3615116995 shutdown_connections 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 --2- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3ca80a19d0 0x7f3ca80a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 --2- 172.21.15.47:0/3615116995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca80a42d0 0x7f3ca80701b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 -- 172.21.15.47:0/3615116995 >> 172.21.15.47:0/3615116995 conn(0x7f3ca80a4a50 msgr2=0x7f3ca80a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 -- 172.21.15.47:0/3615116995 shutdown_connections 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.305+0000 7f3cad22c700 1 -- 172.21.15.47:0/3615116995 wait complete. 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.306+0000 7f3cad22c700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.306+0000 7f3cad22c700 10 monclient: monmap: 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:33.323 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 librados: starting msgr at 2024-04-20T02:24:33.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 librados: starting objecter 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 Processor -- start 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 -- start start 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 librados: setting wanted keys 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 librados: calling monclient init 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient: init 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient: _add_conns ranks=[0,2,1,6,7,4,5,3,8] 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca80a19d0 0x7f3ca81c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): picked mon.noname-a con 0x7f3ca80a19d0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ca80a42d0 0x7f3ca8134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): picked mon.noname-b con 0x7f3ca80a42d0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8135db0 0x7f3ca81cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): picked mon.noname-f con 0x7f3ca8135db0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81c9da0 con 0x7f3ca80a19d0 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8071230 con 0x7f3ca8135db0 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81247b0 con 0x7f3ca80a42d0 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.307+0000 7f3cad22c700 10 monclient(hunting): authenticate will time out at 2152.708999s 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ca80a42d0 0x7f3ca8134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca80a19d0 0x7f3ca81c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ca80a42d0 0x7f3ca8134dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59252/0 (socket says 172.21.15.47:59252) 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 1 -- 172.21.15.47:0/956994944 learned_addr learned my addr 172.21.15.47:0/956994944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): get_auth_request con 0x7f3ca80a42d0 auth_method 0 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): get_auth_request con 0x7f3ca80a19d0 auth_method 0 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca759e700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8135db0 0x7f3ca81cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca759e700 10 monclient(hunting): get_auth_request con 0x7f3ca8135db0 auth_method 0 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient(hunting): handle_auth_done global_id 35685 payload 824 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ca80a42d0 msgr2=0x7f3ca8134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ca80a42d0 0x7f3ca8134dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8135db0 msgr2=0x7f3ca81cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8135db0 0x7f3ca81cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 monclient: found mon.noname-a 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980094a0 con 0x7f3ca80a19d0 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca659c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3ca80a42d0 0x7f3ca8134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca759e700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8135db0 0x7f3ca81cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca6d9d700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca80a19d0 0x7f3ca81c8e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e9b0 tx=0x7f3c9800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 client.?.objecter ms_handle_connect 0x7f3ca80a19d0 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9800ccc0 con 0x7f3ca80a19d0 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:33.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: dump: 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.330 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3cad22c700 5 monclient: authenticate success, global_id 35685 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800ce60 con 0x7f3ca80a19d0 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3cad22c700 10 monclient: _renew_subs 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3cad22c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c009a70 con 0x7f3ca80a19d0 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c980138e0 con 0x7f3ca80a19d0 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3cad22c700 10 client.35685.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3ca559a700 4 set_mon_vals no callback set 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.308+0000 7f3c8f7fe700 10 monclient: dump: 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.332 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 10 monclient: _renew_subs 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca8123f10 con 0x7f3ca80a19d0 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 1 librados: init done 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 0x7f3ca81cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8068430 con 0x7f3ca806aac0 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.309+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3c98013470 con 0x7f3ca80a19d0 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 0x7f3ca81cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient: get_auth_request con 0x7f3ca806aac0 auth_method 0 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f3c9804a9d0 con 0x7f3ca80a19d0 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 client.35685.objecter ms_dispatch 0x7f3ca8001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 3 client.35685.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:33.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 3 client.35685.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 20 client.35685.objecter dump_active .. 0 homeless 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 10 monclient(hunting): handle_auth_done global_id 54577 payload 824 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ca8067f90 con 0x7f3ca806aac0 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca759e700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca759e700 10 monclient: get_auth_request con 0x7f3c9403a2f0 auth_method 0 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 client.35685.objecter ms_handle_connect 0x7f3ca806aac0 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca659c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 0x7f3ca81cc850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80a6fe0 tx=0x7f3c9c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 client.35685.objecter resend_mon_ops 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9c0098f0 con 0x7f3ca806aac0 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 monclient: dump: 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.335 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3c8f7fe700 10 client.35685.objecter ms_handle_connect 0x7f3c9403a2f0 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.310+0000 7f3ca759e700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0005910 tx=0x7f3ca0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3c9c012430 con 0x7f3ca806aac0 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 msgr2=0x7f3ca81cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 0x7f3ca81cc850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80a6fe0 tx=0x7f3c9c004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 msgr2=0x7f3ca81cc850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.311+0000 7f3c8f7fe700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca806aac0 0x7f3ca81cc850 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80a6fe0 tx=0x7f3c9c004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3cad22c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:33.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3cad22c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 0x7f3ca81ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3cad22c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8124560 con 0x7f3ca8124100 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 0x7f3ca81ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient: get_auth_request con 0x7f3ca8124100 auth_method 0 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 10 monclient(hunting): handle_auth_done global_id 54586 payload 824 2024-04-20T02:24:33.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 1 -- 172.21.15.47:0/956994944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3ca806e240 con 0x7f3ca8124100 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3ca659c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 0x7f3ca81ccd90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ca81c9770 tx=0x7f3c9c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3c8f7fe700 10 client.35685.objecter ms_handle_connect 0x7f3ca8124100 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3c8f7fe700 10 client.35685.objecter resend_mon_ops 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9c02c330 con 0x7f3ca8124100 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3c8f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3c8f7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.319+0000 7f3c8f7fe700 10 monclient: dump: 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.341 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.342 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f3c9c030070 con 0x7f3ca8124100 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 msgr2=0x7f3ca81ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 0x7f3ca81ccd90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ca81c9770 tx=0x7f3c9c012000 comp rx=0 tx=0).stop 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 msgr2=0x7f3ca81ccd90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3c8f7fe700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca8124100 0x7f3ca81ccd90 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ca81c9770 tx=0x7f3c9c012000 comp rx=0 tx=0).stop 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:33.347 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.348 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.349 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:33.350 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:33.352 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.353 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:33.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.356 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.358 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.361 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:33.363 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:33.366 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:33.370 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.371 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:33.371 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 10 librados: watch_flush enter 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 10 librados: watch_flush exit 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 20 client.35685.objecter shutdown clearing up homeless session... 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 10 client.35685.objecter successfully canceled tick 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 msgr2=0x7f3c9403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 secure :-1 s=READY pgs=3217 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0005910 tx=0x7f3ca0009040 comp rx=0 tx=0).stop 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 10 monclient: shutdown 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca80a19d0 msgr2=0x7f3ca81c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 --2- 172.21.15.47:0/956994944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ca80a19d0 0x7f3ca81c8e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e9b0 tx=0x7f3c9800ecc0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3ca759e700 1 -- 172.21.15.47:0/956994944 reap_dead start 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3ca759e700 1 -- 172.21.15.47:0/956994944 reap_dead start 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 shutdown_connections 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 >> 172.21.15.47:0/956994944 conn(0x7f3ca80a4a50 msgr2=0x7f3ca80a6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 shutdown_connections 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 -- 172.21.15.47:0/956994944 wait complete. 2024-04-20T02:24:33.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.320+0000 7f3cad22c700 1 librados: shutdown 2024-04-20T02:24:33.374 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:33.375 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:24:33.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:24:33.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.514+0000 7f424166d700 10 monclient: get_monmap_and_config 2024-04-20T02:24:33.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.514+0000 7f424166d700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.514+0000 7f424166d700 10 monclient: monmap: 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:33.515298+0000 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:33.515298+0000 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.515 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:33.516 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.514+0000 7f424166d700 1 Processor -- start 2024-04-20T02:24:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.514+0000 7f424166d700 1 -- start start 2024-04-20T02:24:33.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.514+0000 7f424166d700 10 monclient: init 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient: _add_conns ranks=[6,8,1,5,10,9,0,7,4,2,3] 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 0x7f423c071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): picked mon.noname-d con 0x7f423c071730 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 0x7f423c0a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): picked mon.noname-e con 0x7f423c06f020 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f423c0a2db0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c0751e0 con 0x7f423c0a2db0 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f423c09fea0 con 0x7f423c071730 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f423c09e7b0 con 0x7f423c06f020 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f424166d700 10 monclient(hunting): authenticate will time out at 2152.916423s 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 0x7f423c071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 0x7f423c0a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 0x7f423c071b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51378/0 (socket says 172.21.15.47:51378) 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 0x7f423c0a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55728/0 (socket says 172.21.15.47:55728) 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 1 -- 172.21.15.47:0/2032591421 learned_addr learned my addr 172.21.15.47:0/2032591421 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): get_auth_request con 0x7f423c071730 auth_method 0 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): get_auth_request con 0x7f423c06f020 auth_method 0 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient(hunting): handle_auth_done global_id 35270 payload 274 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 -- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 msgr2=0x7f423c071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 --2- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 0x7f423c071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 -- 172.21.15.47:0/2032591421 >> v1:172.21.15.179:6789/0 conn(0x7f423c0a2db0 legacy=0x7f423c13b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 monclient: found mon.noname-e 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423a7fc700 1 -- 172.21.15.47:0/2032591421 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c06d190 con 0x7f423c06f020 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.515+0000 7f423affd700 1 --2- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 0x7f423c071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f423a7fc700 1 --2- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 0x7f423c0a2870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f422c009870 tx=0x7f422c009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2d87093a42c20ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 1 -- 172.21.15.47:0/2032591421 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f422c011aa0 con 0x7f423c06f020 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: dump: 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.529 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 5 monclient: authenticate success, global_id 35270 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 1 -- 172.21.15.47:0/2032591421 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c011c00 con 0x7f423c06f020 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 1 -- 172.21.15.47:0/2032591421 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f422c011d70 con 0x7f423c06f020 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 4 set_mon_vals no callback set 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f4239ffb700 10 monclient: dump: 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.531 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 10 monclient: shutdown 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 -- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 msgr2=0x7f423c0a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 --2- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 0x7f423c0a2870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f422c009870 tx=0x7f422c009b80 comp rx=0 tx=0).stop 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 -- 172.21.15.47:0/2032591421 shutdown_connections 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 --2- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f423c071730 0x7f423c071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 --2- 172.21.15.47:0/2032591421 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c06f020 0x7f423c0a2870 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 -- 172.21.15.47:0/2032591421 >> 172.21.15.47:0/2032591421 conn(0x7f423c0a3f90 msgr2=0x7f423c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.516+0000 7f424166d700 1 -- 172.21.15.47:0/2032591421 shutdown_connections 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.517+0000 7f424166d700 1 -- 172.21.15.47:0/2032591421 wait complete. 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.517+0000 7f424166d700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.517+0000 7f424166d700 10 monclient: monmap: 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.533 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.517+0000 7f424166d700 1 librados: starting msgr at 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.517+0000 7f424166d700 1 librados: starting objecter 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 Processor -- start 2024-04-20T02:24:33.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 -- start start 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 librados: setting wanted keys 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 librados: calling monclient init 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient: init 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient: _add_conns ranks=[3,4,8,1,5,6,2,0,7] 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06f020 0x7f423c1c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): picked mon.noname-g con 0x7f423c06f020 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f423c071730 0x7f423c134b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): picked mon.noname-c con 0x7f423c071730 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c0a2db0 0x7f423c135040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): picked mon.noname-e con 0x7f423c0a2db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c09f220 con 0x7f423c06f020 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f423c072410 con 0x7f423c071730 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f423c120180 con 0x7f423c0a2db0 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f424166d700 10 monclient(hunting): authenticate will time out at 2152.919709s 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c0a2db0 0x7f423c135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f423c071730 0x7f423c134b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c0a2db0 0x7f423c135040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55740/0 (socket says 172.21.15.47:55740) 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 1 -- 172.21.15.47:0/2194585467 learned_addr learned my addr 172.21.15.47:0/2194585467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): get_auth_request con 0x7f423c0a2db0 auth_method 0 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423affd700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06f020 0x7f423c1c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 10 monclient(hunting): get_auth_request con 0x7f423c071730 auth_method 0 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423affd700 10 monclient(hunting): get_auth_request con 0x7f423c06f020 auth_method 0 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.518+0000 7f423a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 10 monclient(hunting): handle_auth_done global_id 35279 payload 824 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f423c071730 msgr2=0x7f423c134b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f423c071730 0x7f423c134b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06f020 msgr2=0x7f423c1c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06f020 0x7f423c1c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 monclient: found mon.noname-e 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224009550 con 0x7f423c0a2db0 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f423b7fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c0a2db0 0x7f423c135040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f423400cb50 tx=0x7f423400ce60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 client.?.objecter ms_handle_connect 0x7f423c0a2db0 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4234003e70 con 0x7f423c0a2db0 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: dump: 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.539 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 5 monclient: authenticate success, global_id 35279 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400a7e0 con 0x7f423c0a2db0 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f423400ae00 con 0x7f423c0a2db0 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 10 monclient: _renew_subs 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0094a0 con 0x7f423c0a2db0 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42397fa700 4 set_mon_vals no callback set 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f42337fe700 10 monclient: dump: 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.541 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 10 client.35279.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:33.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 10 monclient: _renew_subs 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c1224e0 con 0x7f423c0a2db0 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.519+0000 7f424166d700 1 librados: init done 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f424166d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f424166d700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 0x7f423c1c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c122670 con 0x7f423c06aac0 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f423400a990 con 0x7f423c0a2db0 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f421c03a320 0x7f421c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f423404a720 con 0x7f423c0a2db0 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 10 client.35279.objecter ms_dispatch 0x7f423c001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 3 client.35279.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 3 client.35279.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:33.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423affd700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 0x7f423c1c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423affd700 10 monclient: get_auth_request con 0x7f423c06aac0 auth_method 0 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423a7fc700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f421c03a320 0x7f421c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f42337fe700 20 client.35279.objecter dump_active .. 0 homeless 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.520+0000 7f423a7fc700 10 monclient: get_auth_request con 0x7f421c03a320 auth_method 0 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423affd700 10 monclient(hunting): handle_auth_done global_id 22233 payload 824 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423affd700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f423c0a2c10 con 0x7f423c06aac0 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423a7fc700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f421c03a320 0x7f421c03c7e0 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7f422c009870 tx=0x7f422c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 client.35279.objecter ms_handle_connect 0x7f421c03a320 2024-04-20T02:24:33.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 client.35279.objecter ms_handle_connect 0x7f423c06aac0 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 client.35279.objecter resend_mon_ops 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f423affd700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 0x7f423c1c6ac0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4224011bd0 tx=0x7f4224011f90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f422400fe50 con 0x7f423c06aac0 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 monclient: dump: 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.545 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f422400f9e0 con 0x7f423c06aac0 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 msgr2=0x7f423c1c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 0x7f423c1c6ac0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4224011bd0 tx=0x7f4224011f90 comp rx=0 tx=0).stop 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 msgr2=0x7f423c1c6ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.521+0000 7f42337fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c06aac0 0x7f423c1c6ac0 secure :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4224011bd0 tx=0x7f4224011f90 comp rx=0 tx=0).stop 2024-04-20T02:24:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.529+0000 7f424166d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:33.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.529+0000 7f424166d700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 0x7f423c1c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.529+0000 7f424166d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.529+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c068280 con 0x7f423c121c40 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 0x7f423c1c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient: get_auth_request con 0x7f423c121c40 auth_method 0 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 10 monclient(hunting): handle_auth_done global_id 22242 payload 824 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 1 -- 172.21.15.47:0/2194585467 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f423c122160 con 0x7f423c121c40 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f423affd700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 0x7f423c1c7000 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4224011bd0 tx=0x7f4224004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f42337fe700 10 client.35279.objecter ms_handle_connect 0x7f423c121c40 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f42337fe700 10 client.35279.objecter resend_mon_ops 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f422402c8b0 con 0x7f423c121c40 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f42337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f42337fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.530+0000 7f42337fe700 10 monclient: dump: 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.550 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f422402c440 con 0x7f423c121c40 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 msgr2=0x7f423c1c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 0x7f423c1c7000 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4224011bd0 tx=0x7f4224004750 comp rx=0 tx=0).stop 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 msgr2=0x7f423c1c7000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f42337fe700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f423c121c40 0x7f423c1c7000 secure :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4224011bd0 tx=0x7f4224004750 comp rx=0 tx=0).stop 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:33.556 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:24:33.557 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.558 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.559 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:33.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.561 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:33.562 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.563 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:33.565 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.566 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.569 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:33.571 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.572 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.576 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:33.577 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:33.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 10 librados: watch_flush enter 2024-04-20T02:24:33.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 10 librados: watch_flush exit 2024-04-20T02:24:33.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 20 client.35279.objecter shutdown clearing up homeless session... 2024-04-20T02:24:33.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 10 client.35279.objecter successfully canceled tick 2024-04-20T02:24:33.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f421c03a320 msgr2=0x7f421c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f421c03a320 0x7f421c03c7e0 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 crypto rx=0x7f422c009870 tx=0x7f422c008030 comp rx=0 tx=0).stop 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 10 monclient: shutdown 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c0a2db0 msgr2=0x7f423c135040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 --2- 172.21.15.47:0/2194585467 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f423c0a2db0 0x7f423c135040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f423400cb50 tx=0x7f423400ce60 comp rx=0 tx=0).stop 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f423b7fe700 1 -- 172.21.15.47:0/2194585467 reap_dead start 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f423b7fe700 1 -- 172.21.15.47:0/2194585467 reap_dead start 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 shutdown_connections 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 >> 172.21.15.47:0/2194585467 conn(0x7f423c0a3f90 msgr2=0x7f423c0a4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 shutdown_connections 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 -- 172.21.15.47:0/2194585467 wait complete. 2024-04-20T02:24:33.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.531+0000 7f424166d700 1 librados: shutdown 2024-04-20T02:24:33.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:33.581 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:24:33.581 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:24:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.719+0000 7f921ae32700 10 monclient: get_monmap_and_config 2024-04-20T02:24:33.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.719+0000 7f921ae32700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.719+0000 7f921ae32700 10 monclient: monmap: 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:33.720785+0000 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:33.720785+0000 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.720 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.721 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.721 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:33.721 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:33.721 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 1 Processor -- start 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 1 -- start start 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 10 monclient: init 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 10 monclient: _add_conns ranks=[2,9,7,1,4,6,5,8,3,0,10] 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 0x7f9214137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 10 monclient(hunting): picked mon.noname-b con 0x7f92140a50f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92141378a0 0x7f9214138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.720+0000 7f921ae32700 10 monclient(hunting): picked mon.noname-a con 0x7f92141378a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f92140a38f0 0x7f92140a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 10 monclient(hunting): picked mon.noname-i con 0x7f92140a38f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92140a54d0 con 0x7f92141378a0 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92141387b0 con 0x7f92140a50f0 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f921406da20 con 0x7f92140a38f0 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f921ae32700 10 monclient(hunting): authenticate will time out at 2153.122047s 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92141378a0 0x7f9214138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 0x7f9214137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92141378a0 0x7f9214138270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40858/0 (socket says 172.21.15.47:40858) 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 -- 172.21.15.47:0/1468041714 learned_addr learned my addr 172.21.15.47:0/1468041714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 0x7f9214137d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59254/0 (socket says 172.21.15.47:59254) 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): get_auth_request con 0x7f92141378a0 auth_method 0 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): get_auth_request con 0x7f92140a50f0 auth_method 0 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient(hunting): handle_auth_done global_id 35694 payload 274 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f92193cf700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f92140a38f0 0x7f92140a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 -- 172.21.15.47:0/1468041714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f92140a38f0 msgr2=0x7f92140a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f92140a38f0 0x7f92140a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 -- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 msgr2=0x7f9214137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 0x7f9214137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 monclient: found mon.noname-a 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 -- 172.21.15.47:0/1468041714 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140751e0 con 0x7f92141378a0 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f92193cf700 10 monclient: get_auth_request con 0x7f92140a38f0 auth_method 0 2024-04-20T02:24:33.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f92193cf700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f92140a38f0 0x7f92140a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9218bce700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 0x7f9214137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f9213fff700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92141378a0 0x7f9214138270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92000098f0 tx=0x7f9200009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36860bc8dac7fe54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f92137fe700 1 -- 172.21.15.47:0/1468041714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9200011aa0 con 0x7f92141378a0 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.721+0000 7f92137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: dump: 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.736 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 5 monclient: authenticate success, global_id 35694 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 1 -- 172.21.15.47:0/1468041714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9200011c00 con 0x7f92141378a0 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 4 set_mon_vals no callback set 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 1 -- 172.21.15.47:0/1468041714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9200011d70 con 0x7f92141378a0 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f92137fe700 10 monclient: dump: 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.738 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 10 monclient: shutdown 2024-04-20T02:24:33.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 -- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92141378a0 msgr2=0x7f9214138270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92141378a0 0x7f9214138270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92000098f0 tx=0x7f9200009c00 comp rx=0 tx=0).stop 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 -- 172.21.15.47:0/1468041714 shutdown_connections 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f92140a38f0 0x7f92140a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a50f0 0x7f9214137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 --2- 172.21.15.47:0/1468041714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92141378a0 0x7f9214138270 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 -- 172.21.15.47:0/1468041714 >> 172.21.15.47:0/1468041714 conn(0x7f92140a8510 msgr2=0x7f92140a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 -- 172.21.15.47:0/1468041714 shutdown_connections 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.722+0000 7f921ae32700 1 -- 172.21.15.47:0/1468041714 wait complete. 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 10 monclient: monmap: 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:33.740 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 1 librados: starting msgr at 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 1 librados: starting objecter 2024-04-20T02:24:33.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 1 Processor -- start 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 1 -- start start 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 1 librados: setting wanted keys 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 1 librados: calling monclient init 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.723+0000 7f921ae32700 10 monclient: init 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient: _add_conns ranks=[2,8,1,0,4,3,5,7,6] 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a38f0 0x7f92141cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): picked mon.noname-b con 0x7f92140a38f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f92140a50f0 0x7f9214136920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): picked mon.noname-e con 0x7f92140a50f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9214136e60 0x7f92141203a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): picked mon.noname-f con 0x7f9214136e60 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9214123b60 con 0x7f9214136e60 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9214124300 con 0x7f92140a38f0 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9214121f30 con 0x7f92140a50f0 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f921ae32700 10 monclient(hunting): authenticate will time out at 2153.125591s 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a38f0 0x7f92141cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a38f0 0x7f92141cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59264/0 (socket says 172.21.15.47:59264) 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 1 -- 172.21.15.47:0/961281849 learned_addr learned my addr 172.21.15.47:0/961281849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): get_auth_request con 0x7f92140a38f0 auth_method 0 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f92140a50f0 0x7f9214136920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): get_auth_request con 0x7f92140a50f0 auth_method 0 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9218bce700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f92193cf700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9214136e60 0x7f92141203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f92193cf700 10 monclient(hunting): get_auth_request con 0x7f9214136e60 auth_method 0 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f92193cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f92193cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.724+0000 7f92193cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92193cf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92193cf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92193cf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 10 monclient(hunting): handle_auth_done global_id 24095 payload 824 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f92140a50f0 msgr2=0x7f9214136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f92140a50f0 0x7f9214136920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9214136e60 msgr2=0x7f92141203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9214136e60 0x7f92141203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 monclient: found mon.noname-b 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080094a0 con 0x7f92140a38f0 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9213fff700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f92140a50f0 0x7f9214136920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9218bce700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a38f0 0x7f92141cc270 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f920800ec10 tx=0x7f920800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 client.?.objecter ms_handle_connect 0x7f92140a38f0 2024-04-20T02:24:33.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f920800ccc0 con 0x7f92140a38f0 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: dump: 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.746 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 5 monclient: authenticate success, global_id 24095 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920800ce20 con 0x7f92140a38f0 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 10 monclient: _renew_subs 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92080138e0 con 0x7f92140a38f0 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9200009550 con 0x7f92140a38f0 2024-04-20T02:24:33.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f9212ffd700 4 set_mon_vals no callback set 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f92117fa700 10 monclient: dump: 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.748 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 10 client.24095.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 10 monclient: _renew_subs 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9214071e80 con 0x7f92140a38f0 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.725+0000 7f921ae32700 1 librados: init done 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f921ae32700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f921ae32700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 0x7f92141208e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92140682f0 con 0x7f92141378a0 2024-04-20T02:24:33.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9208013470 con 0x7f92140a38f0 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 0x7f92141208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient: get_auth_request con 0x7f92141378a0 auth_method 0 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91f803a2f0 0x7f91f803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f920804a8d0 con 0x7f92140a38f0 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 10 client.24095.objecter ms_dispatch 0x7f9214001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 3 client.24095.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 3 client.24095.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92117fa700 20 client.24095.objecter dump_active .. 0 homeless 2024-04-20T02:24:33.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f9213fff700 10 monclient(hunting): handle_auth_done global_id 64112 payload 824 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.726+0000 7f92193cf700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91f803a2f0 0x7f91f803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f9213fff700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f921406e370 con 0x7f92141378a0 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92193cf700 10 monclient: get_auth_request con 0x7f91f803a2f0 auth_method 0 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 client.24095.objecter ms_handle_connect 0x7f92141378a0 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 client.24095.objecter resend_mon_ops 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f9213fff700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 0x7f92141208e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92140a49b0 tx=0x7f9200004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9200020070 con 0x7f92141378a0 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 monclient: dump: 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.751 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 client.24095.objecter ms_handle_connect 0x7f91f803a2f0 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92193cf700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91f803a2f0 0x7f91f803c7b0 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7f9214121b00 tx=0x7f920c0076e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f92000036a0 con 0x7f92141378a0 2024-04-20T02:24:33.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:33.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 msgr2=0x7f92141208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 0x7f92141208e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92140a49b0 tx=0x7f9200004750 comp rx=0 tx=0).stop 2024-04-20T02:24:33.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 msgr2=0x7f92141208e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.727+0000 7f92117fa700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92141378a0 0x7f92141208e0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f92140a49b0 tx=0x7f9200004750 comp rx=0 tx=0).stop 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f921ae32700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f921ae32700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 0x7f9214120e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f921ae32700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f921406a400 con 0x7f921406aac0 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 0x7f9214120e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 10 monclient: get_auth_request con 0x7f921406aac0 auth_method 0 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.735+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f9213fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f9213fff700 10 monclient(hunting): handle_auth_done global_id 64121 payload 824 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f9213fff700 1 -- 172.21.15.47:0/961281849 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f921406e650 con 0x7f921406aac0 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f9213fff700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 0x7f9214120e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9200009c40 tx=0x7f9200011000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f92117fa700 10 client.24095.objecter ms_handle_connect 0x7f921406aac0 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f92117fa700 10 client.24095.objecter resend_mon_ops 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f920002b2a0 con 0x7f921406aac0 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f92117fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f92117fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.736+0000 7f92117fa700 10 monclient: dump: 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.756 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.757 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f920001b640 con 0x7f921406aac0 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 msgr2=0x7f9214120e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 0x7f9214120e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9200009c40 tx=0x7f9200011000 comp rx=0 tx=0).stop 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 msgr2=0x7f9214120e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92117fa700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f921406aac0 0x7f9214120e20 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9200009c40 tx=0x7f9200011000 comp rx=0 tx=0).stop 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:33.758 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.759 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:33.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:33.763 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:33.764 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.765 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.766 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.771 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:33.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.781 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:33.781 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 10 librados: watch_flush enter 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 10 librados: watch_flush exit 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 20 client.24095.objecter shutdown clearing up homeless session... 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 10 client.24095.objecter successfully canceled tick 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91f803a2f0 msgr2=0x7f91f803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91f803a2f0 0x7f91f803c7b0 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 crypto rx=0x7f9214121b00 tx=0x7f920c0076e0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 10 monclient: shutdown 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a38f0 msgr2=0x7f92141cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 --2- 172.21.15.47:0/961281849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92140a38f0 0x7f92141cc270 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f920800ec10 tx=0x7f920800ef20 comp rx=0 tx=0).stop 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92193cf700 1 -- 172.21.15.47:0/961281849 reap_dead start 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f92193cf700 1 -- 172.21.15.47:0/961281849 reap_dead start 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 shutdown_connections 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 >> 172.21.15.47:0/961281849 conn(0x7f92140a8510 msgr2=0x7f92140a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 shutdown_connections 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 -- 172.21.15.47:0/961281849 wait complete. 2024-04-20T02:24:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.737+0000 7f921ae32700 1 librados: shutdown 2024-04-20T02:24:33.784 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:33.784 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:24:33.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:24:33.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.922+0000 7f1ab89e1700 10 monclient: get_monmap_and_config 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.922+0000 7f1ab89e1700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.922+0000 7f1ab89e1700 10 monclient: monmap: 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:33.923959+0000 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:33.923959+0000 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:33.924 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.923+0000 7f1ab89e1700 1 Processor -- start 2024-04-20T02:24:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.923+0000 7f1ab89e1700 1 -- start start 2024-04-20T02:24:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.923+0000 7f1ab89e1700 10 monclient: init 2024-04-20T02:24:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient: _add_conns ranks=[0,10,2,3,6,5,7,9,8,4,1] 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1ab40998c0 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1ab4099ca0 0x7f1ab413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): picked mon.noname-f con 0x7f1ab4099ca0 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ab413caf0 0x7f1ab413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): picked mon.noname-b con 0x7f1ab413caf0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40751e0 con 0x7f1ab40998c0 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab409a600 con 0x7f1ab4099ca0 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab409b460 con 0x7f1ab413caf0 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab89e1700 10 monclient(hunting): authenticate will time out at 2153.325214s 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ab413caf0 0x7f1ab413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1ab40998c0 0x7f1ab409c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33536/0 (socket says 172.21.15.47:33536) 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab259c700 1 -- 172.21.15.47:0/1044827047 learned_addr learned my addr 172.21.15.47:0/1044827047 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): get_auth_request con 0x7f1ab413caf0 auth_method 0 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 1 --2- 172.21.15.47:0/1044827047 >> v2:172.21.15.179:3300/0 conn(0x7f1ab4099ca0 0x7f1ab413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): get_auth_request con 0x7f1ab4099ca0 auth_method 0 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_done global_id 24104 payload 274 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab159a700 1 -- 172.21.15.47:0/1044827047 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384541687 0 0) 0x7f1ab40751e0 con 0x7f1ab40998c0 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/1044827047 >> v2:172.21.15.179:3300/0 conn(0x7f1ab4099ca0 msgr2=0x7f1ab413c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 --2- 172.21.15.47:0/1044827047 >> v2:172.21.15.179:3300/0 conn(0x7f1ab4099ca0 0x7f1ab413c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/1044827047 >> v1:172.21.15.47:6789/0 conn(0x7f1ab40998c0 legacy=0x7f1ab409c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 monclient: found mon.noname-b 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/1044827047 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab409b740 con 0x7f1ab413caf0 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab159a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.924+0000 7f1ab2d9d700 1 --2- 172.21.15.47:0/1044827047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ab413caf0 0x7f1ab413eee0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009dc0 tx=0x7f1aac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d0ab9a2c24c2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 1 -- 172.21.15.47:0/1044827047 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aac00e070 con 0x7f1ab413caf0 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: dump: 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 1 -- 172.21.15.47:0/1044827047 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac0098f0 con 0x7f1ab413caf0 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 5 monclient: authenticate success, global_id 24104 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:33.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 4 set_mon_vals no callback set 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 1 -- 172.21.15.47:0/1044827047 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aac0128a0 con 0x7f1ab413caf0 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab159a700 10 monclient: dump: 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.940 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 10 monclient: shutdown 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 -- 172.21.15.47:0/1044827047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ab413caf0 msgr2=0x7f1ab413eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/1044827047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ab413caf0 0x7f1ab413eee0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009dc0 tx=0x7f1aac00b330 comp rx=0 tx=0).stop 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 -- 172.21.15.47:0/1044827047 shutdown_connections 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/1044827047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1ab413caf0 0x7f1ab413eee0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/1044827047 >> v2:172.21.15.179:3300/0 conn(0x7f1ab4099ca0 0x7f1ab413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 -- 172.21.15.47:0/1044827047 >> 172.21.15.47:0/1044827047 conn(0x7f1ab40a2f00 msgr2=0x7f1ab40a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 -- 172.21.15.47:0/1044827047 shutdown_connections 2024-04-20T02:24:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.925+0000 7f1ab89e1700 1 -- 172.21.15.47:0/1044827047 wait complete. 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.926+0000 7f1ab89e1700 10 monclient: build_initial_monmap 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.926+0000 7f1ab89e1700 10 monclient: monmap: 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.926+0000 7f1ab89e1700 1 librados: starting msgr at 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.926+0000 7f1ab89e1700 1 librados: starting objecter 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 Processor -- start 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 -- start start 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 librados: setting wanted keys 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 librados: calling monclient init 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient: init 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient: _add_conns ranks=[5,3,4,6,7,8,1,0,2] 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1ab40998c0 0x7f1ab41cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): picked mon.noname-h con 0x7f1ab40998c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ab4099ca0 0x7f1ab4098710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): picked mon.noname-g con 0x7f1ab4099ca0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 0x7f1ab4098c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): picked mon.noname-c con 0x7f1ab413caf0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab41cd110 con 0x7f1ab4099ca0 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4090bb0 con 0x7f1ab413caf0 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab41cc900 con 0x7f1ab40998c0 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab89e1700 10 monclient(hunting): authenticate will time out at 2153.328708s 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 0x7f1ab4098c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 0x7f1ab4098c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48466/0 (socket says 172.21.15.47:48466) 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/3032668098 learned_addr learned my addr 172.21.15.47:0/3032668098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): get_auth_request con 0x7f1ab413caf0 auth_method 0 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab259c700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1ab40998c0 0x7f1ab41cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab1d9b700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ab4099ca0 0x7f1ab4098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab1d9b700 10 monclient(hunting): get_auth_request con 0x7f1ab4099ca0 auth_method 0 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab259c700 10 monclient(hunting): get_auth_request con 0x7f1ab40998c0 auth_method 0 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.927+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 10 monclient(hunting): handle_auth_done global_id 54436 payload 824 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1ab40998c0 msgr2=0x7f1ab41cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1ab40998c0 0x7f1ab41cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ab4099ca0 msgr2=0x7f1ab4098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ab4099ca0 0x7f1ab4098710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 monclient: found mon.noname-c 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac009a70 con 0x7f1ab413caf0 2024-04-20T02:24:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab2d9d700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 0x7f1ab4098c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00b370 tx=0x7f1aac004ad0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 client.?.objecter ms_handle_connect 0x7f1ab413caf0 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aac004740 con 0x7f1ab413caf0 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: dump: 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac004920 con 0x7f1ab413caf0 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aac0128a0 con 0x7f1ab413caf0 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab0d99700 4 set_mon_vals no callback set 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1aa2ffd700 10 monclient: dump: 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.949 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 5 monclient: authenticate success, global_id 54436 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 10 monclient: _renew_subs 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab409a870 con 0x7f1ab413caf0 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 10 client.54436.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 10 monclient: _renew_subs 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab4099190 con 0x7f1ab413caf0 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.928+0000 7f1ab89e1700 1 librados: init done 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab89e1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 0x7f1ab4091e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4068180 con 0x7f1ab406aac0 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 0x7f1ab4091e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient: get_auth_request con 0x7f1ab406aac0 auth_method 0 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1aac012430 con 0x7f1ab413caf0 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 10 monclient(hunting): handle_auth_done global_id 54403 payload 824 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.929+0000 7f1ab259c700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ab409a470 con 0x7f1ab406aac0 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9403a2f0 0x7f1a9403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1aac04c0f0 con 0x7f1ab413caf0 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 client.54436.objecter ms_dispatch 0x7f1ab4001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 3 client.54436.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 3 client.54436.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1ab259c700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 0x7f1ab4091e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000f60 tx=0x7f1aa40018f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 20 client.54436.objecter dump_active .. 0 homeless 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 client.54436.objecter ms_handle_connect 0x7f1ab406aac0 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 client.54436.objecter resend_mon_ops 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1ab1d9b700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9403a2f0 0x7f1a9403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1ab1d9b700 10 monclient: get_auth_request con 0x7f1a9403a2f0 auth_method 0 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aa4017070 con 0x7f1ab406aac0 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 monclient: dump: 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1aa4012560 con 0x7f1ab406aac0 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:33.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 msgr2=0x7f1ab4091e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 0x7f1ab4091e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000f60 tx=0x7f1aa40018f0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 msgr2=0x7f1ab4091e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab406aac0 0x7f1ab4091e10 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000f60 tx=0x7f1aa40018f0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1aa2ffd700 10 client.54436.objecter ms_handle_connect 0x7f1a9403a2f0 2024-04-20T02:24:33.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.930+0000 7f1ab1d9b700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9403a2f0 0x7f1a9403c7b0 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4072750 tx=0x7f1aa8009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab89e1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:33.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 0x7f1ab4071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab89e1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab406a400 con 0x7f1ab4070ff0 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 0x7f1ab4071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 10 monclient: get_auth_request con 0x7f1ab4070ff0 auth_method 0 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.938+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1ab259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1ab259c700 10 monclient(hunting): handle_auth_done global_id 54412 payload 824 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1ab259c700 1 -- 172.21.15.47:0/3032668098 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1ab409bd00 con 0x7f1ab4070ff0 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 client.54436.objecter ms_handle_connect 0x7f1ab4070ff0 2024-04-20T02:24:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 client.54436.objecter resend_mon_ops 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1ab259c700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 0x7f1ab4071400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4015040 tx=0x7f1aa4000ed0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aa4023070 con 0x7f1ab4070ff0 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 monclient: dump: 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:33.957 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f1aa4012e90 con 0x7f1ab4070ff0 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 msgr2=0x7f1ab4071400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 0x7f1ab4071400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4015040 tx=0x7f1aa4000ed0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 msgr2=0x7f1ab4071400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.939+0000 7f1aa2ffd700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1ab4070ff0 0x7f1ab4071400 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4015040 tx=0x7f1aa4000ed0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.959 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:33.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:33.962 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:33.963 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.964 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:33.966 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:33.967 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:33.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.971 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:33.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:33.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:33.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:33.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:33.984 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:33.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:33.984 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:33.984 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 10 librados: watch_flush enter 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 10 librados: watch_flush exit 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 20 client.54436.objecter shutdown clearing up homeless session... 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 10 client.54436.objecter successfully canceled tick 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9403a2f0 msgr2=0x7f1a9403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9403a2f0 0x7f1a9403c7b0 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4072750 tx=0x7f1aa8009210 comp rx=0 tx=0).stop 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 10 monclient: shutdown 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 msgr2=0x7f1ab4098c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 0x7f1ab4098c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00b370 tx=0x7f1aac004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab2d9d700 1 -- 172.21.15.47:0/3032668098 reap_dead start 2024-04-20T02:24:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 shutdown_connections 2024-04-20T02:24:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 --2- 172.21.15.47:0/3032668098 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1ab413caf0 0x7f1ab4098c50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 >> 172.21.15.47:0/3032668098 conn(0x7f1ab40a2f00 msgr2=0x7f1ab40a5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 shutdown_connections 2024-04-20T02:24:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 -- 172.21.15.47:0/3032668098 wait complete. 2024-04-20T02:24:33.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:33.940+0000 7f1ab89e1700 1 librados: shutdown 2024-04-20T02:24:33.987 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:33.987 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:24:33.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:24:34.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.136+0000 7f6b85aa1700 10 monclient: get_monmap_and_config 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.136+0000 7f6b85aa1700 10 monclient: build_initial_monmap 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.136+0000 7f6b85aa1700 10 monclient: monmap: 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:34.137749+0000 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:34.137749+0000 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:34.137 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:34.138 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:34.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.137+0000 7f6b85aa1700 1 Processor -- start 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.137+0000 7f6b85aa1700 1 -- start start 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.137+0000 7f6b85aa1700 10 monclient: init 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient: _add_conns ranks=[7,0,5,3,2,1,10,6,4,8,9] 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80134ba0 0x7f6b80134f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): picked mon.noname-i con 0x7f6b80134ba0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6b80134790 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6b80072330 0x7f6b800a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): picked mon.noname-h con 0x7f6b80072330 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b800751e0 con 0x7f6b80134790 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800a0710 con 0x7f6b80072330 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800a0430 con 0x7f6b80134ba0 2024-04-20T02:24:34.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b85aa1700 10 monclient(hunting): authenticate will time out at 2153.539223s 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7effd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6b80134790 0x7f6b800a2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33540/0 (socket says 172.21.15.47:33540) 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7effd700 1 -- 172.21.15.47:0/2729190755 learned_addr learned my addr 172.21.15.47:0/2729190755 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80134ba0 0x7f6b80134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6b80072330 0x7f6b800a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): get_auth_request con 0x7f6b80134ba0 auth_method 0 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): get_auth_request con 0x7f6b80072330 auth_method 0 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:34.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7e7fc700 1 -- 172.21.15.47:0/2729190755 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756973312 0 0) 0x7f6b800751e0 con 0x7f6b80134790 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7e7fc700 10 monclient(hunting): _negotiate 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7e7fc700 10 monclient(hunting): my global_id is 35712 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7e7fc700 1 -- 172.21.15.47:0/2729190755 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b680035b0 con 0x7f6b80134790 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_done global_id 54421 payload 274 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 -- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6b80072330 msgr2=0x7f6b800a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6b80072330 0x7f6b800a2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 -- 172.21.15.47:0/2729190755 >> v1:172.21.15.47:6789/0 conn(0x7f6b80134790 legacy=0x7f6b800a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:34.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 monclient: found mon.noname-i 2024-04-20T02:24:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:24:34.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 -- 172.21.15.47:0/2729190755 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800a0150 con 0x7f6b80134ba0 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.138+0000 7f6b7f7fe700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80134ba0 0x7f6b80134f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009870 tx=0x7f6b70009b80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=83dc57865e7d3188 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 1 -- 172.21.15.47:0/2729190755 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b70011aa0 con 0x7f6b80134ba0 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: dump: 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:34.153 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 5 monclient: authenticate success, global_id 54421 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 1 -- 172.21.15.47:0/2729190755 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70011c00 con 0x7f6b80134ba0 2024-04-20T02:24:34.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 4 set_mon_vals no callback set 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 1 -- 172.21.15.47:0/2729190755 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b70011d70 con 0x7f6b80134ba0 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b7e7fc700 10 monclient: dump: 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:34.155 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 10 monclient: shutdown 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 -- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80134ba0 msgr2=0x7f6b80134f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80134ba0 0x7f6b80134f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009870 tx=0x7f6b70009b80 comp rx=0 tx=0).stop 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 -- 172.21.15.47:0/2729190755 shutdown_connections 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80134ba0 0x7f6b80134f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/2729190755 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6b80072330 0x7f6b800a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 -- 172.21.15.47:0/2729190755 >> 172.21.15.47:0/2729190755 conn(0x7f6b800a75a0 msgr2=0x7f6b800a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 -- 172.21.15.47:0/2729190755 shutdown_connections 2024-04-20T02:24:34.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.139+0000 7f6b85aa1700 1 -- 172.21.15.47:0/2729190755 wait complete. 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.140+0000 7f6b85aa1700 10 monclient: build_initial_monmap 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.140+0000 7f6b85aa1700 10 monclient: monmap: 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:34.157 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.140+0000 7f6b85aa1700 1 librados: starting msgr at 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.140+0000 7f6b85aa1700 1 librados: starting objecter 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.140+0000 7f6b85aa1700 1 Processor -- start 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 -- start start 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 librados: setting wanted keys 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 librados: calling monclient init 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient: init 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient: _add_conns ranks=[7,1,4,3,0,6,2,8,5] 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80072330 0x7f6b8006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): picked mon.noname-i con 0x7f6b80072330 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:34.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b80134790 0x7f6b8006f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): picked mon.noname-f con 0x7f6b80134790 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 0x7f6b8006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): picked mon.noname-c con 0x7f6b80134ba0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80132040 con 0x7f6b80134790 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80070c10 con 0x7f6b80134ba0 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80072780 con 0x7f6b80072330 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b85aa1700 10 monclient(hunting): authenticate will time out at 2153.542766s 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 0x7f6b8006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 0x7f6b8006fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48480/0 (socket says 172.21.15.47:48480) 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 1 -- 172.21.15.47:0/493597102 learned_addr learned my addr 172.21.15.47:0/493597102 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 10 monclient(hunting): get_auth_request con 0x7f6b80134ba0 auth_method 0 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:34.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.141+0000 7f6b7f7fe700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80072330 0x7f6b8006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): get_auth_request con 0x7f6b80072330 auth_method 0 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b80134790 0x7f6b8006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): get_auth_request con 0x7f6b80134790 auth_method 0 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:34.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 10 monclient(hunting): handle_auth_done global_id 54445 payload 824 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 -- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80072330 msgr2=0x7f6b8006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80072330 0x7f6b8006f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 -- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b80134790 msgr2=0x7f6b8006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b80134790 0x7f6b8006f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 monclient: found mon.noname-c 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 -- 172.21.15.47:0/493597102 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700094a0 con 0x7f6b80134ba0 2024-04-20T02:24:34.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7ffff700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 0x7f6b8006fb60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800efc0 tx=0x7f6b7800c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 client.?.objecter ms_handle_connect 0x7f6b80134ba0 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b78009ea0 con 0x7f6b80134ba0 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: dump: 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:34.162 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: _finish_auth 0 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7800f040 con 0x7f6b80134ba0 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 5 monclient: authenticate success, global_id 54445 2024-04-20T02:24:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 10 monclient: _renew_subs 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b80132270 con 0x7f6b80134ba0 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6b780148f0 con 0x7f6b80134ba0 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b67fff700 10 monclient: dump: 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b7dffb700 4 set_mon_vals no callback set 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 10 client.54445.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 10 monclient: _renew_subs 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b80125b40 con 0x7f6b80134ba0 2024-04-20T02:24:34.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.142+0000 7f6b85aa1700 1 librados: init done 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6b78014480 con 0x7f6b80134ba0 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b85aa1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b85aa1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b85aa1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b800754c0 con 0x7f6b80134ba0 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6b6c03a2f0 0x7f6b6c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6b7804afa0 con 0x7f6b80134ba0 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 10 client.54445.objecter ms_dispatch 0x7f6b80001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 3 client.54445.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 3 client.54445.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.143+0000 7f6b67fff700 20 client.54445.objecter dump_active .. 0 homeless 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.144+0000 7f6b7f7fe700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6b6c03a2f0 0x7f6b6c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:34.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.144+0000 7f6b7f7fe700 10 monclient: get_auth_request con 0x7f6b6c03a2f0 auth_method 0 2024-04-20T02:24:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.144+0000 7f6b67fff700 10 client.54445.objecter ms_handle_connect 0x7f6b6c03a2f0 2024-04-20T02:24:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.144+0000 7f6b7f7fe700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6b6c03a2f0 0x7f6b6c03c7b0 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7f6b700090b0 tx=0x7f6b70008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:34.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.148+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f6b7801a2c0 con 0x7f6b80134ba0 2024-04-20T02:24:34.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.148+0000 7f6b67fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:34.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.148+0000 7f6b67fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.460+0000 7f6b85aa1700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:24:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.460+0000 7f6b85aa1700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:24:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.460+0000 7f6b85aa1700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:24:34.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.460+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6b80124900 con 0x7f6b80134ba0 2024-04-20T02:24:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.462+0000 7f6b67fff700 1 -- 172.21.15.47:0/493597102 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f6b78018020 con 0x7f6b80134ba0 2024-04-20T02:24:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.462+0000 7f6b67fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:24:34.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.462+0000 7f6b67fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.462+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.462+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.462+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=11,kv=9,logm=65} crc {auth=760631056,config=692619927,health=3581076842,kv=105211281,logm=3288610307}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.462+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=14,config=2,health=12,kv=10,logm=62} crc {auth=3206937617,config=692619927,health=241004519,kv=3014085110,logm=1117257459}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=760631056,config=692619927,health=241004519,kv=1530195301,logm=4101670630}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=14,config=2,health=12,kv=10,logm=62} crc {auth=3206937617,config=692619927,health=3541615967,kv=2226345239,logm=2077276131}) 2024-04-20T02:24:34.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=13,config=2,health=11,kv=11,logm=63} crc {auth=604845746,config=692619927,health=684908111,kv=1530195301,logm=1401666421}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=13,config=2,health=11,kv=11,logm=63} crc {auth=760631056,config=692619927,health=749776571,kv=1530195301,logm=2014000661}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=12,config=2,health=12,kv=11,logm=63} crc {auth=4261964011,config=692619927,health=241004519,kv=1530195301,logm=2623955764}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=1,health=12,kv=11,logm=62} crc {auth=3206937617,config=3678701417,health=241004519,kv=1530195301,logm=798842788}) 2024-04-20T02:24:34.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.463+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=12,config=2,health=9,kv=11,logm=66} crc {auth=3787436684,config=692619927,health=2022437498,kv=1530195301,logm=3739957295}) 2024-04-20T02:24:34.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.464+0000 7f6b85aa1700 10 librados: watch_flush enter 2024-04-20T02:24:34.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.464+0000 7f6b85aa1700 10 librados: watch_flush exit 2024-04-20T02:24:34.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.464+0000 7f6b85aa1700 20 client.54445.objecter shutdown clearing up homeless session... 2024-04-20T02:24:34.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.464+0000 7f6b85aa1700 10 client.54445.objecter successfully canceled tick 2024-04-20T02:24:34.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6b6c03a2f0 msgr2=0x7f6b6c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6b6c03a2f0 0x7f6b6c03c7b0 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 crypto rx=0x7f6b700090b0 tx=0x7f6b70008030 comp rx=0 tx=0).stop 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 10 monclient: shutdown 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 msgr2=0x7f6b8006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 0x7f6b8006fb60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800efc0 tx=0x7f6b7800c3a0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 shutdown_connections 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b80134ba0 0x7f6b8006fb60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6b80072330 0x7f6b8006f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6b6c03a2f0 0x7f6b6c03c7b0 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 --2- 172.21.15.47:0/493597102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b80134790 0x7f6b8006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 >> 172.21.15.47:0/493597102 conn(0x7f6b800a75a0 msgr2=0x7f6b800712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.465+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 shutdown_connections 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.466+0000 7f6b85aa1700 1 -- 172.21.15.47:0/493597102 wait complete. 2024-04-20T02:24:34.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:34.466+0000 7f6b85aa1700 1 librados: shutdown 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966935116}) 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3170138010}) 2024-04-20T02:24:34.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2687249675}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1471691500}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=148743767}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3855054527}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2084474834}) 2024-04-20T02:24:34.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.466+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.467+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609829214}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.467+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3464596969}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1716540827}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=631190175}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1381474749}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1912088493}) 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.468+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2542587060}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2707678936}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3459769006}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=196729259}) 2024-04-20T02:24:34.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.469+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=630721423}) 2024-04-20T02:24:34.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1653438815}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268992688}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2081191897}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2389212440}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1441506000}) 2024-04-20T02:24:34.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2824842684}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3825776776}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047606202}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.472+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2007867207}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1401239386}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2372613852}) 2024-04-20T02:24:34.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2118923258}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1863619268}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2478460761}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1178728001}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=810411840}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=436237343}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.474+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1728062607}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2568809317}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1861897025}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3470125405}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1324466765}) 2024-04-20T02:24:34.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1202429834}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2805279043}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.475+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.476+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3980924020}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.476+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.476+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1810315310}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.476+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3845251953}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1963002400}) 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=11483602}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3293069851}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1499685433}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2743218536}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3082014413}) 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4196754118}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473988605}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.478+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1392853607}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1137337868}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624968302}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3953315443}) 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1256586524}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.479+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2156139784}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194916198}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3085268968}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=659989681}) 2024-04-20T02:24:34.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.480+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2558416980}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3212259370}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554362507}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.481+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1890100191}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4181478937}) 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1371869122}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1465030464}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2077019040}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2251642196}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.482+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=811889606}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2446944909}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2430417}) 2024-04-20T02:24:34.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3657430984}) 2024-04-20T02:24:34.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.495 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1356732936}) 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3904468004}) 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145827577}) 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.605 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=868503581}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.483+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2094776447}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.484+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2594241563}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3388358375}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2444753709}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2392397199}) 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.606 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1623905951}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=124014562}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=849889117}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2678749154}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3419565306}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.485+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=495245009}) 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.607 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=500115057}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632129556}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3338837262}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1870622408}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4236616424}) 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.608 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3756399672}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1142109065}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=959630782}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.487+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=379686524}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4000484161}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2613474156}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.609 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2426061711}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1859153279}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035942512}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=234932703}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=977818443}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1270418601}) 2024-04-20T02:24:34.610 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.488+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=915758906}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1317108382}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2981320852}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3189255535}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2196401472}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.611 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2027925148}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2154061885}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1463303424}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1919767995}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.490+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=952024113}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=618267290}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.612 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=948265515}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2161138567}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=523819888}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.491+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=886239615}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1935721636}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.613 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1953857760}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3035126128}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.492+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=551202251}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4186487372}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=520104875}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3821314250}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.614 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2912065481}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3532174957}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=870414510}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2435418993}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696396496}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.493+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2077225930}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.615 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287035770}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4163251358}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2778373821}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1813626992}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1763573473}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1472021259}) 2024-04-20T02:24:34.616 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=509394502}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=468099539}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.495+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2646077418}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3401552891}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4116865850}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3235418633}) 2024-04-20T02:24:34.617 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2274662648}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619337742}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2115743450}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1962393025}) 2024-04-20T02:24:34.618 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244514763}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.497+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4241341853}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1599064672}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3230441889}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3734275325}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3486403054}) 2024-04-20T02:24:34.619 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=899382714}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402552342}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2658928788}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1544745167}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.499+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4063249992}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3794456907}) 2024-04-20T02:24:34.620 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2369589931}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2218591684}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2635005276}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154960293}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3795634245}) 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.621 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=431513333}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134910844}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.501+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1381882217}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2894546791}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1482464426}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=308742287}) 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.622 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=371115025}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4293167081}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2354391501}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2624542912}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600515906}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.502+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2210119010}) 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.623 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2168408983}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2462231831}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3800548396}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2814313713}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=5680851}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.624 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3003555646}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4157568312}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=990064697}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.504+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2570096750}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.505+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.505+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.505+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121123362}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.505+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.505+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.505+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=627776905}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.625 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3734871739}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2681669948}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=899329451}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3119389958}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3464269454}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=258138243}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.506+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2822673533}) 2024-04-20T02:24:34.626 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2249180710}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1777830121}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3893229592}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3727015625}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=616384845}) 2024-04-20T02:24:34.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3417977696}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2626873312}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573460301}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.507+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3926515679}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=98348197}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=813307768}) 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3642079900}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2887726340}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=698110161}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1288763568}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=259462441}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2654942424}) 2024-04-20T02:24:34.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.509+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=640411078}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2446031717}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2864027096}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4267456163}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2131180197}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2880213560}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2238378955}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4072541273}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=898913368}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.511+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=934021015}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3768054283}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2883351725}) 2024-04-20T02:24:34.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1558680688}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.512+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3806442146}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2617224904}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2330894849}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2904134672}) 2024-04-20T02:24:34.632 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385922558}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.513+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3537052039}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1666270740}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2698089013}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4255217246}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2898976792}) 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.633 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2679477261}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=803085236}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=803985497}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521780949}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.514+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=644534433}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=647830408}) 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.634 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4156370981}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=728136772}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2920325413}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=864204556}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2607399510}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.635 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2176684680}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=191882496}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.516+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2654711259}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2038933758}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3437630833}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2271391242}) 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.636 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4079453863}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.517+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3280458794}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3389903733}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=482444173}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3272249521}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.518+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1863286243}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.637 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1226199261}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3074893247}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=135725063}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1356155067}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=740146285}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.638 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3002487890}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=456871007}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3811883889}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.519+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=873176014}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.520+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.520+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.520+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3669954957}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.520+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.520+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1868771201}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.639 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1357523368}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1005380294}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1422004105}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3699190308}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4101250821}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1370612350}) 2024-04-20T02:24:34.640 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.521+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=151821498}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4255521710}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3962968239}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2152597194}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=196266268}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.641 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3230837301}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=986201945}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=891334212}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3500484347}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.522+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1920393116}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=298250336}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.642 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=311633700}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2832545900}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1768272681}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3650708584}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3086616526}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2516761113}) 2024-04-20T02:24:34.643 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=767404054}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.524+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=96579056}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2757409291}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1725397110}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2939240948}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=157159458}) 2024-04-20T02:24:34.644 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1183507067}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=311205086}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2349463537}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=156177061}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.526+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1394431002}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.527+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.527+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.527+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3069921003}) 2024-04-20T02:24:34.645 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.527+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.527+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.527+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3582407517}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3463381387}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3125612544}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3009930182}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.646 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2018681839}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3599465224}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1382151683}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.528+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1975716883}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=64451596}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1448630246}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.647 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4132917394}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3030755938}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3062703672}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1339306226}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2026364219}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3176662718}) 2024-04-20T02:24:34.648 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.529+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476101533}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3043459344}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3278343141}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=369926343}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=773568672}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4078042087}) 2024-04-20T02:24:34.649 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3763208450}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4196526991}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2078196010}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.531+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1441408308}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3656849296}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.650 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3809509415}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1477927410}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2298042122}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3395216006}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1999869129}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3492146145}) 2024-04-20T02:24:34.651 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=664858250}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.532+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2299872501}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2496612}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2538168734}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3020037868}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1260695495}) 2024-04-20T02:24:34.652 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3932054116}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4006425770}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=758633744}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3734539965}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.535+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=722367717}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1761658955}) 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.653 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=825217028}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1873238773}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1397729300}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2485967374}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=601199374}) 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.654 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3940534026}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617720295}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.536+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3337889389}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1462326373}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=494018078}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=597651034}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.655 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3380038417}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2319504253}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=681138917}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1218617966}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3221088636}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.538+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=627241052}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3855348376}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=380112893}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=25755701}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2309276553}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=873317535}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1537975872}) 2024-04-20T02:24:34.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1807477365}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1229416739}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.540+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1863492165}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1014642763}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4125979086}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2528482164}) 2024-04-20T02:24:34.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2171397487}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=596958015}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=436377787}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3226443838}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602241687}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.541+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=713207564}) 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3449650281}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3167851648}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3629354742}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1448722798}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3361897345}) 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2606836553}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3286782720}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=251522062}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.544+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2451828932}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002226997}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347119259}) 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2327379833}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3678254293}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1981609835}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3682021865}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=203718332}) 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.545+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1880498003}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.546+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=904056908}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2457737012}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=790345409}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=834905918}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3614447535}) 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3280352668}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1155760297}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=90415408}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=746646396}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.547+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2444021334}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.548+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.548+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.548+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1395915815}) 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=16243098}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4030964039}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1800530819}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1628110358}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3851427351}) 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963728819}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985775738}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.549+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3437251287}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1130709285}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=572280622}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1465432951}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=720288967}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1006226794}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=156602197}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1415364302}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1931030982}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.550+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3123957107}) 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68602157}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2894896076}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4150157139}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=386210734}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019786524}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1639548715}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2946633928}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.552+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.553+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375361987}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.553+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2397611055}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643562363}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3606226046}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=108215381}) 2024-04-20T02:24:34.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2099256718}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409685183}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265650243}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3782849543}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1988322506}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.554+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3608103147}) 2024-04-20T02:24:34.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3758719942}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4150405181}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=457197890}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3510007634}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3451132817}) 2024-04-20T02:24:34.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.556+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301064710}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3985788009}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2155887601}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.557+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4120914773}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2598966923}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=275458888}) 2024-04-20T02:24:34.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1549511998}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3609599528}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.558+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2241732036}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2843329937}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3827255312}) 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1215340511}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.559+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=932951613}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3105599115}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3402900731}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2649038560}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1056000146}) 2024-04-20T02:24:34.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.560+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1999286300}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1857708276}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2370652486}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=685191714}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.561+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3874181017}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2206525473}) 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.675 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=452092343}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=231281090}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3185921916}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154398967}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=487052693}) 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.676 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3980535319}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131737567}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.562+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1396044994}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1035092013}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4124588191}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1759962717}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.677 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1767479652}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1241231473}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3601264}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2339381709}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1757945701}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.564+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232747140}) 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.565+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.678 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.565+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.565+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2266465693}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.565+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.565+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.565+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=971666621}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4173913601}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1614716747}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453647205}) 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.679 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3121955569}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1397724599}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=891545198}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.566+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3801349634}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019360683}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=631370029}) 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.680 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3084281079}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2566125142}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=12128426}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=393853793}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.567+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.568+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2222305123}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.568+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.568+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369201987}) 2024-04-20T02:24:34.681 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.568+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1915029515}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=82,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=5} crc {logm=717610642,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1657719613}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7} crc {logm=2911819113,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=235540247}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=84,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=4} crc {logm=4079501302,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3553808849}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.569+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=84,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=3} crc {logm=1747205868,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2897469061}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.682 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=83,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=4} crc {logm=4173004897,mdsmap=2394902515,mgr=2890147634,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3574838952}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=84,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=3} crc {logm=2853639409,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1560611122}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=91663746,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1124089021}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=7,mon_config_key=1} crc {logm=1701577387,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgrstat=1096436490,mon_config_key=616194535}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.570+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=82,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=6} crc {logm=764372091,mdsmap=1531151339,mgr=1969983733,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=726799030}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=33,osdmap=49} crc {mon_config_key=3396760964,monmap=3140514009,osd_metadata=3566517105,osd_pg_creating=2479375589,osd_snap=1704733134,osdmap=604842760}) 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.683 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=33,osdmap=49} crc {mon_config_key=3954964383,monmap=3385036551,osd_metadata=1356405380,osd_pg_creating=2479375589,osd_snap=1323390046,osdmap=3050061804}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.571+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=9,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=30,osdmap=51} crc {mon_config_key=1064859455,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2479375589,osd_snap=854289923,osdmap=1464986527}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=10,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=33,osdmap=48} crc {mon_config_key=3954964383,monmap=2370797809,osd_metadata=3566517105,osd_pg_creating=2479375589,osd_snap=1165923605,osdmap=2254691181}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=31,osdmap=50} crc {mon_config_key=305563582,monmap=3385036551,osd_metadata=2946886609,osd_pg_creating=2479375589,osd_snap=1167691856,osdmap=3034064675}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=10,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=33,osdmap=49} crc {mon_config_key=3954964383,monmap=2370797809,osd_metadata=1133748594,osd_pg_creating=2479375589,osd_snap=1396868874,osdmap=2959090208}) 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.684 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=28,osdmap=52} crc {mon_config_key=3954964383,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2479375589,osd_snap=3359452313,osdmap=3806564341}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=9,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=28,osdmap=55} crc {mon_config_key=2278087863,monmap=2370797809,osd_metadata=1881248415,osd_pg_creating=2479375589,osd_snap=2752815603,osdmap=1075813907}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.572+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=8,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=33,osdmap=50} crc {mon_config_key=853457861,monmap=2370797809,osd_metadata=3566517105,osd_pg_creating=2479375589,osd_snap=3087148638,osdmap=4272137910}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.573+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.573+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.573+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1781575552}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1818173993}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=774314181}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.685 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=678462685}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2704806645}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=312953780}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=105303844}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2667533588}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.574+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1700432538}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.686 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=75} crc {osdmap=488900495}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=79} crc {osdmap=1699957958}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=76} crc {osdmap=1742467375}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=80} crc {osdmap=2184713953}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=1537965217}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=80} crc {osdmap=116982999}) 2024-04-20T02:24:34.687 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.688 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.688 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.589+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=76} crc {osdmap=1817493391}) 2024-04-20T02:24:34.688 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.590+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:24:34.688 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.590+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=80} crc {osdmap=4055360404}) 2024-04-20T02:24:34.688 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:24:34.590+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=80} crc {osdmap=819562422}) 2024-04-20T02:24:35.496 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:24:35.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.638+0000 7f22f48f1700 10 monclient: get_monmap_and_config 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.638+0000 7f22f48f1700 10 monclient: build_initial_monmap 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.638+0000 7f22f48f1700 10 monclient: monmap: 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:35.639340+0000 2024-04-20T02:24:35.638 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:35.639340+0000 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:35.639 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.638+0000 7f22f48f1700 1 Processor -- start 2024-04-20T02:24:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.638+0000 7f22f48f1700 1 -- start start 2024-04-20T02:24:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.638+0000 7f22f48f1700 10 monclient: init 2024-04-20T02:24:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient: _add_conns ranks=[10,9,6,4,3,1,7,5,8,0,2] 2024-04-20T02:24:35.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f22f00a2950 0x7f22f00a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): picked mon.noname-f con 0x7f22f00a2950 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 0x7f22f0137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): picked mon.noname-a con 0x7f22f00a3270 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 0x7f22f0138470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): picked mon.noname-d con 0x7f22f0138040 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f01389b0 con 0x7f22f00a3270 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f009ef20 con 0x7f22f00a2950 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f22f009e900 con 0x7f22f0138040 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:35.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22f48f1700 10 monclient(hunting): authenticate will time out at 2155.040590s 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 0x7f22f0137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 0x7f22f0138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 0x7f22f0137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40872/0 (socket says 172.21.15.47:40872) 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 0x7f22f0138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51392/0 (socket says 172.21.15.47:51392) 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 1 -- 172.21.15.47:0/2669040075 learned_addr learned my addr 172.21.15.47:0/2669040075 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): get_auth_request con 0x7f22f00a3270 auth_method 0 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): get_auth_request con 0x7f22f0138040 auth_method 0 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:35.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22edd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22eed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22ee59c700 1 --2- 172.21.15.47:0/2669040075 >> v2:172.21.15.179:3300/0 conn(0x7f22f00a2950 0x7f22f00a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22ee59c700 10 monclient(hunting): get_auth_request con 0x7f22f00a2950 auth_method 0 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22ee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22ee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.639+0000 7f22ee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 10 monclient(hunting): handle_auth_done global_id 54411 payload 274 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 -- 172.21.15.47:0/2669040075 >> v2:172.21.15.179:3300/0 conn(0x7f22f00a2950 msgr2=0x7f22f00a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 --2- 172.21.15.47:0/2669040075 >> v2:172.21.15.179:3300/0 conn(0x7f22f00a2950 0x7f22f00a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 -- 172.21.15.47:0/2669040075 >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 msgr2=0x7f22f0137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:35.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 --2- 172.21.15.47:0/2669040075 >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 0x7f22f0137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 monclient: found mon.noname-d 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 -- 172.21.15.47:0/2669040075 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f00751e0 con 0x7f22f0138040 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22edd9b700 1 --2- 172.21.15.47:0/2669040075 >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 0x7f22f0137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22eed9d700 1 --2- 172.21.15.47:0/2669040075 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 0x7f22f0138470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009d90 tx=0x7f22e800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=32df4698f5ab6ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 1 -- 172.21.15.47:0/2669040075 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e800e070 con 0x7f22f0138040 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: dump: 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:35.653 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: _finish_auth 0 2024-04-20T02:24:35.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: _check_auth_tickets 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 1 -- 172.21.15.47:0/2669040075 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e80098f0 con 0x7f22f0138040 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22f48f1700 5 monclient: authenticate success, global_id 54411 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22f48f1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 1 -- 172.21.15.47:0/2669040075 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e80128a0 con 0x7f22f0138040 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22f48f1700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22f48f1700 4 set_mon_vals no callback set 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.640+0000 7f22ed59a700 10 monclient: dump: 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:35.655 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 10 monclient: shutdown 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:35.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 -- 172.21.15.47:0/2669040075 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 msgr2=0x7f22f0138470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2669040075 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 0x7f22f0138470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009d90 tx=0x7f22e800b330 comp rx=0 tx=0).stop 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 -- 172.21.15.47:0/2669040075 shutdown_connections 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2669040075 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f22f0138040 0x7f22f0138470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2669040075 >> v2:172.21.15.179:3300/0 conn(0x7f22f00a2950 0x7f22f00a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2669040075 >> v2:172.21.15.47:3300/0 conn(0x7f22f00a3270 0x7f22f0137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 -- 172.21.15.47:0/2669040075 >> 172.21.15.47:0/2669040075 conn(0x7f22f00a7570 msgr2=0x7f22f00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 -- 172.21.15.47:0/2669040075 shutdown_connections 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.641+0000 7f22f48f1700 1 -- 172.21.15.47:0/2669040075 wait complete. 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 10 monclient: build_initial_monmap 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 10 monclient: monmap: 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:35.657 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 1 librados: starting msgr at 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 1 librados: starting objecter 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 1 Processor -- start 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 1 -- start start 2024-04-20T02:24:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 1 librados: setting wanted keys 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 1 librados: calling monclient init 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.642+0000 7f22f48f1700 10 monclient: init 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient: _add_conns ranks=[5,3,2,1,4,8,0,6,7] 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22f00a2950 0x7f22f01bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): picked mon.noname-h con 0x7f22f00a2950 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22f00a3270 0x7f22f0136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): picked mon.noname-g con 0x7f22f00a3270 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 0x7f22f01373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): picked mon.noname-b con 0x7f22f0138040 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f009e560 con 0x7f22f0138040 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00720c0 con 0x7f22f00a3270 2024-04-20T02:24:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22f01c0390 con 0x7f22f00a2950 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 10 monclient(hunting): authenticate will time out at 2155.044137s 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 0x7f22f01373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 0x7f22f01373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59308/0 (socket says 172.21.15.47:59308) 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 -- 172.21.15.47:0/2814535042 learned_addr learned my addr 172.21.15.47:0/2814535042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): get_auth_request con 0x7f22f0138040 auth_method 0 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22ee59c700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22f00a2950 0x7f22f01bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22edd9b700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22f00a3270 0x7f22f0136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22ee59c700 10 monclient(hunting): get_auth_request con 0x7f22f00a2950 auth_method 0 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22ee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22ee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22ee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22edd9b700 10 monclient(hunting): get_auth_request con 0x7f22f00a3270 auth_method 0 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22edd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22edd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22edd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient(hunting): handle_auth_done global_id 24113 payload 824 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 -- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22f00a2950 msgr2=0x7f22f01bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22f00a2950 0x7f22f01bf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 -- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22f00a3270 msgr2=0x7f22f0136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22f00a3270 0x7f22f0136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 monclient: found mon.noname-b 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 -- 172.21.15.47:0/2814535042 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009550 con 0x7f22f0138040 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22edd9b700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22f00a3270 0x7f22f0136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22ee59c700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22f00a2950 0x7f22f01bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22eed9d700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 0x7f22f01373f0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f22e8014040 tx=0x7f22e8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 client.?.objecter ms_handle_connect 0x7f22f0138040 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e8004870 con 0x7f22f0138040 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 monclient: dump: 2024-04-20T02:24:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e8012430 con 0x7f22f0138040 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.643+0000 7f22f48f1700 5 monclient: authenticate success, global_id 24113 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 10 monclient: _renew_subs 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e8009a70 con 0x7f22f0138040 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22deffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22e8012a10 con 0x7f22f0138040 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22deffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22ecd99700 4 set_mon_vals no callback set 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22deffd700 10 monclient: dump: 2024-04-20T02:24:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 10 client.24113.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 10 monclient: _renew_subs 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0123fd0 con 0x7f22f0138040 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22f48f1700 1 librados: init done 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.644+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f22e80125a0 con 0x7f22f0138040 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22d003a320 0x7f22d003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22f48f1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22f48f1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22f48f1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f22e804c4f0 con 0x7f22f0138040 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f0137930 con 0x7f22f0138040 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 10 client.24113.objecter ms_dispatch 0x7f22f0001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 3 client.24113.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 3 client.24113.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 20 client.24113.objecter dump_active .. 0 homeless 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22ee59c700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22d003a320 0x7f22d003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22ee59c700 10 monclient: get_auth_request con 0x7f22d003a320 auth_method 0 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22deffd700 10 client.24113.objecter ms_handle_connect 0x7f22d003a320 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.645+0000 7f22ee59c700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22d003a320 0x7f22d003c7e0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7f22e00099f0 tx=0x7f22e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.650+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f22e802b020 con 0x7f22f0138040 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.650+0000 7f22deffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:35.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.650+0000 7f22deffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22f48f1700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:24:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22f48f1700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22f48f1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:35.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f22f0121eb0 con 0x7f22f0138040 2024-04-20T02:24:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22deffd700 1 -- 172.21.15.47:0/2814535042 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f22e8034080 con 0x7f22f0138040 2024-04-20T02:24:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22deffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:35.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.958+0000 7f22deffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:35.959 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":62,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:24:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.960+0000 7f22f48f1700 10 librados: watch_flush enter 2024-04-20T02:24:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.960+0000 7f22f48f1700 10 librados: watch_flush exit 2024-04-20T02:24:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.960+0000 7f22f48f1700 20 client.24113.objecter shutdown clearing up homeless session... 2024-04-20T02:24:35.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.960+0000 7f22f48f1700 10 client.24113.objecter successfully canceled tick 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22d003a320 msgr2=0x7f22d003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22d003a320 0x7f22d003c7e0 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 crypto rx=0x7f22e00099f0 tx=0x7f22e0008040 comp rx=0 tx=0).stop 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 10 monclient: shutdown 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 msgr2=0x7f22f01373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 0x7f22f01373f0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f22e8014040 tx=0x7f22e8004ad0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 shutdown_connections 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f22d003a320 0x7f22d003c7e0 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22f0138040 0x7f22f01373f0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f22f00a2950 0x7f22f01bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 --2- 172.21.15.47:0/2814535042 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f22f00a3270 0x7f22f0136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 >> 172.21.15.47:0/2814535042 conn(0x7f22f00a7570 msgr2=0x7f22f00a6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:35.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 shutdown_connections 2024-04-20T02:24:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 -- 172.21.15.47:0/2814535042 wait complete. 2024-04-20T02:24:35.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:35.961+0000 7f22f48f1700 1 librados: shutdown 2024-04-20T02:24:35.974 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:24:35.974 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:24:35.974 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:24:35.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:24:36.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.111+0000 7f8a0dc7f700 10 monclient: get_monmap_and_config 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.111+0000 7f8a0dc7f700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.111+0000 7f8a0dc7f700 10 monclient: monmap: 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:36.112755+0000 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:36.112755+0000 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.112 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:36.113 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 1 Processor -- start 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 1 -- start start 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient: init 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient: _add_conns ranks=[9,4,5,0,1,2,10,6,3,7,8] 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 0x7f8a080a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient(hunting): picked mon.noname-a con 0x7f8a080a0830 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:36.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 0x7f8a080a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient(hunting): picked mon.noname-c con 0x7f8a080a01b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a080a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient(hunting): picked mon.noname-h con 0x7f8a080a1150 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.112+0000 7f8a0dc7f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a08067820 con 0x7f8a080a0830 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0809deb0 con 0x7f8a080a01b0 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08067330 con 0x7f8a080a1150 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 10 monclient(hunting): authenticate will time out at 2155.514033s 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 0x7f8a080a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 0x7f8a080a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 0x7f8a080a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48482/0 (socket says 172.21.15.47:48482) 2024-04-20T02:24:36.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 -- 172.21.15.47:0/3131377292 learned_addr learned my addr 172.21.15.47:0/3131377292 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): get_auth_request con 0x7f8a080a01b0 auth_method 0 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 0x7f8a080a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40880/0 (socket says 172.21.15.47:40880) 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): get_auth_request con 0x7f8a080a0830 auth_method 0 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a07fff700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a080a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a07fff700 10 monclient(hunting): get_auth_request con 0x7f8a080a1150 auth_method 0 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a07fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a07fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a07fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_done global_id 54454 payload 274 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 -- 172.21.15.47:0/3131377292 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 msgr2=0x7f8a080a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a080a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 -- 172.21.15.47:0/3131377292 >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 msgr2=0x7f8a080a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 --2- 172.21.15.47:0/3131377292 >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 0x7f8a080a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 monclient: found mon.noname-c 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 -- 172.21.15.47:0/3131377292 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080751e0 con 0x7f8a080a01b0 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a07fff700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a080a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a077fe700 1 --2- 172.21.15.47:0/3131377292 >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 0x7f8a080a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a06ffd700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 0x7f8a080a2950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098f0 tx=0x7f89fc009c00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2b94d26b40b243ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 1 -- 172.21.15.47:0/3131377292 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89fc011aa0 con 0x7f8a080a01b0 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: dump: 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.128 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.129 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 1 -- 172.21.15.47:0/3131377292 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc011c00 con 0x7f8a080a01b0 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 5 monclient: authenticate success, global_id 54454 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 1 -- 172.21.15.47:0/3131377292 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89fc011d70 con 0x7f8a080a01b0 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a0dc7f700 4 set_mon_vals no callback set 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.113+0000 7f8a067fc700 10 monclient: dump: 2024-04-20T02:24:36.130 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.131 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 10 monclient: shutdown 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/3131377292 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 msgr2=0x7f8a080a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 0x7f8a080a2950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098f0 tx=0x7f89fc009c00 comp rx=0 tx=0).stop 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/3131377292 shutdown_connections 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a080a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/3131377292 >> v2:172.21.15.47:3300/0 conn(0x7f8a080a0830 0x7f8a080a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/3131377292 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a080a01b0 0x7f8a080a2950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/3131377292 >> 172.21.15.47:0/3131377292 conn(0x7f8a080a7570 msgr2=0x7f8a080a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/3131377292 shutdown_connections 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.114+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/3131377292 wait complete. 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 10 monclient: monmap: 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.132 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.133 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 1 librados: starting msgr at 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 1 librados: starting objecter 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 1 Processor -- start 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 1 -- start start 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 1 librados: setting wanted keys 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 1 librados: calling monclient init 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.115+0000 7f8a0dc7f700 10 monclient: init 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient: _add_conns ranks=[7,6,5,3,1,2,4,0,8] 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a080a01b0 0x7f8a081cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): picked mon.noname-i con 0x7f8a080a01b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a080a0830 0x7f8a081bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): picked mon.noname-d con 0x7f8a080a0830 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a081a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): picked mon.noname-h con 0x7f8a080a1150 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081cc480 con 0x7f8a080a1150 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a081bcb10 con 0x7f8a080a0830 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0809e440 con 0x7f8a080a01b0 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a0dc7f700 10 monclient(hunting): authenticate will time out at 2155.517353s 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a080a0830 0x7f8a081bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a080a0830 0x7f8a081bbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51404/0 (socket says 172.21.15.47:51404) 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 -- 172.21.15.47:0/2843723838 learned_addr learned my addr 172.21.15.47:0/2843723838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): get_auth_request con 0x7f8a080a0830 auth_method 0 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a077fe700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a080a01b0 0x7f8a081cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a07fff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a081a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a07fff700 10 monclient(hunting): get_auth_request con 0x7f8a080a1150 auth_method 0 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a07fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a07fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a07fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a077fe700 10 monclient(hunting): get_auth_request con 0x7f8a080a01b0 auth_method 0 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a077fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a077fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a077fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient(hunting): handle_auth_done global_id 54420 payload 824 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a080a01b0 msgr2=0x7f8a081cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a080a01b0 0x7f8a081cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 msgr2=0x7f8a081a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8a080a1150 0x7f8a081a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 monclient: found mon.noname-d 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80094a0 con 0x7f8a080a0830 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.116+0000 7f8a06ffd700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a080a0830 0x7f8a081bbf90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89fc013000 tx=0x7f89fc004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 client.?.objecter ms_handle_connect 0x7f8a080a0830 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89fc0045f0 con 0x7f8a080a0830 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: dump: 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.137 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc003b10 con 0x7f8a080a0830 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 5 monclient: authenticate success, global_id 54420 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 10 monclient: _renew_subs 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc009550 con 0x7f8a080a0830 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89fc003cc0 con 0x7f8a080a0830 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f89effff700 10 monclient: dump: 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.139 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.140 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a05ffb700 4 set_mon_vals no callback set 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 10 client.54420.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 10 monclient: _renew_subs 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0809e900 con 0x7f8a080a0830 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.117+0000 7f8a0dc7f700 1 librados: init done 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a0dc7f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f89fc021070 con 0x7f8a080a0830 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 0x7f8a081a8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08068310 con 0x7f8a0806aac0 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 0x7f8a081a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient: get_auth_request con 0x7f8a0806aac0 auth_method 0 2024-04-20T02:24:36.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f89f403a320 0x7f89f403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f89fc04bc60 con 0x7f8a080a0830 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 10 client.54420.objecter ms_dispatch 0x7f8a08001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 3 client.54420.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 3 client.54420.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 20 client.54420.objecter dump_active .. 0 homeless 2024-04-20T02:24:36.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 10 monclient(hunting): handle_auth_done global_id 35739 payload 824 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a07fff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f89f403a320 0x7f89f403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a0809f1c0 con 0x7f8a0806aac0 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a07fff700 10 monclient: get_auth_request con 0x7f89f403a320 auth_method 0 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 10 client.54420.objecter ms_handle_connect 0x7f8a0806aac0 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 10 client.54420.objecter resend_mon_ops 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f8a077fe700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 0x7f8a081a8060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89f80117c0 tx=0x7f89f8011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89f80044a0 con 0x7f8a0806aac0 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.118+0000 7f89effff700 10 monclient: dump: 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.143 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 10 client.54420.objecter ms_handle_connect 0x7f89f403a320 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f8a07fff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f89f403a320 0x7f89f403c7e0 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7f8a00003fd0 tx=0x7f8a000094f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f89f8004030 con 0x7f8a0806aac0 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 msgr2=0x7f8a081a8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 0x7f8a081a8060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89f80117c0 tx=0x7f89f8011b80 comp rx=0 tx=0).stop 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 msgr2=0x7f8a081a8060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.119+0000 7f89effff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0806aac0 0x7f8a081a8060 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f89f80117c0 tx=0x7f89f8011b80 comp rx=0 tx=0).stop 2024-04-20T02:24:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a0dc7f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 0x7f8a081a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a0dc7f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0806a400 con 0x7f8a081bc4d0 2024-04-20T02:24:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 0x7f8a081a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient: get_auth_request con 0x7f8a081bc4d0 auth_method 0 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 10 monclient(hunting): handle_auth_done global_id 35748 payload 824 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 1 -- 172.21.15.47:0/2843723838 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8a080a4450 con 0x7f8a081bc4d0 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f8a077fe700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 0x7f8a081a85a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89f80117c0 tx=0x7f89f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f89effff700 10 client.54420.objecter ms_handle_connect 0x7f8a081bc4d0 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f89effff700 10 client.54420.objecter resend_mon_ops 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89f8024070 con 0x7f8a081bc4d0 2024-04-20T02:24:36.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f89effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f89effff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.127+0000 7f89effff700 10 monclient: dump: 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.149 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7f89f8015670 con 0x7f8a081bc4d0 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 msgr2=0x7f8a081a85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 0x7f8a081a85a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89f80117c0 tx=0x7f89f8004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 msgr2=0x7f8a081a85a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f89effff700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a081bc4d0 0x7f8a081a85a0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89f80117c0 tx=0x7f89f8004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.150 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:36.151 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.152 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:36.153 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.154 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:36.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.160 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.165 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:36.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.170 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:36.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.172 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.173 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.173 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.173 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.173 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:36.173 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 10 librados: watch_flush enter 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 10 librados: watch_flush exit 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 20 client.54420.objecter shutdown clearing up homeless session... 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 10 client.54420.objecter successfully canceled tick 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f89f403a320 msgr2=0x7f89f403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f89f403a320 0x7f89f403c7e0 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 crypto rx=0x7f8a00003fd0 tx=0x7f8a000094f0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 10 monclient: shutdown 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a080a0830 msgr2=0x7f8a081bbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 --2- 172.21.15.47:0/2843723838 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a080a0830 0x7f8a081bbf90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89fc013000 tx=0x7f89fc004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a07fff700 1 -- 172.21.15.47:0/2843723838 reap_dead start 2024-04-20T02:24:36.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a07fff700 1 -- 172.21.15.47:0/2843723838 reap_dead start 2024-04-20T02:24:36.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 shutdown_connections 2024-04-20T02:24:36.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 >> 172.21.15.47:0/2843723838 conn(0x7f8a080a7570 msgr2=0x7f8a080a5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 shutdown_connections 2024-04-20T02:24:36.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.128+0000 7f8a0dc7f700 1 -- 172.21.15.47:0/2843723838 wait complete. 2024-04-20T02:24:36.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.129+0000 7f8a0dc7f700 1 librados: shutdown 2024-04-20T02:24:36.176 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:36.176 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:24:36.176 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:24:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.315+0000 7f11bff69700 10 monclient: get_monmap_and_config 2024-04-20T02:24:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.315+0000 7f11bff69700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.315+0000 7f11bff69700 10 monclient: monmap: 2024-04-20T02:24:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:36.316807+0000 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:36.316807+0000 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:36.318 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.316+0000 7f11bff69700 1 Processor -- start 2024-04-20T02:24:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.316+0000 7f11bff69700 1 -- start start 2024-04-20T02:24:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.316+0000 7f11bff69700 10 monclient: init 2024-04-20T02:24:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.316+0000 7f11bff69700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.316+0000 7f11bff69700 10 monclient: _add_conns ranks=[7,9,3,10,4,2,6,1,0,8,5] 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11b8071630 0x7f11b8071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): picked mon.noname-i con 0x7f11b8071630 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11b8071f50 0x7f11b80a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): picked mon.noname-a con 0x7f11b8071f50 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11b80a2e90 0x7f11b80a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): picked mon.noname-g con 0x7f11b80a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b8072720 con 0x7f11b8071f50 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80a0820 con 0x7f11b80a2e90 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11b809e890 con 0x7f11b8071630 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bff69700 10 monclient(hunting): authenticate will time out at 2155.718071s 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11b8071f50 0x7f11b80a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11b8071f50 0x7f11b80a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40890/0 (socket says 172.21.15.47:40890) 2024-04-20T02:24:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 -- 172.21.15.47:0/271554896 learned_addr learned my addr 172.21.15.47:0/271554896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): get_auth_request con 0x7f11b8071f50 auth_method 0 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bdd05700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11b8071630 0x7f11b8071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11be506700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11b80a2e90 0x7f11b80a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11be506700 10 monclient(hunting): get_auth_request con 0x7f11b80a2e90 auth_method 0 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11be506700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11be506700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11be506700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bdd05700 10 monclient(hunting): get_auth_request con 0x7f11b8071630 auth_method 0 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bdd05700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bdd05700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bdd05700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient(hunting): handle_auth_done global_id 35757 payload 274 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 -- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11b8071630 msgr2=0x7f11b8071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11b8071630 0x7f11b8071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 -- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11b80a2e90 msgr2=0x7f11b80a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11b80a2e90 0x7f11b80a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 monclient: found mon.noname-a 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 -- 172.21.15.47:0/271554896 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80751e0 con 0x7f11b8071f50 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bd504700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071f50 0x7f11b80a2950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009950 tx=0x7f11b4009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba19fc37179f97a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bcd03700 1 -- 172.21.15.47:0/271554896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11b4009ea0 con 0x7f11b8071f50 2024-04-20T02:24:36.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bcd03700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bcd03700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.317+0000 7f11bcd03700 10 monclient: dump: 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 1 -- 172.21.15.47:0/271554896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b40044a0 con 0x7f11b8071f50 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 5 monclient: authenticate success, global_id 35757 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 1 -- 172.21.15.47:0/271554896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11b400bc50 con 0x7f11b8071f50 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 4 set_mon_vals no callback set 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bcd03700 10 monclient: dump: 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 10 monclient: shutdown 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 -- 172.21.15.47:0/271554896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071f50 msgr2=0x7f11b80a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071f50 0x7f11b80a2950 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009950 tx=0x7f11b4009c60 comp rx=0 tx=0).stop 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 -- 172.21.15.47:0/271554896 shutdown_connections 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f11b80a2e90 0x7f11b80a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11b8071630 0x7f11b8071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 --2- 172.21.15.47:0/271554896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071f50 0x7f11b80a2950 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 -- 172.21.15.47:0/271554896 >> 172.21.15.47:0/271554896 conn(0x7f11b80a7570 msgr2=0x7f11b80a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 -- 172.21.15.47:0/271554896 shutdown_connections 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.318+0000 7f11bff69700 1 -- 172.21.15.47:0/271554896 wait complete. 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 10 monclient: monmap: 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 1 librados: starting msgr at 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 1 librados: starting objecter 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 1 Processor -- start 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 1 -- start start 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 1 librados: setting wanted keys 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 1 librados: calling monclient init 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.319+0000 7f11bff69700 10 monclient: init 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient: _add_conns ranks=[0,4,5,3,8,7,6,2,1] 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 0x7f11b81bed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): picked mon.noname-a con 0x7f11b8071630 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:36.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11b8071f50 0x7f11b81bf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): picked mon.noname-c con 0x7f11b8071f50 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11b80a2e90 0x7f11b81cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): picked mon.noname-h con 0x7f11b80a2e90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81bf7d0 con 0x7f11b8071630 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81ac150 con 0x7f11b8071f50 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11b809f270 con 0x7f11b80a2e90 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bff69700 10 monclient(hunting): authenticate will time out at 2155.721477s 2024-04-20T02:24:36.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 0x7f11b81bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 0x7f11b81bed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40898/0 (socket says 172.21.15.47:40898) 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 -- 172.21.15.47:0/4000438963 learned_addr learned my addr 172.21.15.47:0/4000438963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): get_auth_request con 0x7f11b8071630 auth_method 0 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11b8071f50 0x7f11b81bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): get_auth_request con 0x7f11b8071f50 auth_method 0 2024-04-20T02:24:36.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11b80a2e90 0x7f11b81cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): get_auth_request con 0x7f11b80a2e90 auth_method 0 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11be506700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient(hunting): handle_auth_done global_id 35766 payload 824 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11b80a2e90 msgr2=0x7f11b81cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f11b80a2e90 0x7f11b81cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11b8071f50 msgr2=0x7f11b81bf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11b8071f50 0x7f11b81bf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 monclient: found mon.noname-a 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.320+0000 7f11bdd05700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4009550 con 0x7f11b8071630 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bdd05700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 0x7f11b81bed50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e810 tx=0x7f11a800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 client.?.objecter ms_handle_connect 0x7f11b8071630 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:36.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11a800c070 con 0x7f11b8071630 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: dump: 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.345 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 5 monclient: authenticate success, global_id 35766 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8013880 con 0x7f11b8071630 2024-04-20T02:24:36.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 10 monclient: _renew_subs 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b81a9d70 con 0x7f11b8071630 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11a8013e60 con 0x7f11b8071630 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 10 client.35766.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11affff700 4 set_mon_vals no callback set 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11ae7fc700 10 monclient: dump: 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.347 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 10 monclient: _renew_subs 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b81aa3b0 con 0x7f11b8071630 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.321+0000 7f11bff69700 1 librados: init done 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bff69700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bff69700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 0x7f11b81cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8068310 con 0x7f11b806aac0 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f11a80139f0 con 0x7f11b8071630 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 0x7f11b81cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient: get_auth_request con 0x7f11b806aac0 auth_method 0 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119c03a2f0 0x7f119c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 10 monclient(hunting): handle_auth_done global_id 24122 payload 824 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f11a804a460 con 0x7f11b8071630 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11b80a4480 con 0x7f11b806aac0 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 10 client.35766.objecter ms_dispatch 0x7f11b8001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:36.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 3 client.35766.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 3 client.35766.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 20 client.35766.objecter dump_active .. 0 homeless 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 10 client.35766.objecter ms_handle_connect 0x7f11b806aac0 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 10 client.35766.objecter resend_mon_ops 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11bd504700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 0x7f11b81cc800 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009950 tx=0x7f11b40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11be506700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119c03a2f0 0x7f119c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11be506700 10 monclient: get_auth_request con 0x7f119c03a2f0 auth_method 0 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11b401f040 con 0x7f11b806aac0 2024-04-20T02:24:36.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.322+0000 7f11ae7fc700 10 monclient: dump: 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.351 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f11b40036a0 con 0x7f11b806aac0 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 msgr2=0x7f11b81cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 0x7f11b81cc800 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009950 tx=0x7f11b40047c0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 msgr2=0x7f11b81cc800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b806aac0 0x7f11b81cc800 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11ae7fc700 10 client.35766.objecter ms_handle_connect 0x7f119c03a2f0 2024-04-20T02:24:36.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.323+0000 7f11be506700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119c03a2f0 0x7f119c03c7b0 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f11b0003a20 tx=0x7f11b0009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bff69700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:36.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bff69700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 0x7f11b806f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bff69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b806a400 con 0x7f11b81ccd40 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 0x7f11b806f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient: get_auth_request con 0x7f11b81ccd40 auth_method 0 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 10 monclient(hunting): handle_auth_done global_id 24131 payload 824 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 1 -- 172.21.15.47:0/4000438963 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f11b806f090 con 0x7f11b81ccd40 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11ae7fc700 10 client.35766.objecter ms_handle_connect 0x7f11b81ccd40 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11ae7fc700 10 client.35766.objecter resend_mon_ops 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11bd504700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 0x7f11b806f300 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11b402c300 con 0x7f11b81ccd40 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11ae7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.331+0000 7f11ae7fc700 10 monclient: dump: 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.356 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f11b402c8e0 con 0x7f11b81ccd40 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 msgr2=0x7f11b806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 0x7f11b806f300 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b4004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 msgr2=0x7f11b806f300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11ae7fc700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11b81ccd40 0x7f11b806f300 secure :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b4004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:36.358 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.359 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.360 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.361 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.362 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:36.363 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:36.365 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.366 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:36.367 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.369 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:36.371 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.372 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.373 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.374 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.376 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.377 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.378 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.379 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:36.379 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 10 librados: watch_flush enter 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 10 librados: watch_flush exit 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 20 client.35766.objecter shutdown clearing up homeless session... 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 10 client.35766.objecter successfully canceled tick 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119c03a2f0 msgr2=0x7f119c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119c03a2f0 0x7f119c03c7b0 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 crypto rx=0x7f11b0003a20 tx=0x7f11b0009780 comp rx=0 tx=0).stop 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 10 monclient: shutdown 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 msgr2=0x7f11b81bed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 0x7f11b81bed50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11a800e810 tx=0x7f11a800eb20 comp rx=0 tx=0).stop 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11be506700 1 -- 172.21.15.47:0/4000438963 reap_dead start 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 shutdown_connections 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 --2- 172.21.15.47:0/4000438963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11b8071630 0x7f11b81bed50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 >> 172.21.15.47:0/4000438963 conn(0x7f11b80a7570 msgr2=0x7f11b809c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 shutdown_connections 2024-04-20T02:24:36.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 -- 172.21.15.47:0/4000438963 wait complete. 2024-04-20T02:24:36.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.332+0000 7f11bff69700 1 librados: shutdown 2024-04-20T02:24:36.382 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:36.382 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:24:36.382 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:24:36.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.520+0000 7efc2e9bf700 10 monclient: get_monmap_and_config 2024-04-20T02:24:36.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.520+0000 7efc2e9bf700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.520+0000 7efc2e9bf700 10 monclient: monmap: 2024-04-20T02:24:36.520 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:36.521542+0000 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:36.521542+0000 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.521 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.522 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.522 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.522 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:36.522 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:36.522 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 Processor -- start 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 -- start start 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient: init 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient: _add_conns ranks=[10,4,0,8,6,1,7,2,5,9,3] 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efc280a19d0 0x7efc280a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): picked mon.noname-f con 0x7efc280a19d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135f00 0x7efc280701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): picked mon.noname-c con 0x7efc28135f00 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efc28136380 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc280751e0 con 0x7efc28136380 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280a4040 con 0x7efc280a19d0 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc28072020 con 0x7efc28135f00 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2e9bf700 10 monclient(hunting): authenticate will time out at 2155.922887s 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc2cf5c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efc28136380 0x7efc280706f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33554/0 (socket says 172.21.15.47:33554) 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.521+0000 7efc277fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135f00 0x7efc280701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc2cf5c700 1 -- 172.21.15.47:0/2025997262 learned_addr learned my addr 172.21.15.47:0/2025997262 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): get_auth_request con 0x7efc28135f00 auth_method 0 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 1 --2- 172.21.15.47:0/2025997262 >> v2:172.21.15.179:3300/0 conn(0x7efc280a19d0 0x7efc280a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): get_auth_request con 0x7efc280a19d0 auth_method 0 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 1 -- 172.21.15.47:0/2025997262 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709093589 0 0) 0x7efc280751e0 con 0x7efc28136380 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient(hunting): _negotiate 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient(hunting): my global_id is 35775 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 1 -- 172.21.15.47:0/2025997262 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc100035b0 con 0x7efc28136380 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc27fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient(hunting): handle_auth_done global_id 54472 payload 274 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 1 -- 172.21.15.47:0/2025997262 >> v2:172.21.15.179:3300/0 conn(0x7efc280a19d0 msgr2=0x7efc280a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 1 --2- 172.21.15.47:0/2025997262 >> v2:172.21.15.179:3300/0 conn(0x7efc280a19d0 0x7efc280a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 1 -- 172.21.15.47:0/2025997262 >> v1:172.21.15.47:6789/0 conn(0x7efc28136380 legacy=0x7efc280706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 1 monclient: found mon.noname-c 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 1 -- 172.21.15.47:0/2025997262 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc28072300 con 0x7efc28135f00 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc277fe700 1 --2- 172.21.15.47:0/2025997262 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135f00 0x7efc280701b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009840 tx=0x7efc1c009b50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=21c2b404e466fcba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 1 -- 172.21.15.47:0/2025997262 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc1c011aa0 con 0x7efc28135f00 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: dump: 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.537 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc2e9bf700 5 monclient: authenticate success, global_id 54472 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 1 -- 172.21.15.47:0/2025997262 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c011c00 con 0x7efc28135f00 2024-04-20T02:24:36.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc2e9bf700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc2e9bf700 4 set_mon_vals no callback set 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 1 -- 172.21.15.47:0/2025997262 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc1c011d70 con 0x7efc28135f00 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.522+0000 7efc26ffd700 10 monclient: dump: 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.539 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 10 monclient: shutdown 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 -- 172.21.15.47:0/2025997262 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135f00 msgr2=0x7efc280701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/2025997262 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135f00 0x7efc280701b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009840 tx=0x7efc1c009b50 comp rx=0 tx=0).stop 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 -- 172.21.15.47:0/2025997262 shutdown_connections 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/2025997262 >> v2:172.21.15.179:3300/0 conn(0x7efc280a19d0 0x7efc280a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/2025997262 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135f00 0x7efc280701b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 -- 172.21.15.47:0/2025997262 >> 172.21.15.47:0/2025997262 conn(0x7efc280a4a50 msgr2=0x7efc280a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 -- 172.21.15.47:0/2025997262 shutdown_connections 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.523+0000 7efc2e9bf700 1 -- 172.21.15.47:0/2025997262 wait complete. 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 10 monclient: monmap: 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.541 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 1 librados: starting msgr at 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 1 librados: starting objecter 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 1 Processor -- start 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 1 -- start start 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 1 librados: setting wanted keys 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 1 librados: calling monclient init 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.524+0000 7efc2e9bf700 10 monclient: init 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient: _add_conns ranks=[6,3,5,4,0,2,1,7,8] 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efc280a19d0 0x7efc281cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): picked mon.noname-d con 0x7efc280a19d0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc28135f00 0x7efc28134ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): picked mon.noname-g con 0x7efc28135f00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efc28136380 0x7efc2806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): picked mon.noname-h con 0x7efc28136380 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc281241c0 con 0x7efc28135f00 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc2806e0e0 con 0x7efc28136380 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efc28071230 con 0x7efc280a19d0 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2e9bf700 10 monclient(hunting): authenticate will time out at 2155.926295s 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efc280a19d0 0x7efc281cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efc280a19d0 0x7efc281cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51416/0 (socket says 172.21.15.47:51416) 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 -- 172.21.15.47:0/829214771 learned_addr learned my addr 172.21.15.47:0/829214771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): get_auth_request con 0x7efc280a19d0 auth_method 0 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2cf5c700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efc28136380 0x7efc2806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc277fe700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc28135f00 0x7efc28134ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2cf5c700 10 monclient(hunting): get_auth_request con 0x7efc28136380 auth_method 0 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2cf5c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2cf5c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc2cf5c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient(hunting): handle_auth_done global_id 54429 payload 824 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efc28136380 msgr2=0x7efc2806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efc28136380 0x7efc2806c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc28135f00 msgr2=0x7efc28134ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc28135f00 0x7efc28134ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 monclient: found mon.noname-d 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0094a0 con 0x7efc280a19d0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc277fe700 10 monclient: get_auth_request con 0x7efc28135f00 auth_method 0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc277fe700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc28135f00 0x7efc28134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc27fff700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efc280a19d0 0x7efc281cc330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc1800e9b0 tx=0x7efc1800ed70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc24ff9700 10 client.?.objecter ms_handle_connect 0x7efc280a19d0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.525+0000 7efc24ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc18009710 con 0x7efc280a19d0 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: dump: 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800ccc0 con 0x7efc280a19d0 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 5 monclient: authenticate success, global_id 54429 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc1800ce30 con 0x7efc280a19d0 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 monclient: _renew_subs 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc28123c00 con 0x7efc280a19d0 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc267fc700 4 set_mon_vals no callback set 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc24ff9700 10 monclient: dump: 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.547 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 client.54429.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:36.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 monclient: _renew_subs 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc281cd190 con 0x7efc280a19d0 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 1 librados: init done 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 0x7efc2806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.526+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc28068430 con 0x7efc28135000 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7efc1800c850 con 0x7efc280a19d0 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 0x7efc2806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient: get_auth_request con 0x7efc28135000 auth_method 0 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efc1403a320 0x7efc1403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7efc1804a9a0 con 0x7efc280a19d0 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 10 client.54429.objecter ms_dispatch 0x7efc28001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 3 client.54429.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 3 client.54429.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 10 monclient(hunting): handle_auth_done global_id 54481 payload 824 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc28070e00 con 0x7efc28135000 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 20 client.54429.objecter dump_active .. 0 homeless 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc2cf5c700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efc1403a320 0x7efc1403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc2cf5c700 10 monclient: get_auth_request con 0x7efc1403a320 auth_method 0 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 10 client.54429.objecter ms_handle_connect 0x7efc28135000 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc277fe700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 0x7efc2806cce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009fd0 tx=0x7efc1c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 10 client.54429.objecter resend_mon_ops 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc1c0044b0 con 0x7efc28135000 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.527+0000 7efc24ff9700 10 monclient: dump: 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.551 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 10 client.54429.objecter ms_handle_connect 0x7efc1403a320 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc2cf5c700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efc1403a320 0x7efc1403c7e0 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7efc28070cb0 tx=0x7efc20009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efc1c0036a0 con 0x7efc28135000 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 msgr2=0x7efc2806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 0x7efc2806cce0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009fd0 tx=0x7efc1c004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 msgr2=0x7efc2806cce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.528+0000 7efc24ff9700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc28135000 0x7efc2806cce0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009fd0 tx=0x7efc1c004750 comp rx=0 tx=0).stop 2024-04-20T02:24:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.535+0000 7efc2e9bf700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.535+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 0x7efc2806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.535+0000 7efc2e9bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.535+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc28123d90 con 0x7efc2806aac0 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 0x7efc2806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient: get_auth_request con 0x7efc2806aac0 auth_method 0 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 10 monclient(hunting): handle_auth_done global_id 54490 payload 824 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 1 -- 172.21.15.47:0/829214771 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efc28067f90 con 0x7efc2806aac0 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 client.54429.objecter ms_handle_connect 0x7efc2806aac0 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 client.54429.objecter resend_mon_ops 2024-04-20T02:24:36.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc277fe700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 0x7efc2806d220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009b90 tx=0x7efc1c009210 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efc1c02b300 con 0x7efc2806aac0 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 monclient: dump: 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7efc1c004610 con 0x7efc2806aac0 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 msgr2=0x7efc2806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 0x7efc2806d220 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009b90 tx=0x7efc1c009210 comp rx=0 tx=0).stop 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 msgr2=0x7efc2806d220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.536+0000 7efc24ff9700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efc2806aac0 0x7efc2806d220 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009b90 tx=0x7efc1c009210 comp rx=0 tx=0).stop 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:36.558 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.559 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.560 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:36.561 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.562 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.564 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.568 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.571 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.574 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.578 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:36.579 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 10 librados: watch_flush enter 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 10 librados: watch_flush exit 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 20 client.54429.objecter shutdown clearing up homeless session... 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 10 client.54429.objecter successfully canceled tick 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efc1403a320 msgr2=0x7efc1403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efc1403a320 0x7efc1403c7e0 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 crypto rx=0x7efc28070cb0 tx=0x7efc20009040 comp rx=0 tx=0).stop 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 10 monclient: shutdown 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efc280a19d0 msgr2=0x7efc281cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 --2- 172.21.15.47:0/829214771 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efc280a19d0 0x7efc281cc330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc1800e9b0 tx=0x7efc1800ed70 comp rx=0 tx=0).stop 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2cf5c700 1 -- 172.21.15.47:0/829214771 reap_dead start 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2cf5c700 1 -- 172.21.15.47:0/829214771 reap_dead start 2024-04-20T02:24:36.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 shutdown_connections 2024-04-20T02:24:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 >> 172.21.15.47:0/829214771 conn(0x7efc280a4a50 msgr2=0x7efc281bdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 shutdown_connections 2024-04-20T02:24:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 -- 172.21.15.47:0/829214771 wait complete. 2024-04-20T02:24:36.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.537+0000 7efc2e9bf700 1 librados: shutdown 2024-04-20T02:24:36.582 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:36.582 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:24:36.582 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:24:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.720+0000 7f6fd4209700 10 monclient: get_monmap_and_config 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.720+0000 7f6fd4209700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.720+0000 7f6fd4209700 10 monclient: monmap: 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:36.722018+0000 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:36.722018+0000 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:36.722 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.721+0000 7f6fd4209700 1 Processor -- start 2024-04-20T02:24:36.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.721+0000 7f6fd4209700 1 -- start start 2024-04-20T02:24:36.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.721+0000 7f6fd4209700 10 monclient: init 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient: _add_conns ranks=[0,6,5,10,7,3,9,2,1,8,4] 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6fcc0a0b90 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc0a0f70 0x7f6fcc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): picked mon.noname-d con 0x7f6fcc0a0f70 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6fcc13db40 0x7f6fcc13ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): picked mon.noname-h con 0x7f6fcc13db40 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0751e0 con 0x7f6fcc0a0b90 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0a0210 con 0x7f6fcc13db40 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0673e0 con 0x7f6fcc0a0f70 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd4209700 10 monclient(hunting): authenticate will time out at 2156.123261s 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd1fa5700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6fcc0a0b90 0x7f6fcc0a2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33558/0 (socket says 172.21.15.47:33558) 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd1fa5700 1 -- 172.21.15.47:0/2765554779 learned_addr learned my addr 172.21.15.47:0/2765554779 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc0a0f70 0x7f6fcc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): get_auth_request con 0x7f6fcc0a0f70 auth_method 0 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd27a6700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6fcc13db40 0x7f6fcc13ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd27a6700 10 monclient(hunting): get_auth_request con 0x7f6fcc13db40 auth_method 0 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd27a6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd27a6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd27a6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd0fa3700 1 -- 172.21.15.47:0/2765554779 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261291708 0 0) 0x7f6fcc0751e0 con 0x7f6fcc0a0b90 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd0fa3700 10 monclient(hunting): _negotiate 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd0fa3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd0fa3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd0fa3700 10 monclient(hunting): my global_id is 35784 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd0fa3700 1 -- 172.21.15.47:0/2765554779 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb80035b0 con 0x7f6fcc0a0b90 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient(hunting): handle_auth_done global_id 54438 payload 274 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 1 -- 172.21.15.47:0/2765554779 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6fcc13db40 msgr2=0x7f6fcc13ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6fcc13db40 0x7f6fcc13ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 1 -- 172.21.15.47:0/2765554779 >> v1:172.21.15.47:6789/0 conn(0x7f6fcc0a0b90 legacy=0x7f6fcc0a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 1 monclient: found mon.noname-d 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd17a4700 1 -- 172.21.15.47:0/2765554779 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc067820 con 0x7f6fcc0a0f70 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.722+0000 7f6fd27a6700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6fcc13db40 0x7f6fcc13ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd17a4700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc0a0f70 0x7f6fcc0a3060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009870 tx=0x7f6fbc009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=97ae58978fbd896c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 1 -- 172.21.15.47:0/2765554779 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fbc011aa0 con 0x7f6fcc0a0f70 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: dump: 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 1 -- 172.21.15.47:0/2765554779 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc011c00 con 0x7f6fcc0a0f70 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 5 monclient: authenticate success, global_id 54438 2024-04-20T02:24:36.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 1 -- 172.21.15.47:0/2765554779 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fbc011d70 con 0x7f6fcc0a0f70 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 4 set_mon_vals no callback set 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd0fa3700 10 monclient: dump: 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.739 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 10 monclient: shutdown 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 -- 172.21.15.47:0/2765554779 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc0a0f70 msgr2=0x7f6fcc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc0a0f70 0x7f6fcc0a3060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009870 tx=0x7f6fbc009b80 comp rx=0 tx=0).stop 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 -- 172.21.15.47:0/2765554779 shutdown_connections 2024-04-20T02:24:36.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6fcc13db40 0x7f6fcc13ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 --2- 172.21.15.47:0/2765554779 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc0a0f70 0x7f6fcc0a3060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 -- 172.21.15.47:0/2765554779 >> 172.21.15.47:0/2765554779 conn(0x7f6fcc0a7570 msgr2=0x7f6fcc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 -- 172.21.15.47:0/2765554779 shutdown_connections 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.723+0000 7f6fd4209700 1 -- 172.21.15.47:0/2765554779 wait complete. 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.724+0000 7f6fd4209700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.724+0000 7f6fd4209700 10 monclient: monmap: 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:36.741 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 1 librados: starting msgr at 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 1 librados: starting objecter 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 1 Processor -- start 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 1 -- start start 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 1 librados: setting wanted keys 2024-04-20T02:24:36.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 1 librados: calling monclient init 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.725+0000 7f6fd4209700 10 monclient: init 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient: _add_conns ranks=[1,3,6,5,2,8,7,0,4] 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fcc0a0b90 0x7f6fcc09c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): picked mon.noname-f con 0x7f6fcc0a0b90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fcc0a0f70 0x7f6fcc138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): picked mon.noname-g con 0x7f6fcc0a0f70 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc13db40 0x7f6fcc1cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): picked mon.noname-d con 0x7f6fcc13db40 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc09e560 con 0x7f6fcc0a0b90 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1234e0 con 0x7f6fcc0a0f70 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc123f70 con 0x7f6fcc13db40 2024-04-20T02:24:36.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient(hunting): authenticate will time out at 2156.127120s 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc13db40 0x7f6fcc1cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc13db40 0x7f6fcc1cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51438/0 (socket says 172.21.15.47:51438) 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 -- 172.21.15.47:0/3269866767 learned_addr learned my addr 172.21.15.47:0/3269866767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): get_auth_request con 0x7f6fcc13db40 auth_method 0 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd1fa5700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fcc0a0b90 0x7f6fcc09c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd17a4700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fcc0a0f70 0x7f6fcc138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd17a4700 10 monclient(hunting): get_auth_request con 0x7f6fcc0a0f70 auth_method 0 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd17a4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd17a4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd17a4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd1fa5700 10 monclient(hunting): get_auth_request con 0x7f6fcc0a0b90 auth_method 0 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd1fa5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd1fa5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd1fa5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient(hunting): handle_auth_done global_id 54447 payload 824 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fcc0a0f70 msgr2=0x7f6fcc138e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fcc0a0f70 0x7f6fcc138e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fcc0a0b90 msgr2=0x7f6fcc09c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fcc0a0b90 0x7f6fcc09c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 monclient: found mon.noname-d 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0094a0 con 0x7f6fcc13db40 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd27a6700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc13db40 0x7f6fcc1cc200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4007ef0 tx=0x7f6fc400d3b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 client.?.objecter ms_handle_connect 0x7f6fcc13db40 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc400dcc0 con 0x7f6fcc13db40 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 monclient: dump: 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.746 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.747 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 5 monclient: authenticate success, global_id 54447 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc400f040 con 0x7f6fcc13db40 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient: _renew_subs 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc124250 con 0x7f6fcc13db40 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.726+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc400de40 con 0x7f6fcc13db40 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fc27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fc3fff700 4 set_mon_vals no callback set 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 10 client.54447.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fc27fc700 10 monclient: dump: 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 10 monclient: _renew_subs 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc123010 con 0x7f6fcc13db40 2024-04-20T02:24:36.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 1 librados: init done 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6fc40143a0 con 0x7f6fcc13db40 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 0x7f6fcc1cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc068310 con 0x7f6fcc06aac0 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd1fa5700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 0x7f6fcc1cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd1fa5700 10 monclient: get_auth_request con 0x7f6fcc06aac0 auth_method 0 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd1fa5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd1fa5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.727+0000 7f6fd1fa5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fb003a2f0 0x7f6fb003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6fc404a4c0 con 0x7f6fcc13db40 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 client.54447.objecter ms_dispatch 0x7f6fcc001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 3 client.54447.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 3 client.54447.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_done global_id 54456 payload 824 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd1fa5700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fcc123290 con 0x7f6fcc06aac0 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 20 client.54447.objecter dump_active .. 0 homeless 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 client.54447.objecter ms_handle_connect 0x7f6fcc06aac0 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd1fa5700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 0x7f6fcc1cc740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000f60 tx=0x7f6fc800ea90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 client.54447.objecter resend_mon_ops 2024-04-20T02:24:36.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd17a4700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fb003a2f0 0x7f6fb003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fd17a4700 10 monclient: get_auth_request con 0x7f6fb003a2f0 auth_method 0 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc80070d0 con 0x7f6fcc06aac0 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 monclient: dump: 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.753 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6fc8005b50 con 0x7f6fcc06aac0 2024-04-20T02:24:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 msgr2=0x7f6fcc1cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 0x7f6fcc1cc740 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000f60 tx=0x7f6fc800ea90 comp rx=0 tx=0).stop 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 msgr2=0x7f6fcc1cc740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.728+0000 7f6fc27fc700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc06aac0 0x7f6fcc1cc740 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000f60 tx=0x7f6fc800ea90 comp rx=0 tx=0).stop 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.730+0000 7f6fc27fc700 10 client.54447.objecter ms_handle_connect 0x7f6fb003a2f0 2024-04-20T02:24:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.730+0000 7f6fd17a4700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fb003a2f0 0x7f6fb003c7b0 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009870 tx=0x7f6fbc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd4209700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:36.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd4209700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 0x7f6fcc1ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd4209700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc06a400 con 0x7f6fcc1393b0 2024-04-20T02:24:36.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 0x7f6fcc1ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient: get_auth_request con 0x7f6fcc1393b0 auth_method 0 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.736+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fd1fa5700 10 monclient(hunting): handle_auth_done global_id 54465 payload 824 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fd1fa5700 1 -- 172.21.15.47:0/3269866767 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6fcc09f270 con 0x7f6fcc1393b0 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 client.54447.objecter ms_handle_connect 0x7f6fcc1393b0 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 client.54447.objecter resend_mon_ops 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fd1fa5700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 0x7f6fcc1ccc80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001460 tx=0x7f6fc80010a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6fc801cb40 con 0x7f6fcc1393b0 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 monclient: dump: 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f6fc801cca0 con 0x7f6fcc1393b0 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 msgr2=0x7f6fcc1ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 0x7f6fcc1ccc80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001460 tx=0x7f6fc80010a0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 msgr2=0x7f6fcc1ccc80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fc27fc700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc1393b0 0x7f6fcc1ccc80 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001460 tx=0x7f6fc80010a0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:36.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.761 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.762 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:36.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.765 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:36.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:36.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.772 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:36.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:36.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.780 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.781 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:36.782 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.737+0000 7f6fd4209700 10 librados: watch_flush enter 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 10 librados: watch_flush exit 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 20 client.54447.objecter shutdown clearing up homeless session... 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 10 client.54447.objecter successfully canceled tick 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fb003a2f0 msgr2=0x7f6fb003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6fb003a2f0 0x7f6fb003c7b0 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009870 tx=0x7f6fbc008030 comp rx=0 tx=0).stop 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 10 monclient: shutdown 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc13db40 msgr2=0x7f6fcc1cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 --2- 172.21.15.47:0/3269866767 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6fcc13db40 0x7f6fcc1cc200 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4007ef0 tx=0x7f6fc400d3b0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd27a6700 1 -- 172.21.15.47:0/3269866767 reap_dead start 2024-04-20T02:24:36.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd27a6700 1 -- 172.21.15.47:0/3269866767 reap_dead start 2024-04-20T02:24:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 shutdown_connections 2024-04-20T02:24:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 >> 172.21.15.47:0/3269866767 conn(0x7f6fcc0a7570 msgr2=0x7f6fcc0723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 shutdown_connections 2024-04-20T02:24:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 -- 172.21.15.47:0/3269866767 wait complete. 2024-04-20T02:24:36.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.738+0000 7f6fd4209700 1 librados: shutdown 2024-04-20T02:24:36.785 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:36.785 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:24:36.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.923+0000 7fb1b0b69700 10 monclient: get_monmap_and_config 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.923+0000 7fb1b0b69700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.923+0000 7fb1b0b69700 10 monclient: monmap: 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:36.924385+0000 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:36.924385+0000 2024-04-20T02:24:36.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:36.924 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.923+0000 7fb1b0b69700 1 Processor -- start 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.923+0000 7fb1b0b69700 1 -- start start 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.923+0000 7fb1b0b69700 10 monclient: init 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient: _add_conns ranks=[1,0,6,9,5,2,8,3,10,7,4] 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb1ac0a18a0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb1ac0a1c80 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:36.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a2100 0x7fb1ac13e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient(hunting): picked mon.noname-d con 0x7fb1ac0a2100 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac0751e0 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac136530 con 0x7fb1ac0a18a0 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac135bd0 con 0x7fb1ac0a2100 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1b0b69700 10 monclient(hunting): authenticate will time out at 2156.325683s 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1a9d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1ac0a1c80 0x7fb1ac13ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33564/0 (socket says 172.21.15.47:33564) 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a2100 0x7fb1ac13e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1a9d9b700 1 -- 172.21.15.47:0/3657654046 learned_addr learned my addr 172.21.15.47:0/3657654046 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): get_auth_request con 0x7fb1ac0a2100 auth_method 0 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.924+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 1 -- 172.21.15.47:0/3657654046 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676212482 0 0) 0x7fb1ac0751e0 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient(hunting): _negotiate 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient(hunting): my global_id is 35793 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 1 -- 172.21.15.47:0/3657654046 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1940035b0 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_done global_id 54474 payload 274 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 1 -- 172.21.15.47:0/3657654046 >> v1:172.21.15.179:6789/0 conn(0x7fb1ac0a18a0 legacy=0x7fb1ac135e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 1 -- 172.21.15.47:0/3657654046 >> v1:172.21.15.47:6789/0 conn(0x7fb1ac0a1c80 legacy=0x7fb1ac13ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 1 monclient: found mon.noname-d 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 1 -- 172.21.15.47:0/3657654046 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac071eb0 con 0x7fb1ac0a2100 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1aad9d700 1 --2- 172.21.15.47:0/3657654046 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a2100 0x7fb1ac13e260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009dc0 tx=0x7fb1a400b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=468e223c209be850 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 1 -- 172.21.15.47:0/3657654046 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a400e070 con 0x7fb1ac0a2100 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: dump: 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.938 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.939 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 1 -- 172.21.15.47:0/3657654046 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a4012430 con 0x7fb1ac0a2100 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1b0b69700 5 monclient: authenticate success, global_id 54474 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1b0b69700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 1 -- 172.21.15.47:0/3657654046 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a4012a10 con 0x7fb1ac0a2100 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1b0b69700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1b0b69700 4 set_mon_vals no callback set 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.925+0000 7fb1a959a700 10 monclient: dump: 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.941 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 10 monclient: shutdown 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 -- 172.21.15.47:0/3657654046 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a2100 msgr2=0x7fb1ac13e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 --2- 172.21.15.47:0/3657654046 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a2100 0x7fb1ac13e260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009dc0 tx=0x7fb1a400b330 comp rx=0 tx=0).stop 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 -- 172.21.15.47:0/3657654046 shutdown_connections 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 --2- 172.21.15.47:0/3657654046 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a2100 0x7fb1ac13e260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 -- 172.21.15.47:0/3657654046 >> 172.21.15.47:0/3657654046 conn(0x7fb1ac0a50b0 msgr2=0x7fb1ac066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 -- 172.21.15.47:0/3657654046 shutdown_connections 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.926+0000 7fb1b0b69700 1 -- 172.21.15.47:0/3657654046 wait complete. 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient: build_initial_monmap 2024-04-20T02:24:36.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient: monmap: 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:36.943 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 librados: starting msgr at 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 librados: starting objecter 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 Processor -- start 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 -- start start 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 librados: setting wanted keys 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 librados: calling monclient init 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient: init 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient: _add_conns ranks=[0,6,8,5,3,2,1,7,4] 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1ac0a18a0 0x7fb1ac1c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): picked mon.noname-a con 0x7fb1ac0a18a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a1c80 0x7fb1ac1c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): picked mon.noname-d con 0x7fb1ac0a1c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac0a2100 0x7fb1ac1cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): picked mon.noname-e con 0x7fb1ac0a2100 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac124240 con 0x7fb1ac0a18a0 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac123f60 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac135440 con 0x7fb1ac0a2100 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.927+0000 7fb1b0b69700 10 monclient(hunting): authenticate will time out at 2156.328997s 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1ac0a18a0 0x7fb1ac1c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac0a2100 0x7fb1ac1cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1ac0a18a0 0x7fb1ac1c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40902/0 (socket says 172.21.15.47:40902) 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a1c80 0x7fb1ac1c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 1 -- 172.21.15.47:0/702919459 learned_addr learned my addr 172.21.15.47:0/702919459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac0a2100 0x7fb1ac1cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55756/0 (socket says 172.21.15.47:55756) 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): get_auth_request con 0x7fb1ac0a18a0 auth_method 0 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): get_auth_request con 0x7fb1ac0a2100 auth_method 0 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): get_auth_request con 0x7fb1ac0a1c80 auth_method 0 2024-04-20T02:24:36.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient(hunting): handle_auth_done global_id 54483 payload 824 2024-04-20T02:24:36.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac0a2100 msgr2=0x7fb1ac1cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac0a2100 0x7fb1ac1cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1ac0a18a0 msgr2=0x7fb1ac1c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1ac0a18a0 0x7fb1ac1c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 monclient: found mon.noname-d 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009a70 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aad9d700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac0a2100 0x7fb1ac1cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1aa59c700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1ac0a18a0 0x7fb1ac1c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1a9d9b700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a1c80 0x7fb1ac1c9390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0000f60 tx=0x7fb1a000c740 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 client.?.objecter ms_handle_connect 0x7fb1ac0a1c80 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:36.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a000ce10 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 monclient: dump: 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.949 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 monclient: _finish_auth 0 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb1b0b69700 5 monclient: authenticate success, global_id 54483 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.928+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a00070d0 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 monclient: _renew_subs 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac06e780 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb19affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1a00076b0 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb19affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1a8d99700 4 set_mon_vals no callback set 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb19affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 client.54483.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb19affd700 10 monclient: dump: 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 monclient: _renew_subs 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac06e270 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 1 librados: init done 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 0x7fb1ac1cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac068430 con 0x7fb1ac06aac0 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1aa59c700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 0x7fb1ac1cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1aa59c700 10 monclient: get_auth_request con 0x7fb1ac06aac0 auth_method 0 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.929+0000 7fb1aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb1a0007240 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_done global_id 35306 payload 824 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aa59c700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1ac072470 con 0x7fb1ac06aac0 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb18c03a2f0 0x7fb18c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fb1a00495e0 con 0x7fb1ac0a1c80 2024-04-20T02:24:36.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 client.54483.objecter ms_dispatch 0x7fb1ac001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 3 client.54483.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 3 client.54483.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aa59c700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 0x7fb1ac1cc6e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c00ff20 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 20 client.54483.objecter dump_active .. 0 homeless 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 client.54483.objecter ms_handle_connect 0x7fb1ac06aac0 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 client.54483.objecter resend_mon_ops 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb19c0052e0 con 0x7fb1ac06aac0 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 monclient: dump: 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.954 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aad9d700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb18c03a2f0 0x7fb18c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb1aad9d700 10 monclient: get_auth_request con 0x7fb18c03a2f0 auth_method 0 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb19c005440 con 0x7fb1ac06aac0 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 msgr2=0x7fb1ac1cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 0x7fb1ac1cc6e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c00ff20 comp rx=0 tx=0).stop 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 msgr2=0x7fb1ac1cc6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.930+0000 7fb19affd700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac06aac0 0x7fb1ac1cc6e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.931+0000 7fb19affd700 10 client.54483.objecter ms_handle_connect 0x7fb18c03a2f0 2024-04-20T02:24:36.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.931+0000 7fb1aad9d700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb18c03a2f0 0x7fb18c03c7b0 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009dc0 tx=0x7fb1a4011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1b0b69700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1b0b69700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 0x7fb1ac1ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1b0b69700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac06a400 con 0x7fb1ac1c98d0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 0x7fb1ac1ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient: get_auth_request con 0x7fb1ac1c98d0 auth_method 0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 10 monclient(hunting): handle_auth_done global_id 35315 payload 824 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 1 -- 172.21.15.47:0/702919459 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb1ac070bb0 con 0x7fb1ac1c98d0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb19affd700 10 client.54483.objecter ms_handle_connect 0x7fb1ac1c98d0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb19affd700 10 client.54483.objecter resend_mon_ops 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb1aa59c700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 0x7fb1ac1ccc20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c001120 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb19c022070 con 0x7fb1ac1c98d0 2024-04-20T02:24:36.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb19affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb19affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.939+0000 7fb19affd700 10 monclient: dump: 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fb19c01d070 con 0x7fb1ac1c98d0 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 msgr2=0x7fb1ac1ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 0x7fb1ac1ccc20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c001120 comp rx=0 tx=0).stop 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 msgr2=0x7fb1ac1ccc20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb19affd700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb1ac1c98d0 0x7fb1ac1ccc20 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c001120 comp rx=0 tx=0).stop 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:36.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:36.962 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.963 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.964 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:36.965 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:36.966 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:36.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:36.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:36.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:36.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:36.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:36.983 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 10 librados: watch_flush enter 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 10 librados: watch_flush exit 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 20 client.54483.objecter shutdown clearing up homeless session... 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 10 client.54483.objecter successfully canceled tick 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb18c03a2f0 msgr2=0x7fb18c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb18c03a2f0 0x7fb18c03c7b0 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009dc0 tx=0x7fb1a4011000 comp rx=0 tx=0).stop 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 10 monclient: shutdown 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a1c80 msgr2=0x7fb1ac1c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:36.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 --2- 172.21.15.47:0/702919459 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb1ac0a1c80 0x7fb1ac1c9390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0000f60 tx=0x7fb1a000c740 comp rx=0 tx=0).stop 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1aad9d700 1 -- 172.21.15.47:0/702919459 reap_dead start 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1aad9d700 1 -- 172.21.15.47:0/702919459 reap_dead start 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 shutdown_connections 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 >> 172.21.15.47:0/702919459 conn(0x7fb1ac0a50b0 msgr2=0x7fb1ac1bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 shutdown_connections 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 -- 172.21.15.47:0/702919459 wait complete. 2024-04-20T02:24:36.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:36.940+0000 7fb1b0b69700 1 librados: shutdown 2024-04-20T02:24:36.986 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:36.986 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:24:36.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.126+0000 7fe95dd07700 10 monclient: get_monmap_and_config 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.127+0000 7fe95dd07700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.127+0000 7fe95dd07700 10 monclient: monmap: 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:37.128182+0000 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:37.128182+0000 2024-04-20T02:24:37.127 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.128 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.129 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.129 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.129 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.129 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:37.129 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:37.129 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.127+0000 7fe95dd07700 1 Processor -- start 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.127+0000 7fe95dd07700 1 -- start start 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.127+0000 7fe95dd07700 10 monclient: init 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient: _add_conns ranks=[4,2,8,0,3,1,5,9,6,10,7] 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 0x7fe9580a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): picked mon.noname-c con 0x7fe9580a5300 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a56e0 0x7fe9580a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): picked mon.noname-b con 0x7fe9580a56e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe958137ab0 0x7fe958137f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): picked mon.noname-e con 0x7fe958137ab0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580a3140 con 0x7fe9580a56e0 2024-04-20T02:24:37.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580a1bf0 con 0x7fe9580a5300 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580a1fd0 con 0x7fe958137ab0 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe95dd07700 10 monclient(hunting): authenticate will time out at 2156.529456s 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a56e0 0x7fe9580a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 0x7fe9580a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a56e0 0x7fe9580a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59346/0 (socket says 172.21.15.47:59346) 2024-04-20T02:24:37.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 0x7fe9580a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48524/0 (socket says 172.21.15.47:48524) 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 1 -- 172.21.15.47:0/1429772084 learned_addr learned my addr 172.21.15.47:0/1429772084 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): get_auth_request con 0x7fe9580a56e0 auth_method 0 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): get_auth_request con 0x7fe9580a5300 auth_method 0 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe958137ab0 0x7fe958137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): get_auth_request con 0x7fe958137ab0 auth_method 0 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe957fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 10 monclient(hunting): handle_auth_done global_id 54499 payload 274 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 1 -- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe958137ab0 msgr2=0x7fe958137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe958137ab0 0x7fe958137f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 1 -- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a56e0 msgr2=0x7fe9580a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.128+0000 7fe9577fe700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a56e0 0x7fe9580a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9577fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9577fe700 1 monclient: found mon.noname-c 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9577fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9577fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9577fe700 1 -- 172.21.15.47:0/1429772084 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580751e0 con 0x7fe9580a5300 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9577fe700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 0x7fe9580a3f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe948009870 tx=0x7fe948009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7407dfcc11cdd43b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 1 -- 172.21.15.47:0/1429772084 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe948011aa0 con 0x7fe9580a5300 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: dump: 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.146 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe95dd07700 5 monclient: authenticate success, global_id 54499 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 1 -- 172.21.15.47:0/1429772084 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948011c00 con 0x7fe9580a5300 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe95dd07700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 1 -- 172.21.15.47:0/1429772084 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe948011d70 con 0x7fe9580a5300 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe95dd07700 4 set_mon_vals no callback set 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.129+0000 7fe9567fc700 10 monclient: dump: 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.148 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 10 monclient: shutdown 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 -- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 msgr2=0x7fe9580a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 0x7fe9580a3f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe948009870 tx=0x7fe948009b80 comp rx=0 tx=0).stop 2024-04-20T02:24:37.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 -- 172.21.15.47:0/1429772084 shutdown_connections 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe958137ab0 0x7fe958137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe9580a5300 0x7fe9580a3f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 --2- 172.21.15.47:0/1429772084 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a56e0 0x7fe9580a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 -- 172.21.15.47:0/1429772084 >> 172.21.15.47:0/1429772084 conn(0x7fe9580aa710 msgr2=0x7fe9580aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 -- 172.21.15.47:0/1429772084 shutdown_connections 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 1 -- 172.21.15.47:0/1429772084 wait complete. 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.130+0000 7fe95dd07700 10 monclient: monmap: 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:37.150 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 librados: starting msgr at 2024-04-20T02:24:37.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 librados: starting objecter 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 Processor -- start 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 -- start start 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 librados: setting wanted keys 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 librados: calling monclient init 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient: init 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient: _add_conns ranks=[2,8,0,7,3,6,4,1,5] 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a5300 0x7fe9581cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): picked mon.noname-b con 0x7fe9580a5300 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe9580a56e0 0x7fe958136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): picked mon.noname-e con 0x7fe9580a56e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe958137ab0 0x7fe958137300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): picked mon.noname-a con 0x7fe958137ab0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe958123ee0 con 0x7fe958137ab0 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe958122e20 con 0x7fe9580a5300 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580a2220 con 0x7fe9580a56e0 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.131+0000 7fe95dd07700 10 monclient(hunting): authenticate will time out at 2156.532932s 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a5300 0x7fe9581cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a5300 0x7fe9581cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59350/0 (socket says 172.21.15.47:59350) 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 -- 172.21.15.47:0/3093834152 learned_addr learned my addr 172.21.15.47:0/3093834152 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): get_auth_request con 0x7fe9580a5300 auth_method 0 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe9580a56e0 0x7fe958136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): get_auth_request con 0x7fe9580a56e0 auth_method 0 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe958137ab0 0x7fe958137300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): get_auth_request con 0x7fe958137ab0 auth_method 0 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe957fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient(hunting): handle_auth_done global_id 24149 payload 824 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe9580a56e0 msgr2=0x7fe958136dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe9580a56e0 0x7fe958136dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe958137ab0 msgr2=0x7fe958137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe958137ab0 0x7fe958137300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 monclient: found mon.noname-b 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480094a0 con 0x7fe9580a5300 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe9577fe700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a5300 0x7fe9581cc430 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe9480090b0 tx=0x7fe948004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 client.?.objecter ms_handle_connect 0x7fe9580a5300 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9480036a0 con 0x7fe9580a5300 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: dump: 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.156 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94801f070 con 0x7fe9580a5300 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 5 monclient: authenticate success, global_id 24149 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 10 monclient: _renew_subs 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9580a45e0 con 0x7fe9580a5300 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe948003c90 con 0x7fe9580a5300 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe93ffff700 10 monclient: dump: 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.159 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 10 client.24149.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe955ffb700 4 set_mon_vals no callback set 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 10 monclient: _renew_subs 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe958124780 con 0x7fe9580a5300 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.132+0000 7fe95dd07700 1 librados: init done 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.133+0000 7fe95dd07700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.133+0000 7fe95dd07700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 0x7fe958120940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.133+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.133+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe958068230 con 0x7fe95806aaa0 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.133+0000 7fe956ffd700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 0x7fe958120940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient: get_auth_request con 0x7fe95806aaa0 auth_method 0 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe948021030 con 0x7fe9580a5300 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe94403a320 0x7fe94403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fe94804b930 con 0x7fe9580a5300 2024-04-20T02:24:37.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 10 client.24149.objecter ms_dispatch 0x7fe958001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 3 client.24149.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 3 client.24149.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe93ffff700 20 client.24149.objecter dump_active .. 0 homeless 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 10 monclient(hunting): handle_auth_done global_id 54667 payload 824 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe957fff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe94403a320 0x7fe94403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe956ffd700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe958067f00 con 0x7fe95806aaa0 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.134+0000 7fe957fff700 10 monclient: get_auth_request con 0x7fe94403a320 auth_method 0 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 client.24149.objecter ms_handle_connect 0x7fe94403a320 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe957fff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe94403a320 0x7fe94403c7e0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7fe9581236a0 tx=0x7fe95000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 client.24149.objecter ms_handle_connect 0x7fe95806aaa0 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 client.24149.objecter resend_mon_ops 2024-04-20T02:24:37.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe956ffd700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 0x7fe958120940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe958121f50 tx=0x7fe94c011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe94c00fd00 con 0x7fe95806aaa0 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 monclient: dump: 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.163 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe94c00fe60 con 0x7fe95806aaa0 2024-04-20T02:24:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 msgr2=0x7fe958120940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 0x7fe958120940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe958121f50 tx=0x7fe94c011d30 comp rx=0 tx=0).stop 2024-04-20T02:24:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 msgr2=0x7fe958120940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.135+0000 7fe93ffff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95806aaa0 0x7fe958120940 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe958121f50 tx=0x7fe94c011d30 comp rx=0 tx=0).stop 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe95dd07700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe95dd07700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 0x7fe958121290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe95dd07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806a400 con 0x7fe958120e80 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe956ffd700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 0x7fe958121290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe956ffd700 10 monclient: get_auth_request con 0x7fe958120e80 auth_method 0 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe956ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe956ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.143+0000 7fe956ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe956ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe956ffd700 10 monclient(hunting): handle_auth_done global_id 54676 payload 824 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe956ffd700 1 -- 172.21.15.47:0/3093834152 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe9581217d0 con 0x7fe958120e80 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe93ffff700 10 client.24149.objecter ms_handle_connect 0x7fe958120e80 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe93ffff700 10 client.24149.objecter resend_mon_ops 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe956ffd700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 0x7fe958121290 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006b20 tx=0x7fe94c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe94c00fb80 con 0x7fe958120e80 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe93ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe93ffff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:37.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.144+0000 7fe93ffff700 10 monclient: dump: 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.169 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fe94c02c430 con 0x7fe958120e80 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 msgr2=0x7fe958121290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 0x7fe958121290 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006b20 tx=0x7fe94c004750 comp rx=0 tx=0).stop 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 msgr2=0x7fe958121290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe93ffff700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe958120e80 0x7fe958121290 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006b20 tx=0x7fe94c004750 comp rx=0 tx=0).stop 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:37.170 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:37.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.172 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.173 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:37.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.179 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:37.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.184 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:37.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:37.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:37.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:37.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.193 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:37.194 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 10 librados: watch_flush enter 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 10 librados: watch_flush exit 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 20 client.24149.objecter shutdown clearing up homeless session... 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 10 client.24149.objecter successfully canceled tick 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe94403a320 msgr2=0x7fe94403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe94403a320 0x7fe94403c7e0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7fe9581236a0 tx=0x7fe95000b040 comp rx=0 tx=0).stop 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 10 monclient: shutdown 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a5300 msgr2=0x7fe9581cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 --2- 172.21.15.47:0/3093834152 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe9580a5300 0x7fe9581cc430 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe9480090b0 tx=0x7fe948004750 comp rx=0 tx=0).stop 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe957fff700 1 -- 172.21.15.47:0/3093834152 reap_dead start 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe957fff700 1 -- 172.21.15.47:0/3093834152 reap_dead start 2024-04-20T02:24:37.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 shutdown_connections 2024-04-20T02:24:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 >> 172.21.15.47:0/3093834152 conn(0x7fe9580aa710 msgr2=0x7fe9580a9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 shutdown_connections 2024-04-20T02:24:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 -- 172.21.15.47:0/3093834152 wait complete. 2024-04-20T02:24:37.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.145+0000 7fe95dd07700 1 librados: shutdown 2024-04-20T02:24:37.197 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:37.197 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:24:37.197 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.336+0000 7f54d6883700 10 monclient: get_monmap_and_config 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.336+0000 7f54d6883700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.336+0000 7f54d6883700 10 monclient: monmap: 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:37.337207+0000 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:37.337207+0000 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.336 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.338 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:37.338 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:37.338 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.336+0000 7f54d6883700 1 Processor -- start 2024-04-20T02:24:37.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.336+0000 7f54d6883700 1 -- start start 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.336+0000 7f54d6883700 10 monclient: init 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient: _add_conns ranks=[3,7,10,0,8,6,1,9,4,5,2] 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d00a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): picked mon.noname-g con 0x7f54d00a1510 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f54d009f9a0 0x7f54d00a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): picked mon.noname-i con 0x7f54d009f9a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f54d009fe00 0x7f54d00a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): picked mon.noname-f con 0x7f54d009fe00 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d00720a0 con 0x7f54d009fe00 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0070640 con 0x7f54d00a1510 2024-04-20T02:24:37.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0071d30 con 0x7f54d009f9a0 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d6883700 10 monclient(hunting): authenticate will time out at 2156.738710s 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f54d009fe00 0x7f54d00a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54cffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d00a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f54d009fe00 0x7f54d00a3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:35920/0 (socket says 172.21.15.47:35920) 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 1 -- 172.21.15.47:0/911177386 learned_addr learned my addr 172.21.15.47:0/911177386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 10 monclient(hunting): get_auth_request con 0x7f54d009fe00 auth_method 0 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54d4e20700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54cffff700 10 monclient(hunting): get_auth_request con 0x7f54d00a1510 auth_method 0 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54cffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54cffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.337+0000 7f54cffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cf7fe700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f54d009f9a0 0x7f54d00a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cf7fe700 10 monclient(hunting): get_auth_request con 0x7f54d009f9a0 auth_method 0 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cf7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cf7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 10 monclient(hunting): handle_auth_done global_id 54685 payload 274 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 -- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f54d009f9a0 msgr2=0x7f54d00a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f54d009f9a0 0x7f54d00a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 -- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 msgr2=0x7f54d00a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d00a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 monclient: found mon.noname-f 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 -- 172.21.15.47:0/911177386 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d00751e0 con 0x7f54d009fe00 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54cffff700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d00a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d4e20700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54d009fe00 0x7f54d00a3e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009e50 tx=0x7f54c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19607d3eb5fbcce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 1 -- 172.21.15.47:0/911177386 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54c800e070 con 0x7f54d009fe00 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: dump: 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d6883700 5 monclient: authenticate success, global_id 54685 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 1 -- 172.21.15.47:0/911177386 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c800bd80 con 0x7f54d009fe00 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 1 -- 172.21.15.47:0/911177386 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54c8012860 con 0x7f54d009fe00 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d6883700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54d6883700 4 set_mon_vals no callback set 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.338+0000 7f54ceffd700 10 monclient: dump: 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.357 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 10 monclient: shutdown 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 -- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54d009fe00 msgr2=0x7f54d00a3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54d009fe00 0x7f54d00a3e30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009e50 tx=0x7f54c800b330 comp rx=0 tx=0).stop 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 -- 172.21.15.47:0/911177386 shutdown_connections 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54d009fe00 0x7f54d00a3e30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d00a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 --2- 172.21.15.47:0/911177386 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f54d009f9a0 0x7f54d00a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 -- 172.21.15.47:0/911177386 >> 172.21.15.47:0/911177386 conn(0x7f54d00a8510 msgr2=0x7f54d00a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 -- 172.21.15.47:0/911177386 shutdown_connections 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.339+0000 7f54d6883700 1 -- 172.21.15.47:0/911177386 wait complete. 2024-04-20T02:24:37.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.340+0000 7f54d6883700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.340+0000 7f54d6883700 10 monclient: monmap: 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.359 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.340+0000 7f54d6883700 1 librados: starting msgr at 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.340+0000 7f54d6883700 1 librados: starting objecter 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.340+0000 7f54d6883700 1 Processor -- start 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 -- start start 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 librados: setting wanted keys 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 librados: calling monclient init 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient: init 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient: _add_conns ranks=[6,0,3,4,1,8,2,5,7] 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f54d009f9a0 0x7f54d01c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): picked mon.noname-d con 0x7f54d009f9a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54d009fe00 0x7f54d0135a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): picked mon.noname-a con 0x7f54d009fe00 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d0135f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): picked mon.noname-g con 0x7f54d00a1510 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0121a90 con 0x7f54d009fe00 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0123480 con 0x7f54d00a1510 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0071040 con 0x7f54d009f9a0 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54d6883700 10 monclient(hunting): authenticate will time out at 2156.742806s 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54d009fe00 0x7f54d0135a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f54d009f9a0 0x7f54d01c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54d009fe00 0x7f54d0135a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40918/0 (socket says 172.21.15.47:40918) 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f54d009f9a0 0x7f54d01c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51464/0 (socket says 172.21.15.47:51464) 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 1 -- 172.21.15.47:0/1175385549 learned_addr learned my addr 172.21.15.47:0/1175385549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 10 monclient(hunting): get_auth_request con 0x7f54d009fe00 auth_method 0 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.341+0000 7f54cffff700 10 monclient(hunting): get_auth_request con 0x7f54d009f9a0 auth_method 0 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d0135f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): get_auth_request con 0x7f54d00a1510 auth_method 0 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 10 monclient(hunting): handle_auth_done global_id 35820 payload 824 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f54d009f9a0 msgr2=0x7f54d01c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f54d009f9a0 0x7f54d01c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 msgr2=0x7f54d0135f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d0135f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 monclient: found mon.noname-a 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00094a0 con 0x7f54d009fe00 2024-04-20T02:24:37.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cffff700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f54d009f9a0 0x7f54d01c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d4e20700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d00a1510 0x7f54d0135f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54cf7fe700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54d009fe00 0x7f54d0135a30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e810 tx=0x7f54b800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 client.?.objecter ms_handle_connect 0x7f54d009fe00 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54b800c070 con 0x7f54d009fe00 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: dump: 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.365 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 5 monclient: authenticate success, global_id 35820 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8013880 con 0x7f54d009fe00 2024-04-20T02:24:37.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 10 monclient: _renew_subs 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54b8013e60 con 0x7f54d009fe00 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c8009b30 con 0x7f54d009fe00 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ce7fc700 4 set_mon_vals no callback set 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54ccff9700 10 monclient: dump: 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.367 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 10 client.35820.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:37.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 10 monclient: _renew_subs 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d0121090 con 0x7f54d009fe00 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.342+0000 7f54d6883700 1 librados: init done 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f54b80139f0 con 0x7f54d009fe00 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54d6883700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54d6883700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 0x7f54d011f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d00682f0 con 0x7f54d006aac0 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54ccff9700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f54b003a2f0 0x7f54b003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f54b804a3a0 con 0x7f54d009fe00 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54ccff9700 10 client.35820.objecter ms_dispatch 0x7f54d0001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54ccff9700 3 client.35820.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.343+0000 7f54ccff9700 3 client.35820.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54ccff9700 20 client.35820.objecter dump_active .. 0 homeless 2024-04-20T02:24:37.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 0x7f54d011f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient: get_auth_request con 0x7f54d006aac0 auth_method 0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54d4e20700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f54b003a2f0 0x7f54b003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54d4e20700 10 monclient: get_auth_request con 0x7f54b003a2f0 auth_method 0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.344+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54cffff700 10 monclient(hunting): handle_auth_done global_id 22305 payload 824 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54cffff700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54d00a1e30 con 0x7f54d006aac0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 client.35820.objecter ms_handle_connect 0x7f54d006aac0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54cffff700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 0x7f54d011f410 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009f60 tx=0x7f54c0009f90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 client.35820.objecter resend_mon_ops 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 client.35820.objecter ms_handle_connect 0x7f54b003a2f0 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54d4e20700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f54b003a2f0 0x7f54b003c7b0 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 crypto rx=0x7f54c800b370 tx=0x7f54c8009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54c000fe50 con 0x7f54d006aac0 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 monclient: dump: 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.371 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f54c0004030 con 0x7f54d006aac0 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 msgr2=0x7f54d011f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 0x7f54d011f410 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009f60 tx=0x7f54c0009f90 comp rx=0 tx=0).stop 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 msgr2=0x7f54d011f410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.345+0000 7f54ccff9700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d006aac0 0x7f54d011f410 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54d6883700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54d6883700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 0x7f54d011fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54d6883700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d006a400 con 0x7f54d011f950 2024-04-20T02:24:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 0x7f54d011fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient: get_auth_request con 0x7f54d011f950 auth_method 0 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.353+0000 7f54cffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54cffff700 10 monclient(hunting): handle_auth_done global_id 22314 payload 824 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54cffff700 1 -- 172.21.15.47:0/1175385549 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f54d0070960 con 0x7f54d011f950 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54cffff700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 0x7f54d011fd60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009f60 tx=0x7f54c0004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54ccff9700 10 client.35820.objecter ms_handle_connect 0x7f54d011f950 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54ccff9700 10 client.35820.objecter resend_mon_ops 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54c001ec60 con 0x7f54d011f950 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54ccff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54ccff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.354+0000 7f54ccff9700 10 monclient: dump: 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.376 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f54c001e7f0 con 0x7f54d011f950 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 msgr2=0x7f54d011fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 0x7f54d011fd60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009f60 tx=0x7f54c0004750 comp rx=0 tx=0).stop 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 msgr2=0x7f54d011fd60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54ccff9700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f54d011f950 0x7f54d011fd60 secure :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009f60 tx=0x7f54c0004750 comp rx=0 tx=0).stop 2024-04-20T02:24:37.377 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:37.378 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.379 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.380 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.381 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.383 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:37.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:37.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.389 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:37.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.393 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:37.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.398 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.401 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:37.402 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 10 librados: watch_flush enter 2024-04-20T02:24:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 10 librados: watch_flush exit 2024-04-20T02:24:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 20 client.35820.objecter shutdown clearing up homeless session... 2024-04-20T02:24:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 10 client.35820.objecter successfully canceled tick 2024-04-20T02:24:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f54b003a2f0 msgr2=0x7f54b003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f54b003a2f0 0x7f54b003c7b0 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 crypto rx=0x7f54c800b370 tx=0x7f54c8009000 comp rx=0 tx=0).stop 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 10 monclient: shutdown 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54d009fe00 msgr2=0x7f54d0135a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 --2- 172.21.15.47:0/1175385549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54d009fe00 0x7f54d0135a30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e810 tx=0x7f54b800eb20 comp rx=0 tx=0).stop 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d4e20700 1 -- 172.21.15.47:0/1175385549 reap_dead start 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d4e20700 1 -- 172.21.15.47:0/1175385549 reap_dead start 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 shutdown_connections 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 >> 172.21.15.47:0/1175385549 conn(0x7f54d00a8510 msgr2=0x7f54d01bd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 shutdown_connections 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 -- 172.21.15.47:0/1175385549 wait complete. 2024-04-20T02:24:37.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.355+0000 7f54d6883700 1 librados: shutdown 2024-04-20T02:24:37.406 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:37.406 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:24:37.406 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.544+0000 7f1febff3700 10 monclient: get_monmap_and_config 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.544+0000 7f1febff3700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.544+0000 7f1febff3700 10 monclient: monmap: 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:37.545935+0000 2024-04-20T02:24:37.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:37.545935+0000 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:37.547 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.545+0000 7f1febff3700 1 Processor -- start 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.545+0000 7f1febff3700 1 -- start start 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.545+0000 7f1febff3700 10 monclient: init 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient: _add_conns ranks=[7,1,0,6,9,8,3,2,4,5,10] 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fe40a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient(hunting): picked mon.noname-i con 0x7f1fe4137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1fe4138170 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1fe41385b0 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:37.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe40751e0 con 0x7f1fe41385b0 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe409dfc0 con 0x7f1fe4138170 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe409fa40 con 0x7f1fe4137d90 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1febff3700 10 monclient(hunting): authenticate will time out at 2156.947254s 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fea590700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1fe41385b0 0x7f1fe40a63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33574/0 (socket says 172.21.15.47:33574) 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fea590700 1 -- 172.21.15.47:0/2807435934 learned_addr learned my addr 172.21.15.47:0/2807435934 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548836606 0 0) 0x7f1fe40751e0 con 0x7f1fe41385b0 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient(hunting): _negotiate 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient(hunting): my global_id is 35829 2024-04-20T02:24:37.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fe40a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd00035b0 con 0x7f1fe41385b0 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request con 0x7f1fe4137d90 auth_method 0 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447862463 0 0) 0x7f1fd00035b0 con 0x7f1fe41385b0 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 msgr2=0x7f1fe40a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fe40a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 >> v1:172.21.15.179:6789/0 conn(0x7f1fe4138170 legacy=0x7f1fe40a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40a6ab0 con 0x7f1fe41385b0 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f1fdc001c60 con 0x7f1fe41385b0 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.546+0000 7f1fe8d8d700 10 monclient: dump: 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.568 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:24:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 >> v1:172.21.15.47:6789/0 conn(0x7f1fe41385b0 legacy=0x7f1fe40a63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient: _add_conns ranks=[2,7,6,5,0,1,4,3,8] 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fd0004280 0x7f1fd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): picked mon.b con 0x7f1fd0004280 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fd0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): picked mon.i con 0x7f1fe4137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1fd0009a00 0x7f1fd000bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): picked mon.d con 0x7f1fd0009a00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe409fa40 con 0x7f1fd0004280 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c430 con 0x7f1fd0009a00 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c5a0 con 0x7f1fe4137d90 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe8d8d700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fd0004280 0x7f1fd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1fd0009a00 0x7f1fd000bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fd0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): get_auth_request con 0x7f1fd0009a00 auth_method 0 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): get_auth_request con 0x7f1fe4137d90 auth_method 0 2024-04-20T02:24:37.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request con 0x7f1fd0004280 auth_method 0 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:37.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient(hunting): handle_auth_done global_id 35829 payload 293 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 -- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1fd0009a00 msgr2=0x7f1fd000bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1fd0009a00 0x7f1fd000bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 -- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fd0004280 msgr2=0x7f1fd0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fd0004280 0x7f1fd0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 monclient: found mon.i 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 -- 172.21.15.47:0/2807435934 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd00035b0 con 0x7f1fe4137d90 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1febff3700 5 monclient: authenticate success, global_id 35829 2024-04-20T02:24:37.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1febff3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fea590700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f1fd0009a00 0x7f1fd000bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.547+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fd0004e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fe409e9b0 tx=0x7f1fd0005880 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f9dd8ba79884e75a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fd8005540 con 0x7f1fe4137d90 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: dump: 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.574 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8001290 con 0x7f1fe4137d90 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 1 -- 172.21.15.47:0/2807435934 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fd8003af0 con 0x7f1fe4137d90 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 4 set_mon_vals no callback set 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fe8d8d700 10 monclient: dump: 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.576 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 10 monclient: shutdown 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 1 -- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 msgr2=0x7f1fd0004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 1 --2- 172.21.15.47:0/2807435934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1fe4137d90 0x7f1fd0004e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fe409e9b0 tx=0x7f1fd0005880 comp rx=0 tx=0).stop 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1fea590700 1 -- 172.21.15.47:0/2807435934 reap_dead start 2024-04-20T02:24:37.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 1 -- 172.21.15.47:0/2807435934 shutdown_connections 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 1 -- 172.21.15.47:0/2807435934 >> 172.21.15.47:0/2807435934 conn(0x7f1fe40aa8b0 msgr2=0x7f1fe40a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 1 -- 172.21.15.47:0/2807435934 shutdown_connections 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.548+0000 7f1febff3700 1 -- 172.21.15.47:0/2807435934 wait complete. 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.549+0000 7f1febff3700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.549+0000 7f1febff3700 10 monclient: monmap: 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.578 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.549+0000 7f1febff3700 1 librados: starting msgr at 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.549+0000 7f1febff3700 1 librados: starting objecter 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.549+0000 7f1febff3700 1 Processor -- start 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 -- start start 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 librados: setting wanted keys 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 librados: calling monclient init 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient: init 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient: _add_conns ranks=[5,0,2,8,3,6,7,1,4] 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe4136d10 0x7f1fe41cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): picked mon.noname-h con 0x7f1fe4136d10 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fe41370f0 0x7f1fe41208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): picked mon.noname-a con 0x7f1fe41370f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fe4137510 0x7f1fe4120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): picked mon.noname-b con 0x7f1fe4137510 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4123ec0 con 0x7f1fe41370f0 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41246d0 con 0x7f1fe4137510 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4122300 con 0x7f1fe4136d10 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1febff3700 10 monclient(hunting): authenticate will time out at 2156.951602s 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fe41370f0 0x7f1fe41208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fe4137510 0x7f1fe4120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fe41370f0 0x7f1fe41208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40926/0 (socket says 172.21.15.47:40926) 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 1 -- 172.21.15.47:0/2813400541 learned_addr learned my addr 172.21.15.47:0/2813400541 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fe4137510 0x7f1fe4120e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59370/0 (socket says 172.21.15.47:59370) 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): get_auth_request con 0x7f1fe41370f0 auth_method 0 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): get_auth_request con 0x7f1fe4137510 auth_method 0 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe4136d10 0x7f1fe41cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe958e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request con 0x7f1fe4136d10 auth_method 0 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fea590700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.550+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 10 monclient(hunting): handle_auth_done global_id 24158 payload 824 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe4136d10 msgr2=0x7f1fe41cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe4136d10 0x7f1fe41cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fe41370f0 msgr2=0x7f1fe41208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fe41370f0 0x7f1fe41208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 monclient: found mon.noname-b 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe409e9e0 con 0x7f1fe4137510 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fe41370f0 0x7f1fe41208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fea590700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fe4137510 0x7f1fe4120e10 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4138990 tx=0x7f1fdc007570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 client.?.objecter ms_handle_connect 0x7f1fe4137510 2024-04-20T02:24:37.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fdc001400 con 0x7f1fe4137510 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: dump: 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.584 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc013860 con 0x7f1fe4137510 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 5 monclient: authenticate success, global_id 24158 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 10 monclient: _renew_subs 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe409dc20 con 0x7f1fe4137510 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fdc013e40 con 0x7f1fe4137510 2024-04-20T02:24:37.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd67fc700 10 monclient: dump: 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.586 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1fd7fff700 4 set_mon_vals no callback set 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 10 client.24158.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 10 monclient: _renew_subs 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe41243f0 con 0x7f1fe4137510 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.551+0000 7f1febff3700 1 librados: init done 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1febff3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1febff3700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 0x7f1fe4121350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1febff3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4068180 con 0x7f1fe406aac0 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1fdc0139d0 con 0x7f1fe4137510 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 0x7f1fe4121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient: get_auth_request con 0x7f1fe406aac0 auth_method 0 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1fc803a2e0 0x7f1fc803c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1fdc04b2f0 con 0x7f1fe4137510 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 10 client.24158.objecter ms_dispatch 0x7f1fe4001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 3 client.24158.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 3 client.24158.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fd67fc700 20 client.24158.objecter dump_active .. 0 homeless 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.552+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1fc803a2e0 0x7f1fc803c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_done global_id 64211 payload 824 2024-04-20T02:24:37.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fe9d8f700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fe40a7bc0 con 0x7f1fe406aac0 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fe958e700 10 monclient: get_auth_request con 0x7f1fc803a2e0 auth_method 0 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 client.24158.objecter ms_handle_connect 0x7f1fe406aac0 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 client.24158.objecter resend_mon_ops 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 0x7f1fe4121350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014cf0 tx=0x7f1fe0014dd0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fe0012e20 con 0x7f1fe406aac0 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 monclient: dump: 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.589 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.590 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 client.24158.objecter ms_handle_connect 0x7f1fc803a2e0 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fe958e700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1fc803a2e0 0x7f1fc803c7a0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40a4570 tx=0x7f1fd8011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1fe00129b0 con 0x7f1fe406aac0 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 msgr2=0x7f1fe4121350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 0x7f1fe4121350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014cf0 tx=0x7f1fe0014dd0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 msgr2=0x7f1fe4121350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.553+0000 7f1fd67fc700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe406aac0 0x7f1fe4121350 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014cf0 tx=0x7f1fe0014dd0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.561+0000 7f1febff3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.561+0000 7f1febff3700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 0x7f1fe4061b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.561+0000 7f1febff3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.561+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe406a400 con 0x7f1fe402cd40 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 0x7f1fe4061b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient: get_auth_request con 0x7f1fe402cd40 auth_method 0 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 10 monclient(hunting): handle_auth_done global_id 64220 payload 824 2024-04-20T02:24:37.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 1 -- 172.21.15.47:0/2813400541 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1fe409c0d0 con 0x7f1fe402cd40 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fe9d8f700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 0x7f1fe4061b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014cf0 tx=0x7f1fe00047c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fd67fc700 10 client.24158.objecter ms_handle_connect 0x7f1fe402cd40 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fd67fc700 10 client.24158.objecter resend_mon_ops 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fe0025300 con 0x7f1fe402cd40 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fd67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fd67fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.562+0000 7f1fd67fc700 10 monclient: dump: 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.593 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f1fe0006720 con 0x7f1fe402cd40 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 msgr2=0x7f1fe4061b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 0x7f1fe4061b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014cf0 tx=0x7f1fe00047c0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 msgr2=0x7f1fe4061b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fd67fc700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f1fe402cd40 0x7f1fe4061b00 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0014cf0 tx=0x7f1fe00047c0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:37.595 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.596 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.597 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:37.598 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.599 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:37.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:37.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.603 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.608 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:37.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.613 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.615 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.616 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:37.617 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 10 librados: watch_flush enter 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 10 librados: watch_flush exit 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 20 client.24158.objecter shutdown clearing up homeless session... 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 10 client.24158.objecter successfully canceled tick 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1fc803a2e0 msgr2=0x7f1fc803c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1fc803a2e0 0x7f1fc803c7a0 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40a4570 tx=0x7f1fd8011000 comp rx=0 tx=0).stop 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 10 monclient: shutdown 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fe4137510 msgr2=0x7f1fe4120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 --2- 172.21.15.47:0/2813400541 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fe4137510 0x7f1fe4120e10 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4138990 tx=0x7f1fdc007570 comp rx=0 tx=0).stop 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fea590700 1 -- 172.21.15.47:0/2813400541 reap_dead start 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1fea590700 1 -- 172.21.15.47:0/2813400541 reap_dead start 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 shutdown_connections 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 >> 172.21.15.47:0/2813400541 conn(0x7f1fe40aa8b0 msgr2=0x7f1fe409d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 shutdown_connections 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 -- 172.21.15.47:0/2813400541 wait complete. 2024-04-20T02:24:37.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.563+0000 7f1febff3700 1 librados: shutdown 2024-04-20T02:24:37.620 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:37.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:24:37.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.759+0000 7f53bb497700 10 monclient: get_monmap_and_config 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.759+0000 7f53bb497700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.759+0000 7f53bb497700 10 monclient: monmap: 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:37.761032+0000 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:37.761032+0000 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:37.760 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:37.761 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.760+0000 7f53bb497700 1 Processor -- start 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.760+0000 7f53bb497700 1 -- start start 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.760+0000 7f53bb497700 10 monclient: init 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient: _add_conns ranks=[10,1,8,4,2,9,5,6,3,0,7] 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f53b406d160 0x7f53b406d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): picked mon.noname-f con 0x7f53b406d160 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f53b406c880 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 0x7f53b4138130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): picked mon.noname-e con 0x7f53b406cc80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40751e0 con 0x7f53b406c880 2024-04-20T02:24:37.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b409ee90 con 0x7f53b406d160 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f53b406e450 con 0x7f53b406cc80 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53bb497700 10 monclient(hunting): authenticate will time out at 2157.162407s 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 0x7f53b4138130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 0x7f53b4138130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55796/0 (socket says 172.21.15.47:55796) 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 -- 172.21.15.47:0/1377579184 learned_addr learned my addr 172.21.15.47:0/1377579184 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): get_auth_request con 0x7f53b406cc80 auth_method 0 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 1 --2- 172.21.15.47:0/1377579184 >> v2:172.21.15.179:3300/0 conn(0x7f53b406d160 0x7f53b406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): get_auth_request con 0x7f53b406d160 auth_method 0 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9233700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient(hunting): handle_auth_done global_id 35342 payload 274 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 -- 172.21.15.47:0/1377579184 >> v2:172.21.15.179:3300/0 conn(0x7f53b406d160 msgr2=0x7f53b406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:37.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 --2- 172.21.15.47:0/1377579184 >> v2:172.21.15.179:3300/0 conn(0x7f53b406d160 0x7f53b406d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 -- 172.21.15.47:0/1377579184 >> v1:172.21.15.179:6789/0 conn(0x7f53b406c880 legacy=0x7f53b4137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 monclient: found mon.noname-e 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.761+0000 7f53b9a34700 1 -- 172.21.15.47:0/1377579184 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4071e50 con 0x7f53b406cc80 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53b9a34700 1 --2- 172.21.15.47:0/1377579184 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 0x7f53b4138130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009e50 tx=0x7f53ac00b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9c72dc6c0001a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 1 -- 172.21.15.47:0/1377579184 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53ac00e070 con 0x7f53b406cc80 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: dump: 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.776 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 5 monclient: authenticate success, global_id 35342 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 1 -- 172.21.15.47:0/1377579184 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac0099b0 con 0x7f53b406cc80 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 4 set_mon_vals no callback set 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 1 -- 172.21.15.47:0/1377579184 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53ac00bd80 con 0x7f53b406cc80 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53a7fff700 10 monclient: dump: 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.778 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 10 monclient: shutdown 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 1 -- 172.21.15.47:0/1377579184 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 msgr2=0x7f53b4138130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.762+0000 7f53bb497700 1 --2- 172.21.15.47:0/1377579184 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 0x7f53b4138130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009e50 tx=0x7f53ac00b330 comp rx=0 tx=0).stop 2024-04-20T02:24:37.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 1 -- 172.21.15.47:0/1377579184 shutdown_connections 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 1 --2- 172.21.15.47:0/1377579184 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f53b406cc80 0x7f53b4138130 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 1 --2- 172.21.15.47:0/1377579184 >> v2:172.21.15.179:3300/0 conn(0x7f53b406d160 0x7f53b406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 1 -- 172.21.15.47:0/1377579184 >> 172.21.15.47:0/1377579184 conn(0x7f53b40a4a50 msgr2=0x7f53b40a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 1 -- 172.21.15.47:0/1377579184 shutdown_connections 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 1 -- 172.21.15.47:0/1377579184 wait complete. 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 10 monclient: build_initial_monmap 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.763+0000 7f53bb497700 10 monclient: monmap: 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:37.780 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 librados: starting msgr at 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 librados: starting objecter 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 Processor -- start 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 -- start start 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 librados: setting wanted keys 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 librados: calling monclient init 2024-04-20T02:24:37.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient: init 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient: _add_conns ranks=[0,1,7,4,3,8,2,5,6] 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b406c880 0x7f53b41ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): picked mon.noname-a con 0x7f53b406c880 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b406cc80 0x7f53b41cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): picked mon.noname-f con 0x7f53b406cc80 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406d160 0x7f53b4120ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): picked mon.noname-i con 0x7f53b406d160 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4072400 con 0x7f53b406c880 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4071fe0 con 0x7f53b406cc80 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41230e0 con 0x7f53b406d160 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:37.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53bb497700 10 monclient(hunting): authenticate will time out at 2157.165831s 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b406c880 0x7f53b41ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b406c880 0x7f53b41ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40938/0 (socket says 172.21.15.47:40938) 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 1 -- 172.21.15.47:0/3574014770 learned_addr learned my addr 172.21.15.47:0/3574014770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b8a32700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b406cc80 0x7f53b41cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 10 monclient(hunting): get_auth_request con 0x7f53b406c880 auth_method 0 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.764+0000 7f53b9233700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): get_auth_request con 0x7f53b406cc80 auth_method 0 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406d160 0x7f53b4120ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): get_auth_request con 0x7f53b406d160 auth_method 0 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9a34700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 10 monclient(hunting): handle_auth_done global_id 35847 payload 824 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406d160 msgr2=0x7f53b4120ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406d160 0x7f53b4120ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b406cc80 msgr2=0x7f53b41cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b406cc80 0x7f53b41cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 10 monclient: _finish_hunting 0 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 monclient: found mon.noname-a 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac009b30 con 0x7f53b406c880 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53b9233700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b406c880 0x7f53b41ccb50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f53b000e9b0 tx=0x7f53b000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 client.?.objecter ms_handle_connect 0x7f53b406c880 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53b0009710 con 0x7f53b406c880 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: dump: 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.785 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.786 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b000ee50 con 0x7f53b406c880 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53bb497700 5 monclient: authenticate success, global_id 35847 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53bb497700 10 monclient: _renew_subs 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53bb497700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b41237a0 con 0x7f53b406c880 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53b000ccc0 con 0x7f53b406c880 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a5ffb700 10 monclient: dump: 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.787 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.765+0000 7f53a77fe700 4 set_mon_vals no callback set 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 10 client.35847.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:37.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 10 monclient: _renew_subs 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b4124700 con 0x7f53b406c880 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 1 librados: init done 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 0x7f53b4121530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f53b000c850 con 0x7f53b406c880 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4068430 con 0x7f53b406aac0 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53b8a32700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 0x7f53b4121530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53a5ffb700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f539803a320 0x7f539803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53b8a32700 10 monclient: get_auth_request con 0x7f53b406aac0 auth_method 0 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.766+0000 7f53b8a32700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f53b004a150 con 0x7f53b406c880 2024-04-20T02:24:37.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 client.35847.objecter ms_dispatch 0x7f53b4001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 3 client.35847.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 3 client.35847.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 20 client.35847.objecter dump_active .. 0 homeless 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b9a34700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f539803a320 0x7f539803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b9a34700 10 monclient: get_auth_request con 0x7f539803a320 auth_method 0 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 10 monclient(hunting): handle_auth_done global_id 54484 payload 824 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53b406d540 con 0x7f53b406aac0 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 client.35847.objecter ms_handle_connect 0x7f53b406aac0 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 client.35847.objecter resend_mon_ops 2024-04-20T02:24:37.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b8a32700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 0x7f53b4121530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f53b4122750 tx=0x7f53a800fb00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 client.35847.objecter ms_handle_connect 0x7f539803a320 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53b9a34700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f539803a320 0x7f539803c7e0 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b370 tx=0x7f53ac009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53a80052e0 con 0x7f53b406aac0 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.767+0000 7f53a5ffb700 10 monclient: dump: 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.791 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f53a8005440 con 0x7f53b406aac0 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 msgr2=0x7f53b4121530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 0x7f53b4121530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f53b4122750 tx=0x7f53a800fb00 comp rx=0 tx=0).stop 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 msgr2=0x7f53b4121530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.768+0000 7f53a5ffb700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b406aac0 0x7f53b4121530 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f53b4122750 tx=0x7f53a800fb00 comp rx=0 tx=0).stop 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.775+0000 7f53bb497700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.775+0000 7f53bb497700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 0x7f53b4121e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.775+0000 7f53bb497700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.775+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53b406a400 con 0x7f53b4121a70 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 0x7f53b4121e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient: get_auth_request con 0x7f53b4121a70 auth_method 0 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:37.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 10 monclient(hunting): handle_auth_done global_id 54493 payload 824 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 1 -- 172.21.15.47:0/3574014770 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f53b406e170 con 0x7f53b4121a70 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53b8a32700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 0x7f53b4121e80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53a8017000 tx=0x7f53a8001080 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53a5ffb700 10 client.35847.objecter ms_handle_connect 0x7f53b4121a70 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.776+0000 7f53a5ffb700 10 client.35847.objecter resend_mon_ops 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53a8023070 con 0x7f53b4121a70 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 10 monclient: dump: 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.795 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f53a8003d20 con 0x7f53b4121a70 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:37.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 msgr2=0x7f53b4121e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 0x7f53b4121e80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53a8017000 tx=0x7f53a8001080 comp rx=0 tx=0).stop 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 msgr2=0x7f53b4121e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53a5ffb700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f53b4121a70 0x7f53b4121e80 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53a8017000 tx=0x7f53a8001080 comp rx=0 tx=0).stop 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 62, 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:37.797 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 8, 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.798 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:37.800 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:37.801 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.803 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:37.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.806 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:37.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.810 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:37.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.815 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:37.818 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:37.819 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:37.819 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53bb497700 10 librados: watch_flush enter 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53bb497700 10 librados: watch_flush exit 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53bb497700 20 client.35847.objecter shutdown clearing up homeless session... 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.777+0000 7f53bb497700 10 client.35847.objecter successfully canceled tick 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f539803a320 msgr2=0x7f539803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f539803a320 0x7f539803c7e0 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b370 tx=0x7f53ac009000 comp rx=0 tx=0).stop 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 10 monclient: shutdown 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b406c880 msgr2=0x7f53b41ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:37.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 --2- 172.21.15.47:0/3574014770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53b406c880 0x7f53b41ccb50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f53b000e9b0 tx=0x7f53b000ecc0 comp rx=0 tx=0).stop 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53b9a34700 1 -- 172.21.15.47:0/3574014770 reap_dead start 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53b9a34700 1 -- 172.21.15.47:0/3574014770 reap_dead start 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 shutdown_connections 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 >> 172.21.15.47:0/3574014770 conn(0x7f53b40a4a50 msgr2=0x7f53b40a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 shutdown_connections 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 -- 172.21.15.47:0/3574014770 wait complete. 2024-04-20T02:24:37.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:37.778+0000 7f53bb497700 1 librados: shutdown 2024-04-20T02:24:37.822 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:24:37.822 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-20T02:24:37.823 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['g'] 2024-04-20T02:24:37.823 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-20T02:24:37.823 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:24:37.823 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T02:24:37.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:24:37.836 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T02:24:37.836 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-04-20T02:24:37.836 INFO:tasks.ceph.mon.g:Sent signal 19 2024-04-20T02:24:37.836 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:24:52.837 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-04-20T02:24:52.837 INFO:tasks.ceph.mon.g:Sent signal 18 2024-04-20T02:24:52.837 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-04-20T02:24:52.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:24:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.994+0000 7f9b793d2700 10 monclient: get_monmap_and_config 2024-04-20T02:24:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.994+0000 7f9b793d2700 10 monclient: build_initial_monmap 2024-04-20T02:24:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.994+0000 7f9b793d2700 10 monclient: monmap: 2024-04-20T02:24:52.994 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:52.995596+0000 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:52.995596+0000 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:52.995 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:52.996 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:52.996 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 Processor -- start 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 -- start start 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient: init 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient: _add_conns ranks=[8,6,7,3,10,4,0,2,1,9,5] 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): picked mon.noname-e con 0x7f9b740a4c80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:52.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): picked mon.noname-d con 0x7f9b740a0d20 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): picked mon.noname-i con 0x7f9b740a55a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a0000 con 0x7f9b740a0d20 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a6a10 con 0x7f9b740a55a0 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7409e440 con 0x7f9b740a4c80 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b793d2700 10 monclient(hunting): authenticate will time out at 2172.396734s 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50678/0 (socket says 172.21.15.47:50678) 2024-04-20T02:24:52.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:51430/0 (socket says 172.21.15.47:51430) 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 learned_addr learned my addr 172.21.15.47:0/635072740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 10 monclient(hunting): get_auth_request con 0x7f9b740a0d20 auth_method 0 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b727fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b740a4c80 auth_method 0 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.995+0000 7f9b72ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:53.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 10 monclient(hunting): get_auth_request con 0x7f9b740a55a0 auth_method 0 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:52.996+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b740a4c80 auth_method 0 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 10 monclient(hunting): get_auth_request con 0x7f9b740a0d20 auth_method 0 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:53.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 10 monclient(hunting): get_auth_request con 0x7f9b740a55a0 auth_method 0 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.197+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b740a4c80 auth_method 0 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 10 monclient(hunting): get_auth_request con 0x7f9b740a0d20 auth_method 0 2024-04-20T02:24:53.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 10 monclient(hunting): get_auth_request con 0x7f9b740a55a0 auth_method 0 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:53.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:24:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:53.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:53.598+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:24:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 10 monclient(hunting): get_auth_request con 0x7f9b740a0d20 auth_method 0 2024-04-20T02:24:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:54.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b740a4c80 auth_method 0 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:24:54.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.399+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 10 monclient(hunting): get_auth_request con 0x7f9b740a55a0 auth_method 0 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:54.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:54.400+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:24:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): tick 2024-04-20T02:24:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:24:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 monclient(hunting): continuing hunt 2024-04-20T02:24:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:24:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 msgr2=0x7f9b740a07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:55.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b740a4c80 0x7f9b740a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 msgr2=0x7f9b74137a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9b740a55a0 0x7f9b74137a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 msgr2=0x7f9b740a5060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b740a0d20 0x7f9b740a5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient: _add_conns ranks=[3,0,1,4,5,2,9,10] 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): picked mon.noname-g con 0x7f9b600038d0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9b60006220 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:55.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9b60009f30 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 -- 172.21.15.47:0/635072740 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b740751e0 con 0x7f9b60006220 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 -- 172.21.15.47:0/635072740 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6000eda0 con 0x7f9b60009f30 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 1 -- 172.21.15.47:0/635072740 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a0000 con 0x7f9b600038d0 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b70ff9700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b600038d0 auth_method 0 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:55.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:56.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:55.996+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b600038d0 auth_method 0 2024-04-20T02:24:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:56.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:56.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.197+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:24:56.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b600038d0 auth_method 0 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:24:56.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:56.598+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191399207 0 0) 0x7f9b6000eda0 con 0x7f9b60009f30 2024-04-20T02:24:57.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): my global_id is 54730 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b540035b0 con 0x7f9b60009f30 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921940202 0 0) 0x7f9b740751e0 con 0x7f9b60006220 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 10 monclient(hunting): my global_id is 35856 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.200+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b6000eda0 con 0x7f9b60006220 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413127483 0 0) 0x7f9b540035b0 con 0x7f9b60009f30 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 msgr2=0x7f9b60005ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b600038d0 0x7f9b60005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 >> v1:172.21.15.47:6789/0 conn(0x7f9b60006220 legacy=0x7f9b60008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6000ffa0 con 0x7f9b60009f30 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f9b6c009c40 con 0x7f9b60009f30 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b793d2700 5 monclient: authenticate success, global_id 54730 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b793d2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b737fe700 1 -- 172.21.15.47:0/635072740 reap_dead start 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: dump: 2024-04-20T02:24:57.205 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.206 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 >> v1:172.21.15.179:6789/0 conn(0x7f9b60009f30 legacy=0x7f9b6000c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient: _add_conns ranks=[8,2,6,4,3,5,1,7,0] 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): picked mon.e con 0x7f9b54004390 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9b54004f40 0x7f9b54009a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): picked mon.b con 0x7f9b54004f40 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b54009f40 0x7f9b5400c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): picked mon.d con 0x7f9b54009f40 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740a0000 con 0x7f9b54004f40 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5400ca30 con 0x7f9b54009f40 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5400cba0 con 0x7f9b54004390 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b71ffb700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b793d2700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b54009f40 0x7f9b5400c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9b54004f40 0x7f9b54009a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b737fe700 10 monclient(hunting): get_auth_request con 0x7f9b54009f40 auth_method 0 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b737fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b727fc700 10 monclient(hunting): get_auth_request con 0x7f9b54004f40 auth_method 0 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.201+0000 7f9b727fc700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b737fe700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b54004390 auth_method 0 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b72ffd700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient(hunting): handle_auth_done global_id 54730 payload 293 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 msgr2=0x7f9b54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b54009f40 msgr2=0x7f9b5400c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b54009f40 0x7f9b5400c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 monclient: found mon.b 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 -- 172.21.15.47:0/635072740 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b540035b0 con 0x7f9b54004f40 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b737fe700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b54009f40 0x7f9b5400c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:57.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.202+0000 7f9b727fc700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9b54004f40 0x7f9b54009a00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9b740ab1a0 tx=0x7f9b54005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11ebf274eb6e8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b5c0126a0 con 0x7f9b54004f40 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: dump: 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.222 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b793d2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c00c040 con 0x7f9b54004f40 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 1 -- 172.21.15.47:0/635072740 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b5c014c50 con 0x7f9b54004f40 2024-04-20T02:24:57.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.221+0000 7f9b71ffb700 10 monclient: dump: 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.224 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:57.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 4 set_mon_vals no callback set 2024-04-20T02:24:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 10 monclient: shutdown 2024-04-20T02:24:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 1 -- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9b54004f40 msgr2=0x7f9b54009a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9b54004f40 0x7f9b54009a00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9b740ab1a0 tx=0x7f9b54005880 comp rx=0 tx=0).stop 2024-04-20T02:24:57.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 1 -- 172.21.15.47:0/635072740 shutdown_connections 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.222+0000 7f9b793d2700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9b54009f40 0x7f9b5400c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.223+0000 7f9b793d2700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9b54004f40 0x7f9b54009a00 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.223+0000 7f9b793d2700 1 --2- 172.21.15.47:0/635072740 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b54004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.223+0000 7f9b793d2700 1 -- 172.21.15.47:0/635072740 >> 172.21.15.47:0/635072740 conn(0x7f9b740aa280 msgr2=0x7f9b740aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.223+0000 7f9b793d2700 1 -- 172.21.15.47:0/635072740 shutdown_connections 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.223+0000 7f9b793d2700 1 -- 172.21.15.47:0/635072740 wait complete. 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.224+0000 7f9b793d2700 10 monclient: build_initial_monmap 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.224+0000 7f9b793d2700 10 monclient: monmap: 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:57.233 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.224+0000 7f9b793d2700 1 librados: starting msgr at 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.224+0000 7f9b793d2700 1 librados: starting objecter 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 Processor -- start 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 -- start start 2024-04-20T02:24:57.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 librados: setting wanted keys 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 librados: calling monclient init 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient: init 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient: _add_conns ranks=[8,3,1,6,2,4,7,0,5] 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b741cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): picked mon.noname-e con 0x7f9b54004390 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 0x7f9b741bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): picked mon.noname-g con 0x7f9b54004f40 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 0x7f9b741be140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): picked mon.noname-f con 0x7f9b54009f40 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b741be680 con 0x7f9b54009f40 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b741ad440 con 0x7f9b54004f40 2024-04-20T02:24:57.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7409dd20 con 0x7f9b54004390 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.225+0000 7f9b793d2700 10 monclient(hunting): authenticate will time out at 2176.626905s 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b741cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 0x7f9b741bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 0x7f9b741be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 0x7f9b741bdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:57612/0 (socket says 172.21.15.47:57612) 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 1 -- 172.21.15.47:0/1699412554 learned_addr learned my addr 172.21.15.47:0/1699412554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 0x7f9b741be140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:57692/0 (socket says 172.21.15.47:57692) 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request con 0x7f9b54004390 auth_method 0 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): get_auth_request con 0x7f9b54004f40 auth_method 0 2024-04-20T02:24:57.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): get_auth_request con 0x7f9b54009f40 auth_method 0 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b72ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient(hunting): handle_auth_done global_id 54739 payload 824 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 -- 172.21.15.47:0/1699412554 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 msgr2=0x7f9b741cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b741cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 -- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 msgr2=0x7f9b741bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 0x7f9b741bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 monclient: found mon.noname-f 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 -- 172.21.15.47:0/1699412554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7409ea00 con 0x7f9b54009f40 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b727fc700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 0x7f9b741bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b737fe700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 0x7f9b741be140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b740675f0 tx=0x7f9b6c007c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b6b7fe700 10 client.?.objecter ms_handle_connect 0x7f9b54009f40 2024-04-20T02:24:57.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b6b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.226+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b6c004880 con 0x7f9b54009f40 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: dump: 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.239 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c00f040 con 0x7f9b54009f40 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 5 monclient: authenticate success, global_id 54739 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 10 monclient: _renew_subs 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9b6c0188c0 con 0x7f9b54009f40 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b741aafe0 con 0x7f9b54009f40 2024-04-20T02:24:57.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b717fa700 4 set_mon_vals no callback set 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b6b7fe700 10 monclient: dump: 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.241 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 10 client.54739.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 10 monclient: _renew_subs 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b741cc560 con 0x7f9b54009f40 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.227+0000 7f9b793d2700 1 librados: init done 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9b6c018450 con 0x7f9b54009f40 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9b6003b150 0x7f9b6003d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9b6c04b330 con 0x7f9b54009f40 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 10 client.54739.objecter ms_dispatch 0x7f9b74001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:57.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 3 client.54739.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 3 client.54739.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b6b7fe700 20 client.54739.objecter dump_active .. 0 homeless 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b793d2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9b6003b150 0x7f9b6003d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b793d2700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b793d2700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b741cc6f0 con 0x7f9b54009f40 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.228+0000 7f9b72ffd700 10 monclient: get_auth_request con 0x7f9b6003b150 auth_method 0 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.229+0000 7f9b72ffd700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9b6003b150 0x7f9b6003d610 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006dd0 tx=0x7f9b6401a040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.229+0000 7f9b6b7fe700 10 client.54739.objecter ms_handle_connect 0x7f9b6003b150 2024-04-20T02:24:57.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.234+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f9b6c01c020 con 0x7f9b54009f40 2024-04-20T02:24:57.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.234+0000 7f9b6b7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.234+0000 7f9b6b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.542+0000 7f9b793d2700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.542+0000 7f9b793d2700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.542+0000 7f9b793d2700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.542+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9b741acd50 con 0x7f9b54009f40 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.543+0000 7f9b6b7fe700 1 -- 172.21.15.47:0/1699412554 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3169 (secure 0 0 0) 0x7f9b741acd50 con 0x7f9b54009f40 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.543+0000 7f9b6b7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:24:57.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.543+0000 7f9b6b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:57.544 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":68,"quorum":[0,1,2,3,5,6,7,8],"quorum_names":["a","f","b","g","h","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:24:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 10 librados: watch_flush enter 2024-04-20T02:24:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 10 librados: watch_flush exit 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 20 client.54739.objecter shutdown clearing up homeless session... 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 10 client.54739.objecter successfully canceled tick 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9b6003b150 msgr2=0x7f9b6003d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9b6003b150 0x7f9b6003d610 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006dd0 tx=0x7f9b6401a040 comp rx=0 tx=0).stop 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 10 monclient: shutdown 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 msgr2=0x7f9b741be140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.545+0000 7f9b793d2700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 0x7f9b741be140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b740675f0 tx=0x7f9b6c007c60 comp rx=0 tx=0).stop 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 shutdown_connections 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9b6003b150 0x7f9b6003d610 unknown :-1 s=CLOSED pgs=3232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b54009f40 0x7f9b741be140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9b54004390 0x7f9b741cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 --2- 172.21.15.47:0/1699412554 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9b54004f40 0x7f9b741bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 >> 172.21.15.47:0/1699412554 conn(0x7f9b740aa280 msgr2=0x7f9b740a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 shutdown_connections 2024-04-20T02:24:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 -- 172.21.15.47:0/1699412554 wait complete. 2024-04-20T02:24:57.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.546+0000 7f9b793d2700 1 librados: shutdown 2024-04-20T02:24:57.560 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-04-20T02:24:57.560 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-04-20T02:24:57.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:24:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.696+0000 7f5dd0783700 10 monclient: get_monmap_and_config 2024-04-20T02:24:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.696+0000 7f5dd0783700 10 monclient: build_initial_monmap 2024-04-20T02:24:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.696+0000 7f5dd0783700 10 monclient: monmap: 2024-04-20T02:24:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:57.697398+0000 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:57.697398+0000 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:57.697 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.696+0000 7f5dd0783700 1 Processor -- start 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.696+0000 7f5dd0783700 1 -- start start 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.696+0000 7f5dd0783700 10 monclient: init 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient: _add_conns ranks=[9,5,1,7,10,3,4,2,6,8,0] 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5dc8137d90 0x7f5dc80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): picked mon.noname-a con 0x7f5dc8137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5dc8138170 0x7f5dc80a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): picked mon.noname-h con 0x7f5dc8138170 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5dc81385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:57.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc80751e0 con 0x7f5dc81385b0 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc809fa40 con 0x7f5dc8137d90 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc809dfc0 con 0x7f5dc8138170 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dd0783700 10 monclient(hunting): authenticate will time out at 2177.098684s 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5dc8137d90 0x7f5dc80a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5dc8137d90 0x7f5dc80a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53444/0 (socket says 172.21.15.47:53444) 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 1 -- 172.21.15.47:0/2006001872 learned_addr learned my addr 172.21.15.47:0/2006001872 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): get_auth_request con 0x7f5dc8137d90 auth_method 0 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.697+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcdd1e700 1 --2- 172.21.15.47:0/2006001872 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5dc8138170 0x7f5dc80a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcdd1e700 10 monclient(hunting): get_auth_request con 0x7f5dc8138170 auth_method 0 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcdd1e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcdd1e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcdd1e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 10 monclient(hunting): handle_auth_done global_id 35865 payload 274 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 1 -- 172.21.15.47:0/2006001872 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5dc8138170 msgr2=0x7f5dc80a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 1 --2- 172.21.15.47:0/2006001872 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5dc8138170 0x7f5dc80a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 1 -- 172.21.15.47:0/2006001872 >> v1:172.21.15.179:6789/0 conn(0x7f5dc81385b0 legacy=0x7f5dc80a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 1 monclient: found mon.noname-a 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:57.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 1 -- 172.21.15.47:0/2006001872 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80a7bc0 con 0x7f5dc8137d90 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dce51f700 1 --2- 172.21.15.47:0/2006001872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8137d90 0x7f5dc80a09e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009920 tx=0x7f5dc4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb6e08a41c5e1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 1 -- 172.21.15.47:0/2006001872 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5dc4009e70 con 0x7f5dc8137d90 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: dump: 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.710 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 1 -- 172.21.15.47:0/2006001872 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc400bc90 con 0x7f5dc8137d90 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dd0783700 5 monclient: authenticate success, global_id 35865 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dd0783700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dd0783700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:57.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 1 -- 172.21.15.47:0/2006001872 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5dc400be00 con 0x7f5dc8137d90 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dd0783700 4 set_mon_vals no callback set 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.698+0000 7f5dcd51d700 10 monclient: dump: 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.712 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 10 monclient: shutdown 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 -- 172.21.15.47:0/2006001872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8137d90 msgr2=0x7f5dc80a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 --2- 172.21.15.47:0/2006001872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8137d90 0x7f5dc80a09e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009920 tx=0x7f5dc4009c30 comp rx=0 tx=0).stop 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 -- 172.21.15.47:0/2006001872 shutdown_connections 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 --2- 172.21.15.47:0/2006001872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8137d90 0x7f5dc80a09e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 --2- 172.21.15.47:0/2006001872 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5dc8138170 0x7f5dc80a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 -- 172.21.15.47:0/2006001872 >> 172.21.15.47:0/2006001872 conn(0x7f5dc80aa8b0 msgr2=0x7f5dc80a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 -- 172.21.15.47:0/2006001872 shutdown_connections 2024-04-20T02:24:57.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 1 -- 172.21.15.47:0/2006001872 wait complete. 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 10 monclient: build_initial_monmap 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.699+0000 7f5dd0783700 10 monclient: monmap: 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:57.714 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 librados: starting msgr at 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 librados: starting objecter 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 Processor -- start 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 -- start start 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 librados: setting wanted keys 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 librados: calling monclient init 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient: init 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient: _add_conns ranks=[1,2,3,0,7,8,5,4,6] 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc8137d90 0x7f5dc8120fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): picked mon.noname-f con 0x7f5dc8137d90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:57.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5dc8138170 0x7f5dc81214f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): picked mon.noname-b con 0x7f5dc8138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc81385b0 0x7f5dc8121a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): picked mon.noname-g con 0x7f5dc81385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8123210 con 0x7f5dc8137d90 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc809dc20 con 0x7f5dc8138170 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc809e9e0 con 0x7f5dc81385b0 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dd0783700 10 monclient(hunting): authenticate will time out at 2177.101846s 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5dc8138170 0x7f5dc81214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5dc8138170 0x7f5dc81214f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60502/0 (socket says 172.21.15.47:60502) 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 1 -- 172.21.15.47:0/3712226745 learned_addr learned my addr 172.21.15.47:0/3712226745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 10 monclient(hunting): get_auth_request con 0x7f5dc8138170 auth_method 0 2024-04-20T02:24:57.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.700+0000 7f5dcdd1e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc8137d90 0x7f5dc8120fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): get_auth_request con 0x7f5dc8137d90 auth_method 0 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc81385b0 0x7f5dc8121a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): get_auth_request con 0x7f5dc81385b0 auth_method 0 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dced20700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 10 monclient(hunting): handle_auth_done global_id 24167 payload 824 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc81385b0 msgr2=0x7f5dc8121a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5dc81385b0 0x7f5dc8121a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc8137d90 msgr2=0x7f5dc8120fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dc8137d90 0x7f5dc8120fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 monclient: found mon.noname-b 2024-04-20T02:24:57.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4009550 con 0x7f5dc8138170 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dcdd1e700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5dc8138170 0x7f5dc81214f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f5db800e810 tx=0x7f5db800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 client.?.objecter ms_handle_connect 0x7f5dc8138170 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5db800c070 con 0x7f5dc8138170 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: dump: 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.719 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db8013880 con 0x7f5dc8138170 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 5 monclient: authenticate success, global_id 24167 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:57.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5db8013e60 con 0x7f5dc8138170 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 10 monclient: _renew_subs 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc8124b70 con 0x7f5dc8138170 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dccd1c700 4 set_mon_vals no callback set 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dbeffd700 10 monclient: dump: 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.721 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 10 client.24167.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 10 monclient: _renew_subs 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc8122810 con 0x7f5dc8138170 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.701+0000 7f5dd0783700 1 librados: init done 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dd0783700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dd0783700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 0x7f5dc809a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8068180 con 0x7f5dc806aac0 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5db80139f0 con 0x7f5dc8138170 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 0x7f5dc809a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient: get_auth_request con 0x7f5dc806aac0 auth_method 0 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 10 monclient(hunting): handle_auth_done global_id 35874 payload 824 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dbeffd700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5dac03a2f0 0x7f5dac03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dce51f700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dc809f760 con 0x7f5dc806aac0 2024-04-20T02:24:57.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.702+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5db804a3b0 con 0x7f5dc8138170 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 client.24167.objecter ms_dispatch 0x7f5dc8001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 3 client.24167.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 3 client.24167.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 20 client.24167.objecter dump_active .. 0 homeless 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 client.24167.objecter ms_handle_connect 0x7f5dc806aac0 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 client.24167.objecter resend_mon_ops 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dce51f700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 0x7f5dc809a9a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009fd0 tx=0x7f5dc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dced20700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5dac03a2f0 0x7f5dac03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dced20700 10 monclient: get_auth_request con 0x7f5dac03a2f0 auth_method 0 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5dc4020070 con 0x7f5dc806aac0 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 monclient: dump: 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.724 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5dc400bb20 con 0x7f5dc806aac0 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 msgr2=0x7f5dc809a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 0x7f5dc809a9a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009fd0 tx=0x7f5dc4004750 comp rx=0 tx=0).stop 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 msgr2=0x7f5dc809a9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc806aac0 0x7f5dc809a9a0 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009fd0 tx=0x7f5dc4004750 comp rx=0 tx=0).stop 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dced20700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5dac03a2f0 0x7f5dac03c7b0 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8123170 tx=0x7f5dc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.703+0000 7f5dbeffd700 10 client.24167.objecter ms_handle_connect 0x7f5dac03a2f0 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dd0783700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dd0783700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 0x7f5dc809aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dd0783700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80a1270 con 0x7f5dc8124510 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 0x7f5dc809aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient: get_auth_request con 0x7f5dc8124510 auth_method 0 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 10 monclient(hunting): handle_auth_done global_id 35883 payload 824 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 1 -- 172.21.15.47:0/3712226745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5dc8075620 con 0x7f5dc8124510 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dbeffd700 10 client.24167.objecter ms_handle_connect 0x7f5dc8124510 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dce51f700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 0x7f5dc809aee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4023000 tx=0x7f5dc4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.711+0000 7f5dbeffd700 10 client.24167.objecter resend_mon_ops 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5dc4006d20 con 0x7f5dc8124510 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 10 monclient: dump: 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.729 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9512 (secure 0 0 0) 0x7f5dc402c8e0 con 0x7f5dc8124510 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:57.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 msgr2=0x7f5dc809aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 0x7f5dc809aee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4023000 tx=0x7f5dc4011000 comp rx=0 tx=0).stop 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 msgr2=0x7f5dc809aee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dbeffd700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5dc8124510 0x7f5dc809aee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:57.731 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:57.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:57.733 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:57.734 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:57.735 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.736 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.737 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.738 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.740 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.742 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.746 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.749 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.750 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.752 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:57.752 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 10 librados: watch_flush enter 2024-04-20T02:24:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 10 librados: watch_flush exit 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 20 client.24167.objecter shutdown clearing up homeless session... 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 10 client.24167.objecter successfully canceled tick 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5dac03a2f0 msgr2=0x7f5dac03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5dac03a2f0 0x7f5dac03c7b0 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8123170 tx=0x7f5dc0009040 comp rx=0 tx=0).stop 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 10 monclient: shutdown 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5dc8138170 msgr2=0x7f5dc81214f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 --2- 172.21.15.47:0/3712226745 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5dc8138170 0x7f5dc81214f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f5db800e810 tx=0x7f5db800eb20 comp rx=0 tx=0).stop 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dced20700 1 -- 172.21.15.47:0/3712226745 reap_dead start 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dced20700 1 -- 172.21.15.47:0/3712226745 reap_dead start 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 shutdown_connections 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 >> 172.21.15.47:0/3712226745 conn(0x7f5dc80aa8b0 msgr2=0x7f5dc80a3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 shutdown_connections 2024-04-20T02:24:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 -- 172.21.15.47:0/3712226745 wait complete. 2024-04-20T02:24:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.712+0000 7f5dd0783700 1 librados: shutdown 2024-04-20T02:24:57.755 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:57.755 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-04-20T02:24:57.755 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.892+0000 7fe697006700 10 monclient: get_monmap_and_config 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.892+0000 7fe697006700 10 monclient: build_initial_monmap 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.892+0000 7fe697006700 10 monclient: monmap: 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:57.894103+0000 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:57.894103+0000 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:57.893 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:57.894 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.893+0000 7fe697006700 1 Processor -- start 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.893+0000 7fe697006700 1 -- start start 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.893+0000 7fe697006700 10 monclient: init 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient: _add_conns ranks=[0,7,4,5,8,9,2,1,10,6,3] 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe6900a0830 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6900a01b0 0x7fe69013cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): picked mon.noname-i con 0x7fe6900a01b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe69013d280 0x7fe69013f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): picked mon.noname-c con 0x7fe69013d280 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:57.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6900751e0 con 0x7fe6900a0830 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe690067820 con 0x7fe69013d280 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe69009deb0 con 0x7fe6900a01b0 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe6955a3700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe69013d280 msgr2=0x7fe69013f670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe697006700 10 monclient(hunting): authenticate will time out at 2177.295523s 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe6955a3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe69013d280 0x7fe69013f670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe694da2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe6900a0830 0x7fe6900a0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51246/0 (socket says 172.21.15.47:51246) 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe694da2700 1 -- 172.21.15.47:0/2735717959 learned_addr learned my addr 172.21.15.47:0/2735717959 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:57.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499177646 0 0) 0x7fe6900751e0 con 0x7fe6900a0830 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68f7fe700 10 monclient(hunting): _negotiate 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68f7fe700 10 monclient(hunting): my global_id is 35892 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68ffff700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6900a01b0 0x7fe69013cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6800035b0 con 0x7fe6900a0830 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68ffff700 10 monclient(hunting): get_auth_request con 0x7fe6900a01b0 auth_method 0 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.894+0000 7fe68ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 819064673 0 0) 0x7fe6800035b0 con 0x7fe6900a0830 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6900a01b0 msgr2=0x7fe69013cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6900a01b0 0x7fe69013cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe69013d280 msgr2=0x7fe69013f670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe69013d280 0x7fe69013f670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690067330 con 0x7fe6900a0830 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7fe6840018e0 con 0x7fe6900a0830 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe697006700 5 monclient: authenticate success, global_id 35892 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe697006700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: dump: 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.916 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 >> v1:172.21.15.47:6789/0 conn(0x7fe6900a0830 legacy=0x7fe6900a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient: _add_conns ranks=[3,0,6,8,1,5,4,7,2] 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe680004250 0x7fe680004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): picked mon.g con 0x7fe680004250 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe680004de0 0x7fe680005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): picked mon.a con 0x7fe680004de0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe680009c80 0x7fe68000c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): picked mon.d con 0x7fe680009c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690067820 con 0x7fe680004de0 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69009deb0 con 0x7fe680004250 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe68000ca20 con 0x7fe680009c80 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68f7fe700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe697006700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe680004de0 0x7fe680005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): get_auth_request con 0x7fe680004de0 auth_method 0 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe680009c80 0x7fe68000c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe680004250 0x7fe680004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): get_auth_request con 0x7fe680009c80 auth_method 0 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): get_auth_request con 0x7fe680004250 auth_method 0 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient(hunting): handle_auth_done global_id 35892 payload 293 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 -- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe680009c80 msgr2=0x7fe68000c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe680009c80 0x7fe68000c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 -- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe680004250 msgr2=0x7fe680004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe680004250 0x7fe680004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 monclient: found mon.a 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 -- 172.21.15.47:0/2735717959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6800035b0 con 0x7fe680004de0 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe6955a3700 1 -- 172.21.15.47:0/2735717959 reap_dead start 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.895+0000 7fe68ffff700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe680004de0 0x7fe680005260 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe6900a7e80 tx=0x7fe680005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfee7a45c3af321 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67c00eb00 con 0x7fe680004de0 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: dump: 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00ecc0 con 0x7fe680004de0 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 1 -- 172.21.15.47:0/2735717959 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67c00ee30 con 0x7fe680004de0 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 4 set_mon_vals no callback set 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe68f7fe700 10 monclient: dump: 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 10 monclient: shutdown 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 -- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe680004de0 msgr2=0x7fe680005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe680004de0 0x7fe680005260 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe6900a7e80 tx=0x7fe680005880 comp rx=0 tx=0).stop 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 -- 172.21.15.47:0/2735717959 shutdown_connections 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 --2- 172.21.15.47:0/2735717959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe680004de0 0x7fe680005260 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 -- 172.21.15.47:0/2735717959 >> 172.21.15.47:0/2735717959 conn(0x7fe6900a7570 msgr2=0x7fe6900a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 -- 172.21.15.47:0/2735717959 shutdown_connections 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.896+0000 7fe697006700 1 -- 172.21.15.47:0/2735717959 wait complete. 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 10 monclient: build_initial_monmap 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 10 monclient: monmap: 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 1 librados: starting msgr at 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 1 librados: starting objecter 2024-04-20T02:24:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 1 Processor -- start 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 1 -- start start 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 1 librados: setting wanted keys 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 1 librados: calling monclient init 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.897+0000 7fe697006700 10 monclient: init 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient: _add_conns ranks=[5,4,1,8,2,0,3,7,6] 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe680004de0 0x7fe6901219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): picked mon.noname-h con 0x7fe680004de0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe690122c10 0x7fe690121ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): picked mon.noname-c con 0x7fe690122c10 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 0x7fe6901cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): picked mon.noname-f con 0x7fe690122420 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69009e900 con 0x7fe690122420 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe69009e440 con 0x7fe690122c10 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe690124fa0 con 0x7fe680004de0 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe697006700 10 monclient(hunting): authenticate will time out at 2177.299681s 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe68ffff700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe690122c10 msgr2=0x7fe690121ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe690122c10 0x7fe690121ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 0x7fe6901cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe694da2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe680004de0 0x7fe6901219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 0x7fe6901cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:57716/0 (socket says 172.21.15.47:57716) 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 1 -- 172.21.15.47:0/3932068810 learned_addr learned my addr 172.21.15.47:0/3932068810 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 10 monclient(hunting): get_auth_request con 0x7fe690122420 auth_method 0 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe6955a3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe694da2700 10 monclient(hunting): get_auth_request con 0x7fe680004de0 auth_method 0 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe694da2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe694da2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.898+0000 7fe694da2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 10 monclient(hunting): handle_auth_done global_id 54757 payload 824 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe680004de0 msgr2=0x7fe6901219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe680004de0 0x7fe6901219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe690122c10 msgr2=0x7fe690121ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe690122c10 0x7fe690121ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 10 monclient: _finish_hunting 0 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 monclient: found mon.noname-f 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690124700 con 0x7fe690122420 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe6955a3700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 0x7fe6901cc410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe688000f80 tx=0x7fe68800ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 client.?.objecter ms_handle_connect 0x7fe690122420 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe68800f400 con 0x7fe690122420 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: dump: 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 5 monclient: authenticate success, global_id 54757 2024-04-20T02:24:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68800f5e0 con 0x7fe690122420 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 10 monclient: _renew_subs 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe68800fbc0 con 0x7fe690122420 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6901249e0 con 0x7fe690122420 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68effd700 4 set_mon_vals no callback set 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe68d7fa700 10 monclient: dump: 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 10 client.54757.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 10 monclient: _renew_subs 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6901253c0 con 0x7fe690122420 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.899+0000 7fe697006700 1 librados: init done 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe697006700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe697006700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 0x7fe6901cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69004ed40 con 0x7fe69006aac0 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 0x7fe6901cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe688015020 con 0x7fe690122420 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient: get_auth_request con 0x7fe69006aac0 auth_method 0 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.900+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe67403a2f0 0x7fe67403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe694da2700 10 monclient(hunting): handle_auth_done global_id 24176 payload 824 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe694da2700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe690125610 con 0x7fe69006aac0 2024-04-20T02:24:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fe68804a980 con 0x7fe690122420 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 client.54757.objecter ms_dispatch 0x7fe690001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 3 client.54757.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 3 client.54757.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe694da2700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 0x7fe6901cc950 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe684000fb0 tx=0x7fe6840075d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 20 client.54757.objecter dump_active .. 0 homeless 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 client.54757.objecter ms_handle_connect 0x7fe69006aac0 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 client.54757.objecter resend_mon_ops 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe684012d00 con 0x7fe69006aac0 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 monclient: dump: 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68ffff700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe67403a2f0 0x7fe67403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68ffff700 10 monclient: get_auth_request con 0x7fe67403a2f0 auth_method 0 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe684012e60 con 0x7fe69006aac0 2024-04-20T02:24:57.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 msgr2=0x7fe6901cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 0x7fe6901cc950 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fe684000fb0 tx=0x7fe6840075d0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 msgr2=0x7fe6901cc950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.901+0000 7fe68d7fa700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe69006aac0 0x7fe6901cc950 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.902+0000 7fe68d7fa700 10 client.54757.objecter ms_handle_connect 0x7fe67403a2f0 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.902+0000 7fe68ffff700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe67403a2f0 0x7fe67403c7b0 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7fe690071b90 tx=0x7fe67c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe697006700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe697006700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 0x7fe69009b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe697006700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe690068310 con 0x7fe6901cce90 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 0x7fe69009b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient: get_auth_request con 0x7fe6901cce90 auth_method 0 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:57.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.909+0000 7fe694da2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe694da2700 10 monclient(hunting): handle_auth_done global_id 24185 payload 824 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe694da2700 1 -- 172.21.15.47:0/3932068810 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe69009f1c0 con 0x7fe6901cce90 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 client.54757.objecter ms_handle_connect 0x7fe6901cce90 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 client.54757.objecter resend_mon_ops 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe694da2700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 0x7fe69009b280 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe684019000 tx=0x7fe6900674c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe684025070 con 0x7fe6901cce90 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 monclient: dump: 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:57.937 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9174 (secure 0 0 0) 0x7fe6840158d0 con 0x7fe6901cce90 2024-04-20T02:24:57.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 msgr2=0x7fe69009b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 0x7fe69009b280 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe684019000 tx=0x7fe6900674c0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 msgr2=0x7fe69009b280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe68d7fa700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6901cce90 0x7fe69009b280 secure :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe684019000 tx=0x7fe6900674c0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:57.939 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:57.940 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:57.941 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:57.942 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:57.943 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:57.946 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:57.947 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:57.949 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:57.950 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.951 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:57.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:57.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:57.955 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:57.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.959 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:57.960 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe697006700 10 librados: watch_flush enter 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe697006700 10 librados: watch_flush exit 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe697006700 20 client.54757.objecter shutdown clearing up homeless session... 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.910+0000 7fe697006700 10 client.54757.objecter successfully canceled tick 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe67403a2f0 msgr2=0x7fe67403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe67403a2f0 0x7fe67403c7b0 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 crypto rx=0x7fe690071b90 tx=0x7fe67c008030 comp rx=0 tx=0).stop 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 10 monclient: shutdown 2024-04-20T02:24:57.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 msgr2=0x7fe6901cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 0x7fe6901cc410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe688000f80 tx=0x7fe68800ef80 comp rx=0 tx=0).stop 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe6955a3700 1 -- 172.21.15.47:0/3932068810 reap_dead start 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 shutdown_connections 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 --2- 172.21.15.47:0/3932068810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe690122420 0x7fe6901cc410 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 >> 172.21.15.47:0/3932068810 conn(0x7fe6900a7570 msgr2=0x7fe69009d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 shutdown_connections 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 -- 172.21.15.47:0/3932068810 wait complete. 2024-04-20T02:24:57.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:57.911+0000 7fe697006700 1 librados: shutdown 2024-04-20T02:24:57.963 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:57.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-04-20T02:24:57.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:24:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.103+0000 7f9365553700 10 monclient: get_monmap_and_config 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.103+0000 7f9365553700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.103+0000 7f9365553700 10 monclient: monmap: 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:58.104938+0000 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:58.104938+0000 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:58.105 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 1 Processor -- start 2024-04-20T02:24:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 1 -- start start 2024-04-20T02:24:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 10 monclient: init 2024-04-20T02:24:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 10 monclient: _add_conns ranks=[5,6,7,1,0,4,9,10,2,3,8] 2024-04-20T02:24:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93600a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 10 monclient(hunting): picked mon.noname-h con 0x7f9360137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 0x7f93600a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 10 monclient(hunting): picked mon.noname-d con 0x7f9360138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.104+0000 7f9365553700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f93601385b0 0x7f93600a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient(hunting): picked mon.noname-i con 0x7f93601385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f936009fa40 con 0x7f9360137d90 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f936009dfc0 con 0x7f9360138170 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93600a7bc0 con 0x7f93601385b0 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient(hunting): authenticate will time out at 2177.506047s 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 0x7f93600a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 0x7f93600a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51244/0 (socket says 172.21.15.47:51244) 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 -- 172.21.15.47:0/2153225652 learned_addr learned my addr 172.21.15.47:0/2153225652 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): get_auth_request con 0x7f9360138170 auth_method 0 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93600a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f93601385b0 0x7f93600a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): get_auth_request con 0x7f9360137d90 auth_method 0 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): get_auth_request con 0x7f93601385b0 auth_method 0 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient(hunting): handle_auth_done global_id 54501 payload 274 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 -- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f93601385b0 msgr2=0x7f93600a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f93601385b0 0x7f93600a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 -- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 msgr2=0x7f93600a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93600a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 monclient: found mon.noname-d 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 -- 172.21.15.47:0/2153225652 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600751e0 con 0x7f9360138170 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935e7fc700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 0x7f93600a5c90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9348009950 tx=0x7f9348009c60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d8d5ba958c45de9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 1 -- 172.21.15.47:0/2153225652 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9348011aa0 con 0x7f9360138170 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: dump: 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 1 -- 172.21.15.47:0/2153225652 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348011c20 con 0x7f9360138170 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 5 monclient: authenticate success, global_id 54501 2024-04-20T02:24:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 1 -- 172.21.15.47:0/2153225652 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9348011d90 con 0x7f9360138170 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f9365553700 4 set_mon_vals no callback set 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.105+0000 7f935dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f935dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f935dffb700 10 monclient: dump: 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 10 monclient: shutdown 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 -- 172.21.15.47:0/2153225652 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 msgr2=0x7f93600a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 0x7f93600a5c90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9348009950 tx=0x7f9348009c60 comp rx=0 tx=0).stop 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 -- 172.21.15.47:0/2153225652 shutdown_connections 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f93601385b0 0x7f93600a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93600a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 --2- 172.21.15.47:0/2153225652 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9360138170 0x7f93600a5c90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 -- 172.21.15.47:0/2153225652 >> 172.21.15.47:0/2153225652 conn(0x7f93600aa8b0 msgr2=0x7f93600a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 -- 172.21.15.47:0/2153225652 shutdown_connections 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.106+0000 7f9365553700 1 -- 172.21.15.47:0/2153225652 wait complete. 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 10 monclient: monmap: 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 1 librados: starting msgr at 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 1 librados: starting objecter 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 1 Processor -- start 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 1 -- start start 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 1 librados: setting wanted keys 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 1 librados: calling monclient init 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.107+0000 7f9365553700 10 monclient: init 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient: _add_conns ranks=[5,3,4,7,2,0,1,8,6] 2024-04-20T02:24:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93601cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): picked mon.noname-h con 0x7f9360137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9360138170 0x7f9360136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): picked mon.noname-g con 0x7f9360138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f93601385b0 0x7f93601374e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): picked mon.noname-c con 0x7f93601385b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936009ea50 con 0x7f9360138170 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93601243a0 con 0x7f93601385b0 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f936009dc20 con 0x7f9360137d90 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f9365553700 10 monclient(hunting): authenticate will time out at 2177.509469s 2024-04-20T02:24:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935f7fe700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f93601385b0 msgr2=0x7f93601374e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935f7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f93601385b0 0x7f93601374e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93601cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93601cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:57248/0 (socket says 172.21.15.47:57248) 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 -- 172.21.15.47:0/438066619 learned_addr learned my addr 172.21.15.47:0/438066619 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): get_auth_request con 0x7f9360137d90 auth_method 0 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9360138170 0x7f9360136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): get_auth_request con 0x7f9360138170 auth_method 0 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient(hunting): handle_auth_done global_id 64256 payload 824 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f93601385b0 msgr2=0x7f93601374e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f93601385b0 0x7f93601374e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9360138170 msgr2=0x7f9360136fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9360138170 0x7f9360136fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.108+0000 7f935effd700 1 monclient: found mon.noname-h 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f935effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f935effd700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f935effd700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348009550 con 0x7f9360137d90 2024-04-20T02:24:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f935e7fc700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9360138170 0x7f9360136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f935effd700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93601cc410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f935000e960 tx=0x7f935000ec70 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 client.?.objecter ms_handle_connect 0x7f9360137d90 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935000eeb0 con 0x7f9360137d90 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: dump: 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000ccc0 con 0x7f9360137d90 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 5 monclient: authenticate success, global_id 64256 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f935000ce30 con 0x7f9360137d90 2024-04-20T02:24:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 10 monclient: _renew_subs 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93601240c0 con 0x7f9360137d90 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f935d7fa700 4 set_mon_vals no callback set 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 10 client.64256.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f93577fe700 10 monclient: dump: 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 10 monclient: _renew_subs 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9360123380 con 0x7f9360137d90 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.109+0000 7f9365553700 1 librados: init done 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f9365553700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f9365553700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 0x7f93601c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9360068180 con 0x7f936006aac0 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f935000c850 con 0x7f9360137d90 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 0x7f93601c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient: get_auth_request con 0x7f936006aac0 auth_method 0 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(355..355 src has 206..355) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f935004a660 con 0x7f9360137d90 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 10 client.64256.objecter ms_dispatch 0x7f9360001550 osd_map(355..355 src has 206..355) v4 2024-04-20T02:24:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 3 client.64256.objecter handle_osd_map got epochs [355,355] > 0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 3 client.64256.objecter handle_osd_map decoding full epoch 355 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 10 monclient(hunting): handle_auth_done global_id 54510 payload 824 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935e7fc700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f936009f760 con 0x7f936006aac0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f93577fe700 20 client.64256.objecter dump_active .. 0 homeless 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935f7fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.110+0000 7f935f7fe700 10 monclient: get_auth_request con 0x7f934003a2f0 auth_method 0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 client.64256.objecter ms_handle_connect 0x7f936006aac0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 client.64256.objecter resend_mon_ops 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f935e7fc700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 0x7f93601c8e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f936009e9b0 tx=0x7f9348004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9348020070 con 0x7f936006aac0 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 monclient: dump: 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 client.64256.objecter ms_handle_connect 0x7f934003a2f0 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f935f7fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f9360123330 tx=0x7f935800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9348004490 con 0x7f936006aac0 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 msgr2=0x7f93601c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 0x7f93601c8e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f936009e9b0 tx=0x7f9348004750 comp rx=0 tx=0).stop 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 msgr2=0x7f93601c8e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.111+0000 7f93577fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f936006aac0 0x7f93601c8e90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f9365553700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f9365553700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 0x7f93601c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f9365553700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93600756c0 con 0x7f93601c93d0 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 0x7f93601c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient: get_auth_request con 0x7f93601c93d0 auth_method 0 2024-04-20T02:24:58.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 10 monclient(hunting): handle_auth_done global_id 54519 payload 824 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 1 -- 172.21.15.47:0/438066619 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f93601248c0 con 0x7f93601c93d0 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f93577fe700 10 client.64256.objecter ms_handle_connect 0x7f93601c93d0 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f935e7fc700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 0x7f93601c97e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93480098f0 tx=0x7f9348011000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.119+0000 7f93577fe700 10 client.64256.objecter resend_mon_ops 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9348036440 con 0x7f93601c93d0 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 10 monclient: dump: 2024-04-20T02:24:58.139 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7f934802caa0 con 0x7f93601c93d0 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 msgr2=0x7f93601c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 0x7f93601c97e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93480098f0 tx=0x7f9348011000 comp rx=0 tx=0).stop 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 msgr2=0x7f93601c97e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f93577fe700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f93601c93d0 0x7f93601c97e0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93480098f0 tx=0x7f9348011000 comp rx=0 tx=0).stop 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:58.141 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:58.142 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:58.142 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:58.142 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:58.142 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:58.142 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:58.142 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.147 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:58.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:58.149 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.150 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:58.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:58.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.160 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.162 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.164 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.166 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.167 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.168 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:58.169 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 10 librados: watch_flush enter 2024-04-20T02:24:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 10 librados: watch_flush exit 2024-04-20T02:24:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 20 client.64256.objecter shutdown clearing up homeless session... 2024-04-20T02:24:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 10 client.64256.objecter successfully canceled tick 2024-04-20T02:24:58.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 msgr2=0x7f934003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f934003a2f0 0x7f934003c7b0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f9360123330 tx=0x7f935800b040 comp rx=0 tx=0).stop 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 10 monclient: shutdown 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 msgr2=0x7f93601cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 --2- 172.21.15.47:0/438066619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9360137d90 0x7f93601cc410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f935000e960 tx=0x7f935000ec70 comp rx=0 tx=0).stop 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f935f7fe700 1 -- 172.21.15.47:0/438066619 reap_dead start 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f935f7fe700 1 -- 172.21.15.47:0/438066619 reap_dead start 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 shutdown_connections 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 >> 172.21.15.47:0/438066619 conn(0x7f93600aa8b0 msgr2=0x7f936009d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 shutdown_connections 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 -- 172.21.15.47:0/438066619 wait complete. 2024-04-20T02:24:58.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.120+0000 7f9365553700 1 librados: shutdown 2024-04-20T02:24:58.173 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:58.173 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-04-20T02:24:58.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:24:58.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.322+0000 7fc1f4e1d700 10 monclient: get_monmap_and_config 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.322+0000 7fc1f4e1d700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.322+0000 7fc1f4e1d700 10 monclient: monmap: 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:58.323872+0000 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:58.323872+0000 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.323 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:58.324 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 Processor -- start 2024-04-20T02:24:58.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 -- start start 2024-04-20T02:24:58.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient: init 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient: _add_conns ranks=[8,5,7,10,6,2,3,1,0,9,4] 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 0x7fc1f0071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): picked mon.noname-e con 0x7fc1f0071730 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f006f020 0x7fc1f00a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): picked mon.noname-h con 0x7fc1f006f020 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f00a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): picked mon.noname-i con 0x7fc1f00a2db0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f009fea0 con 0x7fc1f006f020 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f009e7b0 con 0x7fc1f00a2db0 2024-04-20T02:24:58.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006d190 con 0x7fc1f0071730 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.323+0000 7fc1f4e1d700 10 monclient(hunting): authenticate will time out at 2177.724965s 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 0x7fc1f0071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 0x7fc1f0071b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41102/0 (socket says 172.21.15.47:41102) 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 -- 172.21.15.47:0/549010762 learned_addr learned my addr 172.21.15.47:0/549010762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): get_auth_request con 0x7fc1f0071730 auth_method 0 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f006f020 0x7fc1f00a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): get_auth_request con 0x7fc1f006f020 auth_method 0 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f00a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): get_auth_request con 0x7fc1f00a2db0 auth_method 0 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1eed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_done global_id 35360 payload 274 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 -- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 msgr2=0x7fc1f00a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f00a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 -- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f006f020 msgr2=0x7fc1f00a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f006f020 0x7fc1f00a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 monclient: found mon.noname-e 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 -- 172.21.15.47:0/549010762 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00751e0 con 0x7fc1f0071730 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 0x7fc1f0071b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009920 tx=0x7fc1e0009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=efa33ff04eec54f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 1 -- 172.21.15.47:0/549010762 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0011aa0 con 0x7fc1f0071730 2024-04-20T02:24:58.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: dump: 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.346 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1f4e1d700 5 monclient: authenticate success, global_id 35360 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1f4e1d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 1 -- 172.21.15.47:0/549010762 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e0011c00 con 0x7fc1f0071730 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1f4e1d700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1f4e1d700 4 set_mon_vals no callback set 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 1 -- 172.21.15.47:0/549010762 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0011d70 con 0x7fc1f0071730 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.324+0000 7fc1edd9b700 10 monclient: dump: 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.348 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.349 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 10 monclient: shutdown 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/549010762 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 msgr2=0x7fc1f0071b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 0x7fc1f0071b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009920 tx=0x7fc1e0009c30 comp rx=0 tx=0).stop 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/549010762 shutdown_connections 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f00a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0071730 0x7fc1f0071b10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/549010762 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f006f020 0x7fc1f00a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/549010762 >> 172.21.15.47:0/549010762 conn(0x7fc1f00a3f90 msgr2=0x7fc1f006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/549010762 shutdown_connections 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.325+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/549010762 wait complete. 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.326+0000 7fc1f4e1d700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.326+0000 7fc1f4e1d700 10 monclient: monmap: 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.351 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.352 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.326+0000 7fc1f4e1d700 1 librados: starting msgr at 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.326+0000 7fc1f4e1d700 1 librados: starting objecter 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.326+0000 7fc1f4e1d700 1 Processor -- start 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 -- start start 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 librados: setting wanted keys 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 librados: calling monclient init 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient: init 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient: _add_conns ranks=[1,3,7,5,6,4,0,8,2] 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f006f020 0x7fc1f01cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): picked mon.noname-f con 0x7fc1f006f020 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1f0071730 0x7fc1f0133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): picked mon.noname-g con 0x7fc1f0071730 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:58.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f0133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): picked mon.noname-i con 0x7fc1f00a2db0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0127d70 con 0x7fc1f006f020 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f009f220 con 0x7fc1f0071730 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0072410 con 0x7fc1f00a2db0 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1f4e1d700 10 monclient(hunting): authenticate will time out at 2177.728569s 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f0133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f006f020 0x7fc1f01cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1f0071730 0x7fc1f0133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f0133e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:38186/0 (socket says 172.21.15.47:38186) 2024-04-20T02:24:58.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f006f020 0x7fc1f01cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:57720/0 (socket says 172.21.15.47:57720) 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 1 -- 172.21.15.47:0/2264126048 learned_addr learned my addr 172.21.15.47:0/2264126048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): get_auth_request con 0x7fc1f00a2db0 auth_method 0 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): get_auth_request con 0x7fc1f006f020 auth_method 0 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): get_auth_request con 0x7fc1f0071730 auth_method 0 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1eed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.327+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 10 monclient(hunting): handle_auth_done global_id 22350 payload 824 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 msgr2=0x7fc1f0133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f0133e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f006f020 msgr2=0x7fc1f01cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f006f020 0x7fc1f01cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 monclient: found mon.noname-g 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0009550 con 0x7fc1f0071730 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1eed9d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f00a2db0 0x7fc1f0133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e5d9b700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1f0071730 0x7fc1f0133900 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800e8d0 tx=0x7fc1d800ebe0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 client.?.objecter ms_handle_connect 0x7fc1f0071730 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1d8013af0 con 0x7fc1f0071730 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:24:58.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: dump: 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.358 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d8013cb0 con 0x7fc1f0071730 2024-04-20T02:24:58.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 5 monclient: authenticate success, global_id 22350 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 10 monclient: _renew_subs 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f0125fd0 con 0x7fc1f0071730 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1d8013e20 con 0x7fc1f0071730 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1ed59a700 4 set_mon_vals no callback set 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 10 client.22350.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1e77fe700 10 monclient: dump: 2024-04-20T02:24:58.360 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.361 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 10 monclient: _renew_subs 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f0126dc0 con 0x7fc1f0071730 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.328+0000 7fc1f4e1d700 1 librados: init done 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1f4e1d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 0x7fc1f0124440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0068280 con 0x7fc1f006aac0 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 0x7fc1f0124440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient: get_auth_request con 0x7fc1f006aac0 auth_method 0 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc1d8013680 con 0x7fc1f0071730 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_done global_id 35369 payload 824 2024-04-20T02:24:58.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1ee59c700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1f006bb30 con 0x7fc1f006aac0 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1e77fe700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d003a2f0 0x7fc1d003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(356..356 src has 206..356) v4 ==== 5876+0+0 (secure 0 0 0) 0x7fc1d804a5c0 con 0x7fc1f0071730 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1e77fe700 10 client.22350.objecter ms_dispatch 0x7fc1f0001550 osd_map(356..356 src has 206..356) v4 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1e77fe700 3 client.22350.objecter handle_osd_map got epochs [356,356] > 0 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.329+0000 7fc1e77fe700 3 client.22350.objecter handle_osd_map decoding full epoch 356 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 20 client.22350.objecter dump_active .. 0 homeless 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 client.22350.objecter ms_handle_connect 0x7fc1f006aac0 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 client.22350.objecter resend_mon_ops 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 0x7fc1f0124440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009920 tx=0x7fc1e0004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1eed9d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d003a2f0 0x7fc1d003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1eed9d700 10 monclient: get_auth_request con 0x7fc1d003a2f0 auth_method 0 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0020070 con 0x7fc1f006aac0 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 monclient: dump: 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.365 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc1e0004440 con 0x7fc1f006aac0 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1eed9d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d003a2f0 0x7fc1d003c7b0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0126b30 tx=0x7fc1e800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 msgr2=0x7fc1f0124440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 0x7fc1f0124440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009920 tx=0x7fc1e0004750 comp rx=0 tx=0).stop 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 msgr2=0x7fc1f0124440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f006aac0 0x7fc1f0124440 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009920 tx=0x7fc1e0004750 comp rx=0 tx=0).stop 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.330+0000 7fc1e77fe700 10 client.22350.objecter ms_handle_connect 0x7fc1d003a2f0 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1f4e1d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 0x7fc1f0124d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1f4e1d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006a400 con 0x7fc1f0124980 2024-04-20T02:24:58.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 0x7fc1f0124d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient: get_auth_request con 0x7fc1f0124980 auth_method 0 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 10 monclient(hunting): handle_auth_done global_id 35378 payload 824 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.338+0000 7fc1ee59c700 1 -- 172.21.15.47:0/2264126048 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc1f0071090 con 0x7fc1f0124980 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 client.22350.objecter ms_handle_connect 0x7fc1f0124980 2024-04-20T02:24:58.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 client.22350.objecter resend_mon_ops 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1ee59c700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 0x7fc1f0124d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009c70 tx=0x7fc1e0011000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e002baf0 con 0x7fc1f0124980 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 monclient: dump: 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.369 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.370 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7fc1e002bc50 con 0x7fc1f0124980 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 msgr2=0x7fc1f0124d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 0x7fc1f0124d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009c70 tx=0x7fc1e0011000 comp rx=0 tx=0).stop 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 msgr2=0x7fc1f0124d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1e77fe700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc1f0124980 0x7fc1f0124d90 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009c70 tx=0x7fc1e0011000 comp rx=0 tx=0).stop 2024-04-20T02:24:58.371 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:58.372 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.373 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:58.375 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:58.376 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:58.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.382 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:58.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:58.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:58.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.391 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:58.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.395 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.396 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:58.398 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.399 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.400 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:58.400 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:58.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 10 librados: watch_flush enter 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 10 librados: watch_flush exit 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 20 client.22350.objecter shutdown clearing up homeless session... 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 10 client.22350.objecter successfully canceled tick 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d003a2f0 msgr2=0x7fc1d003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d003a2f0 0x7fc1d003c7b0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0126b30 tx=0x7fc1e800b040 comp rx=0 tx=0).stop 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 10 monclient: shutdown 2024-04-20T02:24:58.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1f0071730 msgr2=0x7fc1f0133900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1eed9d700 1 -- 172.21.15.47:0/2264126048 reap_dead start 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.339+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1f0071730 0x7fc1f0133900 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800e8d0 tx=0x7fc1d800ebe0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.340+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 shutdown_connections 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.340+0000 7fc1f4e1d700 1 --2- 172.21.15.47:0/2264126048 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc1f0071730 0x7fc1f0133900 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.340+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 >> 172.21.15.47:0/2264126048 conn(0x7fc1f00a3f90 msgr2=0x7fc1f00701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.340+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 shutdown_connections 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.340+0000 7fc1f4e1d700 1 -- 172.21.15.47:0/2264126048 wait complete. 2024-04-20T02:24:58.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.340+0000 7fc1f4e1d700 1 librados: shutdown 2024-04-20T02:24:58.405 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:58.405 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-04-20T02:24:58.405 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:24:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.543+0000 7fcdb1ca7700 10 monclient: get_monmap_and_config 2024-04-20T02:24:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.543+0000 7fcdb1ca7700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.543+0000 7fcdb1ca7700 10 monclient: monmap: 2024-04-20T02:24:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:58.544982+0000 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:58.544982+0000 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:58.546 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.544+0000 7fcdb1ca7700 1 Processor -- start 2024-04-20T02:24:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.544+0000 7fcdb1ca7700 1 -- start start 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.544+0000 7fcdb1ca7700 10 monclient: init 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient: _add_conns ranks=[10,9,5,4,6,7,2,0,1,8,3] 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcdac0a50f0 0x7fcdac137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): picked mon.noname-f con 0x7fcdac0a50f0 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdac1378a0 0x7fcdac138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): picked mon.noname-a con 0x7fcdac1378a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcdac0a38f0 0x7fcdac0a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): picked mon.noname-h con 0x7fcdac0a38f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac0a54d0 con 0x7fcdac1378a0 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac1387b0 con 0x7fcdac0a50f0 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac06da20 con 0x7fcdac0a38f0 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdb1ca7700 10 monclient(hunting): authenticate will time out at 2177.946369s 2024-04-20T02:24:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdac1378a0 0x7fcdac138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdac1378a0 0x7fcdac138270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53478/0 (socket says 172.21.15.47:53478) 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 -- 172.21.15.47:0/3096623165 learned_addr learned my addr 172.21.15.47:0/3096623165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): get_auth_request con 0x7fcdac1378a0 auth_method 0 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdab7fe700 1 --2- 172.21.15.47:0/3096623165 >> v2:172.21.15.179:3300/0 conn(0x7fcdac0a50f0 0x7fcdac137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdabfff700 1 --2- 172.21.15.47:0/3096623165 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcdac0a38f0 0x7fcdac0a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdabfff700 10 monclient(hunting): get_auth_request con 0x7fcdac0a38f0 auth_method 0 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdabfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdabfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdabfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdab7fe700 10 monclient(hunting): get_auth_request con 0x7fcdac0a50f0 auth_method 0 2024-04-20T02:24:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_done global_id 35901 payload 274 2024-04-20T02:24:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 -- 172.21.15.47:0/3096623165 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcdac0a38f0 msgr2=0x7fcdac0a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 --2- 172.21.15.47:0/3096623165 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcdac0a38f0 0x7fcdac0a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 -- 172.21.15.47:0/3096623165 >> v2:172.21.15.179:3300/0 conn(0x7fcdac0a50f0 msgr2=0x7fcdac137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 --2- 172.21.15.47:0/3096623165 >> v2:172.21.15.179:3300/0 conn(0x7fcdac0a50f0 0x7fcdac137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 monclient: found mon.noname-a 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.545+0000 7fcdaaffd700 1 -- 172.21.15.47:0/3096623165 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0751e0 con 0x7fcdac1378a0 2024-04-20T02:24:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaaffd700 1 --2- 172.21.15.47:0/3096623165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdac1378a0 0x7fcdac138270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a8ed0 tx=0x7fcd94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c69062274aa9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 1 -- 172.21.15.47:0/3096623165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9400bae0 con 0x7fcdac1378a0 2024-04-20T02:24:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: dump: 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 5 monclient: authenticate success, global_id 35901 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 1 -- 172.21.15.47:0/3096623165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9400bc40 con 0x7fcdac1378a0 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 1 -- 172.21.15.47:0/3096623165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9400bdb0 con 0x7fcdac1378a0 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 4 set_mon_vals no callback set 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdaa7fc700 10 monclient: dump: 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 10 monclient: shutdown 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/3096623165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdac1378a0 msgr2=0x7fcdac138270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/3096623165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdac1378a0 0x7fcdac138270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a8ed0 tx=0x7fcd94009bd0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/3096623165 shutdown_connections 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/3096623165 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcdac0a38f0 0x7fcdac0a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/3096623165 >> v2:172.21.15.179:3300/0 conn(0x7fcdac0a50f0 0x7fcdac137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/3096623165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdac1378a0 0x7fcdac138270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/3096623165 >> 172.21.15.47:0/3096623165 conn(0x7fcdac0a8510 msgr2=0x7fcdac0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/3096623165 shutdown_connections 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.546+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/3096623165 wait complete. 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.547+0000 7fcdb1ca7700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.547+0000 7fcdb1ca7700 10 monclient: monmap: 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 1 librados: starting msgr at 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 1 librados: starting objecter 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 1 Processor -- start 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 1 -- start start 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 1 librados: setting wanted keys 2024-04-20T02:24:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 1 librados: calling monclient init 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.548+0000 7fcdb1ca7700 10 monclient: init 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient: _add_conns ranks=[1,6,2,7,8,4,5,3,0] 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac0a38f0 0x7fcdac1cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): picked mon.noname-f con 0x7fcdac0a38f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 0x7fcdac136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): picked mon.noname-d con 0x7fcdac0a50f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcdac1371d0 0x7fcdac1c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): picked mon.noname-b con 0x7fcdac1371d0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0722d0 con 0x7fcdac0a38f0 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac124090 con 0x7fcdac1371d0 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac071e80 con 0x7fcdac0a50f0 2024-04-20T02:24:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdb1ca7700 10 monclient(hunting): authenticate will time out at 2177.950331s 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 0x7fcdac136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcdac1371d0 0x7fcdac1c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 0x7fcdac136c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51260/0 (socket says 172.21.15.47:51260) 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 -- 172.21.15.47:0/2232551006 learned_addr learned my addr 172.21.15.47:0/2232551006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): get_auth_request con 0x7fcdac0a50f0 auth_method 0 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): get_auth_request con 0x7fcdac1371d0 auth_method 0 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac0a38f0 0x7fcdac1cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdabfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): get_auth_request con 0x7fcdac0a38f0 auth_method 0 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient(hunting): handle_auth_done global_id 54528 payload 824 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcdac1371d0 msgr2=0x7fcdac1c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcdac1371d0 0x7fcdac1c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac0a38f0 msgr2=0x7fcdac1cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac0a38f0 0x7fcdac1cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 monclient: found mon.noname-d 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.549+0000 7fcdaaffd700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94009550 con 0x7fcdac0a50f0 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdaaffd700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 0x7fcdac136c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd94013000 tx=0x7fcd94004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 client.?.objecter ms_handle_connect 0x7fcdac0a50f0 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9401bab0 con 0x7fcdac0a50f0 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: dump: 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9401bc70 con 0x7fcdac0a50f0 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 5 monclient: authenticate success, global_id 54528 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 10 monclient: _renew_subs 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac123db0 con 0x7fcdac0a50f0 2024-04-20T02:24:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9401bde0 con 0x7fcdac0a50f0 2024-04-20T02:24:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda3fff700 10 monclient: dump: 2024-04-20T02:24:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.589 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcda9ffb700 4 set_mon_vals no callback set 2024-04-20T02:24:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 10 client.54528.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 10 monclient: _renew_subs 2024-04-20T02:24:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac123070 con 0x7fcdac0a50f0 2024-04-20T02:24:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.550+0000 7fcdb1ca7700 1 librados: init done 2024-04-20T02:24:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdb1ca7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 0x7fcdac1c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0682f0 con 0x7fcdac1378a0 2024-04-20T02:24:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcd94021070 con 0x7fcdac0a50f0 2024-04-20T02:24:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 0x7fcdac1c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient: get_auth_request con 0x7fcdac1378a0 auth_method 0 2024-04-20T02:24:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcd8c03a220 0x7fcd8c03c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(356..356 src has 206..356) v4 ==== 5876+0+0 (secure 0 0 0) 0x7fcd9404b980 con 0x7fcdac0a50f0 2024-04-20T02:24:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 10 client.54528.objecter ms_dispatch 0x7fcdac001550 osd_map(356..356 src has 206..356) v4 2024-04-20T02:24:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 3 client.54528.objecter handle_osd_map got epochs [356,356] > 0 2024-04-20T02:24:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 3 client.54528.objecter handle_osd_map decoding full epoch 356 2024-04-20T02:24:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_done global_id 54793 payload 824 2024-04-20T02:24:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdac071330 con 0x7fcdac1378a0 2024-04-20T02:24:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 20 client.54528.objecter dump_active .. 0 homeless 2024-04-20T02:24:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdabfff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcd8c03a220 0x7fcd8c03c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdabfff700 10 monclient: get_auth_request con 0x7fcd8c03a220 auth_method 0 2024-04-20T02:24:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 10 client.54528.objecter ms_handle_connect 0x7fcdac1378a0 2024-04-20T02:24:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcdab7fe700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 0x7fcdac1c91a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009fd0 tx=0x7fcd9c011960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.551+0000 7fcda3fff700 10 client.54528.objecter resend_mon_ops 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9c00c070 con 0x7fcdac1378a0 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 10 monclient: dump: 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.597 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 10 client.54528.objecter ms_handle_connect 0x7fcd8c03a220 2024-04-20T02:24:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcdabfff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcd8c03a220 0x7fcd8c03c6e0 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7fcdac123020 tx=0x7fcda4009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcd9c015670 con 0x7fcdac1378a0 2024-04-20T02:24:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 msgr2=0x7fcdac1c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 0x7fcdac1c91a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009fd0 tx=0x7fcd9c011960 comp rx=0 tx=0).stop 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 msgr2=0x7fcdac1c91a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.552+0000 7fcda3fff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac1378a0 0x7fcdac1c91a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdb1ca7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 0x7fcdac1c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdb1ca7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac06a400 con 0x7fcdac06aac0 2024-04-20T02:24:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 0x7fcdac1c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient: get_auth_request con 0x7fcdac06aac0 auth_method 0 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 10 monclient(hunting): handle_auth_done global_id 54802 payload 824 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.560+0000 7fcdab7fe700 1 -- 172.21.15.47:0/2232551006 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcdac06e370 con 0x7fcdac06aac0 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcdab7fe700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 0x7fcdac1c96e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0096b0 tx=0x7fcd9c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 client.54528.objecter ms_handle_connect 0x7fcdac06aac0 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 client.54528.objecter resend_mon_ops 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9c024070 con 0x7fcdac06aac0 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 monclient: dump: 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.602 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.603 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.604 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7fcd9c01fca0 con 0x7fcdac06aac0 2024-04-20T02:24:58.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 msgr2=0x7fcdac1c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 0x7fcdac1c96e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0096b0 tx=0x7fcd9c004750 comp rx=0 tx=0).stop 2024-04-20T02:24:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 msgr2=0x7fcdac1c96e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcda3fff700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdac06aac0 0x7fcdac1c96e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.606 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:58.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:58.606 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:58.606 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:58.606 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:58.607 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.608 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.609 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:58.610 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.611 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:58.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.614 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.620 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.625 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.626 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.627 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:24:58.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.627 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.627 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:58.627 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.561+0000 7fcdb1ca7700 10 librados: watch_flush enter 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 10 librados: watch_flush exit 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 20 client.54528.objecter shutdown clearing up homeless session... 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 10 client.54528.objecter successfully canceled tick 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcd8c03a220 msgr2=0x7fcd8c03c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcd8c03a220 0x7fcd8c03c6e0 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7fcdac123020 tx=0x7fcda4009780 comp rx=0 tx=0).stop 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 10 monclient: shutdown 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 msgr2=0x7fcdac136c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 0x7fcdac136c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd94013000 tx=0x7fcd94004750 comp rx=0 tx=0).stop 2024-04-20T02:24:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdabfff700 1 -- 172.21.15.47:0/2232551006 reap_dead start 2024-04-20T02:24:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 shutdown_connections 2024-04-20T02:24:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 --2- 172.21.15.47:0/2232551006 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcdac0a50f0 0x7fcdac136c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 >> 172.21.15.47:0/2232551006 conn(0x7fcdac0a8510 msgr2=0x7fcdac070450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 shutdown_connections 2024-04-20T02:24:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 -- 172.21.15.47:0/2232551006 wait complete. 2024-04-20T02:24:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.562+0000 7fcdb1ca7700 1 librados: shutdown 2024-04-20T02:24:58.630 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:58.630 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-04-20T02:24:58.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.770+0000 7ff684303700 10 monclient: get_monmap_and_config 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.770+0000 7ff684303700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.770+0000 7ff684303700 10 monclient: monmap: 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:58.771264+0000 2024-04-20T02:24:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:58.771264+0000 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:58.771 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.770+0000 7ff684303700 1 Processor -- start 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.770+0000 7ff684303700 1 -- start start 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.770+0000 7ff684303700 10 monclient: init 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient: _add_conns ranks=[6,5,0,7,9,8,3,4,10,2,1] 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff67c0a1510 0x7ff67c0a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): picked mon.noname-d con 0x7ff67c0a1510 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff67c09f9a0 0x7ff67c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): picked mon.noname-h con 0x7ff67c09f9a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff67c09fe00 addr v1:172.21.15.47:6789/0 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0751e0 con 0x7ff67c09fe00 2024-04-20T02:24:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0720a0 con 0x7ff67c09f9a0 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c070640 con 0x7ff67c0a1510 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff684303700 10 monclient(hunting): authenticate will time out at 2178.172537s 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff67c0a1510 0x7ff67c0a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff6828a0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff67c09fe00 0x7ff67c0a3e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:51256/0 (socket says 172.21.15.47:51256) 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff6828a0700 1 -- 172.21.15.47:0/2265203154 learned_addr learned my addr 172.21.15.47:0/2265203154 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): get_auth_request con 0x7ff67c0a1510 auth_method 0 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68189e700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff67c09f9a0 0x7ff67c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68189e700 10 monclient(hunting): get_auth_request con 0x7ff67c09f9a0 auth_method 0 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68189e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68189e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68189e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.771+0000 7ff68209f700 10 monclient(hunting): handle_auth_done global_id 54537 payload 274 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 1 -- 172.21.15.47:0/2265203154 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff67c09f9a0 msgr2=0x7ff67c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff67c09f9a0 0x7ff67c0a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 1 -- 172.21.15.47:0/2265203154 >> v1:172.21.15.47:6789/0 conn(0x7ff67c09fe00 legacy=0x7ff67c0a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 1 -- 172.21.15.47:0/2265203154 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639562384 0 0) 0x7ff67c0751e0 con 0x7ff67c09fe00 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 1 monclient: found mon.noname-d 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:58.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 1 -- 172.21.15.47:0/2265203154 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c071d30 con 0x7ff67c0a1510 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68189e700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff67c09f9a0 0x7ff67c0a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68209f700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff67c0a1510 0x7ff67c0a18f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff678009870 tx=0x7ff678009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d22ac7fb24b6a79f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 1 -- 172.21.15.47:0/2265203154 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff678011aa0 con 0x7ff67c0a1510 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: dump: 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.786 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 5 monclient: authenticate success, global_id 54537 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:58.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 1 -- 172.21.15.47:0/2265203154 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678011c00 con 0x7ff67c0a1510 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 1 -- 172.21.15.47:0/2265203154 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff678011d70 con 0x7ff67c0a1510 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 4 set_mon_vals no callback set 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff68109d700 10 monclient: dump: 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 10 monclient: shutdown 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 1 -- 172.21.15.47:0/2265203154 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff67c0a1510 msgr2=0x7ff67c0a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.772+0000 7ff684303700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff67c0a1510 0x7ff67c0a18f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff678009870 tx=0x7ff678009b80 comp rx=0 tx=0).stop 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 1 -- 172.21.15.47:0/2265203154 shutdown_connections 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff67c0a1510 0x7ff67c0a18f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 1 --2- 172.21.15.47:0/2265203154 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff67c09f9a0 0x7ff67c0a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 1 -- 172.21.15.47:0/2265203154 >> 172.21.15.47:0/2265203154 conn(0x7ff67c0a8510 msgr2=0x7ff67c0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 1 -- 172.21.15.47:0/2265203154 shutdown_connections 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 1 -- 172.21.15.47:0/2265203154 wait complete. 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 10 monclient: build_initial_monmap 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.773+0000 7ff684303700 10 monclient: monmap: 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:58.790 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 1 librados: starting msgr at 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 1 librados: starting objecter 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 1 Processor -- start 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 1 -- start start 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 1 librados: setting wanted keys 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 1 librados: calling monclient init 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.774+0000 7ff684303700 10 monclient: init 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:58.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient: _add_conns ranks=[0,7,8,6,3,4,5,2,1] 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff67c09f9a0 0x7ff67c1ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): picked mon.noname-a con 0x7ff67c09f9a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff67c09fe00 0x7ff67c1359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): picked mon.noname-i con 0x7ff67c09fe00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff67c0a1510 0x7ff67c135f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): picked mon.noname-e con 0x7ff67c0a1510 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c1cad60 con 0x7ff67c09f9a0 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c122b10 con 0x7ff67c09fe00 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c121a50 con 0x7ff67c0a1510 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff684303700 10 monclient(hunting): authenticate will time out at 2178.176330s 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff67c09f9a0 0x7ff67c1ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff67c09f9a0 0x7ff67c1ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53486/0 (socket says 172.21.15.47:53486) 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 -- 172.21.15.47:0/3182020025 learned_addr learned my addr 172.21.15.47:0/3182020025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): get_auth_request con 0x7ff67c09f9a0 auth_method 0 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff67c0a1510 0x7ff67c135f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): get_auth_request con 0x7ff67c0a1510 auth_method 0 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68189e700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff67c09fe00 0x7ff67c1359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff6828a0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68189e700 10 monclient(hunting): get_auth_request con 0x7ff67c09fe00 auth_method 0 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68189e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68189e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68189e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient(hunting): handle_auth_done global_id 35919 payload 824 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff67c0a1510 msgr2=0x7ff67c135f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff67c0a1510 0x7ff67c135f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff67c09fe00 msgr2=0x7ff67c1359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff67c09fe00 0x7ff67c1359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient: _finish_hunting 0 2024-04-20T02:24:58.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 monclient: found mon.noname-a 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780094a0 con 0x7ff67c09f9a0 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff68209f700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff67c09f9a0 0x7ff67c1ca020 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff678014040 tx=0x7ff678004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff672ffd700 10 client.?.objecter ms_handle_connect 0x7ff67c09f9a0 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.775+0000 7ff672ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff678004330 con 0x7ff67c09f9a0 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: dump: 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.795 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: _finish_auth 0 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67801f070 con 0x7ff67c09f9a0 2024-04-20T02:24:58.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 5 monclient: authenticate success, global_id 35919 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 10 monclient: _renew_subs 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c071040 con 0x7ff67c09f9a0 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6780044d0 con 0x7ff67c09f9a0 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff672ffd700 10 monclient: dump: 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.797 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff68089c700 4 set_mon_vals no callback set 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 10 client.35919.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 10 monclient: _renew_subs 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c1233b0 con 0x7ff67c09f9a0 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 1 librados: init done 2024-04-20T02:24:58.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.776+0000 7ff684303700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff684303700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 0x7ff67c11f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0682f0 con 0x7ff67c06aac0 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff678021030 con 0x7ff67c09f9a0 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 0x7ff67c11f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient: get_auth_request con 0x7ff67c06aac0 auth_method 0 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff66003a2f0 0x7ff66003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(356..356 src has 206..356) v4 ==== 5876+0+0 (secure 0 0 0) 0x7ff67804b2f0 con 0x7ff67c09f9a0 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 10 client.35919.objecter ms_dispatch 0x7ff67c001550 osd_map(356..356 src has 206..356) v4 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 3 client.35919.objecter handle_osd_map got epochs [356,356] > 0 2024-04-20T02:24:58.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 3 client.35919.objecter handle_osd_map decoding full epoch 356 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 20 client.35919.objecter dump_active .. 0 homeless 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff6828a0700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff66003a2f0 0x7ff66003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff6828a0700 10 monclient: get_auth_request con 0x7ff66003a2f0 auth_method 0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 10 monclient(hunting): handle_auth_done global_id 22359 payload 824 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff67c122c80 con 0x7ff67c06aac0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 10 client.35919.objecter ms_handle_connect 0x7ff67c06aac0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff672ffd700 10 client.35919.objecter resend_mon_ops 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.777+0000 7ff68189e700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 0x7ff67c11f420 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0a8fa0 tx=0x7ff66c00f900 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 10 client.35919.objecter ms_handle_connect 0x7ff66003a2f0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff6828a0700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff66003a2f0 0x7ff66003c7b0 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 crypto rx=0x7ff674001d10 tx=0x7ff674001ca0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff66c014ae0 con 0x7ff67c06aac0 2024-04-20T02:24:58.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 10 monclient: dump: 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.801 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff66c00c050 con 0x7ff67c06aac0 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 msgr2=0x7ff67c11f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 0x7ff67c11f420 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0a8fa0 tx=0x7ff66c00f900 comp rx=0 tx=0).stop 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 msgr2=0x7ff67c11f420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.778+0000 7ff672ffd700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c06aac0 0x7ff67c11f420 secure :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0a8fa0 tx=0x7ff66c00f900 comp rx=0 tx=0).stop 2024-04-20T02:24:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff684303700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff684303700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 0x7ff67c11fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff684303700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c06a400 con 0x7ff67c11f960 2024-04-20T02:24:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 0x7ff67c11fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:58.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient: get_auth_request con 0x7ff67c11f960 auth_method 0 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.786+0000 7ff68189e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff68189e700 10 monclient(hunting): handle_auth_done global_id 22368 payload 824 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff68189e700 1 -- 172.21.15.47:0/3182020025 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff67c0a1e30 con 0x7ff67c11f960 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff68189e700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 0x7ff67c11fd70 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01a000 tx=0x7ff66c0047c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 client.35919.objecter ms_handle_connect 0x7ff67c11f960 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 client.35919.objecter resend_mon_ops 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff66c014970 con 0x7ff67c11f960 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 monclient: dump: 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7ff66c024070 con 0x7ff67c11f960 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 msgr2=0x7ff67c11fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 0x7ff67c11fd70 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01a000 tx=0x7ff66c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 msgr2=0x7ff67c11fd70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.787+0000 7ff672ffd700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff67c11f960 0x7ff67c11fd70 secure :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01a000 tx=0x7ff66c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:58.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:24:58.808 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.809 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.810 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:58.811 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:58.812 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.813 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.820 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:58.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:58.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:58.829 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 10 librados: watch_flush enter 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 10 librados: watch_flush exit 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 20 client.35919.objecter shutdown clearing up homeless session... 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 10 client.35919.objecter successfully canceled tick 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff66003a2f0 msgr2=0x7ff66003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff66003a2f0 0x7ff66003c7b0 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 crypto rx=0x7ff674001d10 tx=0x7ff674001ca0 comp rx=0 tx=0).stop 2024-04-20T02:24:58.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 10 monclient: shutdown 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff67c09f9a0 msgr2=0x7ff67c1ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 --2- 172.21.15.47:0/3182020025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff67c09f9a0 0x7ff67c1ca020 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff678014040 tx=0x7ff678004750 comp rx=0 tx=0).stop 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff6828a0700 1 -- 172.21.15.47:0/3182020025 reap_dead start 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff6828a0700 1 -- 172.21.15.47:0/3182020025 reap_dead start 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 shutdown_connections 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 >> 172.21.15.47:0/3182020025 conn(0x7ff67c0a8510 msgr2=0x7ff67c0a4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 shutdown_connections 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 -- 172.21.15.47:0/3182020025 wait complete. 2024-04-20T02:24:58.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.788+0000 7ff684303700 1 librados: shutdown 2024-04-20T02:24:58.832 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:58.833 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-04-20T02:24:58.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:24:58.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.971+0000 7f76ef2fc700 10 monclient: get_monmap_and_config 2024-04-20T02:24:59.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.971+0000 7f76ef2fc700 10 monclient: build_initial_monmap 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.971+0000 7f76ef2fc700 10 monclient: monmap: 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:58.972455+0000 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:58.972455+0000 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.033 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:59.034 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.971+0000 7f76ef2fc700 1 Processor -- start 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 -- start start 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient: init 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient: _add_conns ranks=[5,7,10,8,6,0,4,2,9,1,3] 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809c750 0x7f76e8134d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): picked mon.noname-h con 0x7f76e809c750 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 0x7f76e80a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): picked mon.noname-i con 0x7f76e809cb30 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76e809cf30 0x7f76e80a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): picked mon.noname-f con 0x7f76e809cf30 addr v2:172.21.15.179:3300/0 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e81357b0 con 0x7f76e809cf30 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76e809fd40 con 0x7f76e809c750 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80a3df0 con 0x7f76e809cb30 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.972+0000 7f76ef2fc700 10 monclient(hunting): authenticate will time out at 2178.373709s 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809c750 0x7f76e8134d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 0x7f76e80a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76e809cf30 0x7f76e80a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 0x7f76e80a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:38200/0 (socket says 172.21.15.47:38200) 2024-04-20T02:24:59.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 -- 172.21.15.47:0/1641071138 learned_addr learned my addr 172.21.15.47:0/1641071138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): get_auth_request con 0x7f76e809cb30 auth_method 0 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): get_auth_request con 0x7f76e809cf30 auth_method 0 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): get_auth_request con 0x7f76e809c750 auth_method 0 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient(hunting): handle_auth_done global_id 54547 payload 274 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 -- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809c750 msgr2=0x7f76e8134d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 --2- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809c750 0x7f76e8134d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 -- 172.21.15.47:0/1641071138 >> v2:172.21.15.179:3300/0 conn(0x7f76e809cf30 msgr2=0x7f76e80a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 --2- 172.21.15.47:0/1641071138 >> v2:172.21.15.179:3300/0 conn(0x7f76e809cf30 0x7f76e80a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient: _finish_hunting 0 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 monclient: found mon.noname-i 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 -- 172.21.15.47:0/1641071138 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80751e0 con 0x7f76e809cb30 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ed899700 1 --2- 172.21.15.47:0/1641071138 >> v2:172.21.15.179:3300/0 conn(0x7f76e809cf30 0x7f76e80a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ec897700 1 --2- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 0x7f76e80a1f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76d800bb90 tx=0x7f76d800bea0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=d8d516b3b83a1af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 1 -- 172.21.15.47:0/1641071138 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76d8010470 con 0x7f76e809cb30 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:24:59.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: dump: 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.055 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: _finish_auth 0 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: _check_auth_tickets 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 1 -- 172.21.15.47:0/1641071138 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d80105d0 con 0x7f76e809cb30 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ef2fc700 5 monclient: authenticate success, global_id 54547 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ef2fc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ef2fc700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 1 -- 172.21.15.47:0/1641071138 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76d8010bb0 con 0x7f76e809cb30 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76ef2fc700 4 set_mon_vals no callback set 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.973+0000 7f76dffff700 10 monclient: dump: 2024-04-20T02:24:59.056 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 10 monclient: shutdown 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 msgr2=0x7f76e80a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 0x7f76e80a1f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f76d800bb90 tx=0x7f76d800bea0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1641071138 shutdown_connections 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1641071138 >> v2:172.21.15.179:3300/0 conn(0x7f76e809cf30 0x7f76e80a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809c750 0x7f76e8134d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1641071138 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76e809cb30 0x7f76e80a1f10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1641071138 >> 172.21.15.47:0/1641071138 conn(0x7f76e80a3000 msgr2=0x7f76e80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1641071138 shutdown_connections 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.974+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1641071138 wait complete. 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 10 monclient: build_initial_monmap 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 10 monclient: monmap: 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:59.058 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:59.059 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 1 librados: starting msgr at 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 1 librados: starting objecter 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 1 Processor -- start 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 1 -- start start 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 1 librados: setting wanted keys 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 1 librados: calling monclient init 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.975+0000 7f76ef2fc700 10 monclient: init 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient: _add_conns ranks=[6,5,3,7,2,8,4,1,0] 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76e809c750 0x7f76e8133d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): picked mon.noname-d con 0x7f76e809c750 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809cb30 0x7f76e81c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): picked mon.noname-h con 0x7f76e809cb30 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76e809cf30 0x7f76e81c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): picked mon.noname-g con 0x7f76e809cf30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e812f500 con 0x7f76e809cf30 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8130210 con 0x7f76e809cb30 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76e81304f0 con 0x7f76e809c750 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ef2fc700 10 monclient(hunting): authenticate will time out at 2178.377300s 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76e809c750 0x7f76e8133d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76e809c750 0x7f76e8133d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51290/0 (socket says 172.21.15.47:51290) 2024-04-20T02:24:59.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 1 -- 172.21.15.47:0/1469356806 learned_addr learned my addr 172.21.15.47:0/1469356806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): get_auth_request con 0x7f76e809c750 auth_method 0 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809cb30 0x7f76e81c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76e809cf30 0x7f76e81c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): get_auth_request con 0x7f76e809cb30 auth_method 0 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): get_auth_request con 0x7f76e809cf30 auth_method 0 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed899700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient(hunting): handle_auth_done global_id 64301 payload 824 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76e809c750 msgr2=0x7f76e8133d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76e809c750 0x7f76e8133d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76e809cf30 msgr2=0x7f76e81c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76e809cf30 0x7f76e81c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient: _finish_hunting 0 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 monclient: found mon.noname-h 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ec897700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009a70 con 0x7f76e809cb30 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.976+0000 7f76ed098700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76e809c750 0x7f76e8133d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ec897700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809cb30 0x7f76e81c4920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76d8018000 tx=0x7f76d8004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 client.?.objecter ms_handle_connect 0x7f76e809cb30 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76d801f070 con 0x7f76e809cb30 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: dump: 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.064 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:59.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8010dc0 con 0x7f76e809cb30 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 5 monclient: authenticate success, global_id 64301 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 10 monclient: _renew_subs 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d800b7f0 con 0x7f76e809cb30 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76d8010470 con 0x7f76e809cb30 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ddffb700 10 monclient: dump: 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.066 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.067 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 10 client.64301.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 10 monclient: _renew_subs 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76df7fe700 4 set_mon_vals no callback set 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e81c53a0 con 0x7f76e809cb30 2024-04-20T02:24:59.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.977+0000 7f76ef2fc700 1 librados: init done 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ef2fc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 0x7f76e81c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8130010 con 0x7f76e806aac0 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f76d8010950 con 0x7f76e809cb30 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 0x7f76e81c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient: get_auth_request con 0x7f76e806aac0 auth_method 0 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76cc03a2f0 0x7f76cc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(356..356 src has 206..356) v4 ==== 5876+0+0 (secure 0 0 0) 0x7f76d804df50 con 0x7f76e809cb30 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 10 client.64301.objecter ms_dispatch 0x7f76e8001550 osd_map(356..356 src has 206..356) v4 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 3 client.64301.objecter handle_osd_map got epochs [356,356] > 0 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 3 client.64301.objecter handle_osd_map decoding full epoch 356 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 10 monclient(hunting): handle_auth_done global_id 64310 payload 824 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed098700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76e8130380 con 0x7f76e806aac0 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ddffb700 20 client.64301.objecter dump_active .. 0 homeless 2024-04-20T02:24:59.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed899700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76cc03a2f0 0x7f76cc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.978+0000 7f76ed899700 10 monclient: get_auth_request con 0x7f76cc03a2f0 auth_method 0 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 client.64301.objecter ms_handle_connect 0x7f76e806aac0 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 client.64301.objecter resend_mon_ops 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ed098700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 0x7f76e81c7e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76e4011db0 tx=0x7f76e400f520 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76e400fe20 con 0x7f76e806aac0 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 monclient: dump: 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.071 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.072 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 client.64301.objecter ms_handle_connect 0x7f76cc03a2f0 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ed899700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76cc03a2f0 0x7f76cc03c7b0 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f76e80a0ad0 tx=0x7f76e000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f76e4014440 con 0x7f76e806aac0 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:59.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 msgr2=0x7f76e81c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 0x7f76e81c7e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76e4011db0 tx=0x7f76e400f520 comp rx=0 tx=0).stop 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 msgr2=0x7f76e81c7e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.979+0000 7f76ddffb700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e806aac0 0x7f76e81c7e80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ef2fc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 0x7f76e81c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ef2fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76e8068180 con 0x7f76e8134260 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 0x7f76e81c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient: get_auth_request con 0x7f76e8134260 auth_method 0 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.987+0000 7f76ed098700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ed098700 10 monclient(hunting): handle_auth_done global_id 64319 payload 824 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ed098700 1 -- 172.21.15.47:0/1469356806 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76e80a0300 con 0x7f76e8134260 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 client.64301.objecter ms_handle_connect 0x7f76e8134260 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 client.64301.objecter resend_mon_ops 2024-04-20T02:24:59.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ed098700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 0x7f76e81c83c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76e4011db0 tx=0x7f76e4004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76e402b070 con 0x7f76e8134260 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 monclient: dump: 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.076 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.077 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7f76e4023040 con 0x7f76e8134260 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 msgr2=0x7f76e81c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 0x7f76e81c83c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76e4011db0 tx=0x7f76e4004750 comp rx=0 tx=0).stop 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 msgr2=0x7f76e81c83c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:59.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.988+0000 7f76ddffb700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e8134260 0x7f76e81c83c0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f76e4011db0 tx=0x7f76e4004750 comp rx=0 tx=0).stop 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:59.079 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:59.080 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:59.081 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:59.082 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:59.083 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.084 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.085 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.086 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.087 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.089 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:59.091 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:59.094 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.096 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:59.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:59.100 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.101 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:59.102 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 10 librados: watch_flush enter 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 10 librados: watch_flush exit 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 20 client.64301.objecter shutdown clearing up homeless session... 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 10 client.64301.objecter successfully canceled tick 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76cc03a2f0 msgr2=0x7f76cc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76cc03a2f0 0x7f76cc03c7b0 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 crypto rx=0x7f76e80a0ad0 tx=0x7f76e000b040 comp rx=0 tx=0).stop 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 10 monclient: shutdown 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809cb30 msgr2=0x7f76e81c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ed899700 1 -- 172.21.15.47:0/1469356806 reap_dead start 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809cb30 0x7f76e81c4920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f76d8018000 tx=0x7f76d8004750 comp rx=0 tx=0).stop 2024-04-20T02:24:59.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 shutdown_connections 2024-04-20T02:24:59.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 --2- 172.21.15.47:0/1469356806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76e809cb30 0x7f76e81c4920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 >> 172.21.15.47:0/1469356806 conn(0x7f76e80a3000 msgr2=0x7f76e8099430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:59.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 shutdown_connections 2024-04-20T02:24:59.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 -- 172.21.15.47:0/1469356806 wait complete. 2024-04-20T02:24:59.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:58.989+0000 7f76ef2fc700 1 librados: shutdown 2024-04-20T02:24:59.105 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:59.105 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-04-20T02:24:59.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.249+0000 7f30d96a7700 10 monclient: get_monmap_and_config 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.249+0000 7f30d96a7700 10 monclient: build_initial_monmap 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.249+0000 7f30d96a7700 10 monclient: monmap: 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:24:59.251065+0000 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:24:59.251065+0000 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:59.250 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:24:59.251 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.250+0000 7f30d96a7700 1 Processor -- start 2024-04-20T02:24:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.250+0000 7f30d96a7700 1 -- start start 2024-04-20T02:24:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.250+0000 7f30d96a7700 10 monclient: init 2024-04-20T02:24:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:59.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient: _add_conns ranks=[7,8,1,3,6,5,9,0,4,2,10] 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d4137d90 0x7f30d40a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): picked mon.noname-i con 0x7f30d4137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 0x7f30d40a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): picked mon.noname-e con 0x7f30d4138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f30d41385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d40751e0 con 0x7f30d41385b0 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30d409fa40 con 0x7f30d4137d90 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f30d409dfc0 con 0x7f30d4138170 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d96a7700 10 monclient(hunting): authenticate will time out at 2178.652283s 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 0x7f30d40a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 0x7f30d40a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41116/0 (socket says 172.21.15.47:41116) 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 -- 172.21.15.47:0/526920524 learned_addr learned my addr 172.21.15.47:0/526920524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): get_auth_request con 0x7f30d4138170 auth_method 0 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d2ffd700 1 --2- 172.21.15.47:0/526920524 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d4137d90 0x7f30d40a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d2ffd700 10 monclient(hunting): get_auth_request con 0x7f30d4137d90 auth_method 0 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient(hunting): handle_auth_done global_id 35396 payload 274 2024-04-20T02:24:59.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 -- 172.21.15.47:0/526920524 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d4137d90 msgr2=0x7f30d40a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 --2- 172.21.15.47:0/526920524 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d4137d90 0x7f30d40a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 -- 172.21.15.47:0/526920524 >> v1:172.21.15.179:6789/0 conn(0x7f30d41385b0 legacy=0x7f30d40a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient: _finish_hunting 0 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 monclient: found mon.noname-e 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 -- 172.21.15.47:0/526920524 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40a7bc0 con 0x7f30d4138170 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.251+0000 7f30d27fc700 1 --2- 172.21.15.47:0/526920524 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 0x7f30d40a5c90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009920 tx=0x7f30c8009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ac9a6e356b0bfa34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 1 -- 172.21.15.47:0/526920524 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30c8009e70 con 0x7f30d4138170 2024-04-20T02:24:59.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: dump: 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.267 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: _finish_auth 0 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 1 -- 172.21.15.47:0/526920524 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c800bcd0 con 0x7f30d4138170 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 5 monclient: authenticate success, global_id 35396 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 10 monclient: get_monmap_and_config success 2024-04-20T02:24:59.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 4 set_mon_vals no callback set 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 1 -- 172.21.15.47:0/526920524 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30c800be40 con 0x7f30d4138170 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d1ffb700 10 monclient: dump: 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.269 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 10 monclient: shutdown 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 -- 172.21.15.47:0/526920524 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 msgr2=0x7f30d40a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 --2- 172.21.15.47:0/526920524 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 0x7f30d40a5c90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009920 tx=0x7f30c8009c30 comp rx=0 tx=0).stop 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 -- 172.21.15.47:0/526920524 shutdown_connections 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 --2- 172.21.15.47:0/526920524 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d4137d90 0x7f30d40a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 --2- 172.21.15.47:0/526920524 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f30d4138170 0x7f30d40a5c90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 -- 172.21.15.47:0/526920524 >> 172.21.15.47:0/526920524 conn(0x7f30d40aa8b0 msgr2=0x7f30d40a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 -- 172.21.15.47:0/526920524 shutdown_connections 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.252+0000 7f30d96a7700 1 -- 172.21.15.47:0/526920524 wait complete. 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.253+0000 7f30d96a7700 10 monclient: build_initial_monmap 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.253+0000 7f30d96a7700 10 monclient: monmap: 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:24:59.271 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.253+0000 7f30d96a7700 1 librados: starting msgr at 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.253+0000 7f30d96a7700 1 librados: starting objecter 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 Processor -- start 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 -- start start 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 librados: setting wanted keys 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 librados: calling monclient init 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient: init 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient: _reopen_session rank -1 2024-04-20T02:24:59.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient: _add_conns ranks=[6,1,7,4,2,8,0,5,3] 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f30d4137d90 0x7f30d41c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): picked mon.noname-d con 0x7f30d4137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30d4138170 0x7f30d4136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): picked mon.noname-f con 0x7f30d4138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41385b0 0x7f30d4137430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): picked mon.noname-i con 0x7f30d41385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d409ea50 con 0x7f30d4138170 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30d409dc20 con 0x7f30d4137d90 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4124010 con 0x7f30d41385b0 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): _renew_subs 2024-04-20T02:24:59.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d96a7700 10 monclient(hunting): authenticate will time out at 2178.655719s 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f30d4137d90 0x7f30d41c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f30d4137d90 0x7f30d41c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51294/0 (socket says 172.21.15.47:51294) 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 1 -- 172.21.15.47:0/241264371 learned_addr learned my addr 172.21.15.47:0/241264371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): get_auth_request con 0x7f30d4137d90 auth_method 0 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d27fc700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30d4138170 0x7f30d4136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d37fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41385b0 0x7f30d4137430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d27fc700 10 monclient(hunting): get_auth_request con 0x7f30d4138170 auth_method 0 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d27fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d37fe700 10 monclient(hunting): get_auth_request con 0x7f30d41385b0 auth_method 0 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.254+0000 7f30d2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 10 monclient(hunting): handle_auth_done global_id 54555 payload 824 2024-04-20T02:24:59.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41385b0 msgr2=0x7f30d4137430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41385b0 0x7f30d4137430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30d4138170 msgr2=0x7f30d4136ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30d4138170 0x7f30d4136ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 monclient: found mon.noname-d 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009550 con 0x7f30d4137d90 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d2ffd700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f30d4137d90 0x7f30d41c8be0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30c400e810 tx=0x7f30c400eb20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 client.?.objecter ms_handle_connect 0x7f30d4137d90 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30c400c070 con 0x7f30d4137d90 2024-04-20T02:24:59.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: dump: 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.277 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c4013880 con 0x7f30d4137d90 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 5 monclient: authenticate success, global_id 54555 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 10 monclient: _renew_subs 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d41242f0 con 0x7f30d4137d90 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30c4013e60 con 0x7f30d4137d90 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30bb7fe700 10 monclient: dump: 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.279 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d17fa700 4 set_mon_vals no callback set 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 10 client.54555.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 10 monclient: _renew_subs 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d41232d0 con 0x7f30d4137d90 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.255+0000 7f30d96a7700 1 librados: init done 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30d96a7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30d96a7700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 0x7f30d41cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4068180 con 0x7f30d406aac0 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f30c40139f0 con 0x7f30d4137d90 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30bb7fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f30c003a2f0 0x7f30c003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(357..357 src has 206..357) v4 ==== 6248+0+0 (secure 0 0 0) 0x7f30c404a3f0 con 0x7f30d4137d90 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30bb7fe700 10 client.54555.objecter ms_dispatch 0x7f30d4001550 osd_map(357..357 src has 206..357) v4 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30bb7fe700 3 client.54555.objecter handle_osd_map got epochs [357,357] > 0 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30bb7fe700 3 client.54555.objecter handle_osd_map decoding full epoch 357 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.256+0000 7f30d27fc700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 0x7f30d41cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient: get_auth_request con 0x7f30d406aac0 auth_method 0 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 20 client.54555.objecter dump_active .. 0 homeless 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d37fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f30c003a2f0 0x7f30c003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d37fe700 10 monclient: get_auth_request con 0x7f30c003a2f0 auth_method 0 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 10 monclient(hunting): handle_auth_done global_id 54574 payload 824 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30d409f760 con 0x7f30d406aac0 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 10 client.54555.objecter ms_handle_connect 0x7f30d406aac0 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 10 client.54555.objecter resend_mon_ops 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d27fc700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 0x7f30d41cc120 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30d409e9b0 tx=0x7f30c8004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 10 client.54555.objecter ms_handle_connect 0x7f30c003a2f0 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30d37fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f30c003a2f0 0x7f30c003c7b0 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0012e0 tx=0x7f30cc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30c8020070 con 0x7f30d406aac0 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.257+0000 7f30bb7fe700 10 monclient: dump: 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.282 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f30c800bb60 con 0x7f30d406aac0 2024-04-20T02:24:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:24:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:24:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 msgr2=0x7f30d41cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 0x7f30d41cc120 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30d409e9b0 tx=0x7f30c8004750 comp rx=0 tx=0).stop 2024-04-20T02:24:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 msgr2=0x7f30d41cc120 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:59.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.258+0000 7f30bb7fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d406aac0 0x7f30d41cc120 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30d409e9b0 tx=0x7f30c8004750 comp rx=0 tx=0).stop 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.265+0000 7f30d96a7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.265+0000 7f30d96a7700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 0x7f30d41cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d96a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30d406a400 con 0x7f30d41cc660 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 0x7f30d41cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient: get_auth_request con 0x7f30d41cc660 auth_method 0 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:24:59.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 10 monclient(hunting): handle_auth_done global_id 54583 payload 824 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 1 -- 172.21.15.47:0/241264371 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f30d4137970 con 0x7f30d41cc660 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30bb7fe700 10 client.54555.objecter ms_handle_connect 0x7f30d41cc660 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30d27fc700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 0x7f30d41cca40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009840 tx=0x7f30c8011000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.266+0000 7f30bb7fe700 10 client.54555.objecter resend_mon_ops 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30c800be40 con 0x7f30d41cc660 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 10 monclient: dump: 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:24:59.287 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7f30c8029790 con 0x7f30d41cc660 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:24:59.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 msgr2=0x7f30d41cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 0x7f30d41cca40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009840 tx=0x7f30c8011000 comp rx=0 tx=0).stop 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 msgr2=0x7f30d41cca40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.267+0000 7f30bb7fe700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f30d41cc660 0x7f30d41cca40 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009840 tx=0x7f30c8011000 comp rx=0 tx=0).stop 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 68, 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:24:59.289 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:59.290 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:24:59.291 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:24:59.292 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:24:59.293 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:24:59.294 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:24:59.296 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:24:59.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.299 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:24:59.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.301 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:24:59.304 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:24:59.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:24:59.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:24:59.308 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.311 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:24:59.312 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:24:59.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 10 librados: watch_flush enter 2024-04-20T02:24:59.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 10 librados: watch_flush exit 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 20 client.54555.objecter shutdown clearing up homeless session... 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 10 client.54555.objecter successfully canceled tick 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f30c003a2f0 msgr2=0x7f30c003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f30c003a2f0 0x7f30c003c7b0 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0012e0 tx=0x7f30cc009040 comp rx=0 tx=0).stop 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 10 monclient: shutdown 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f30d4137d90 msgr2=0x7f30d41c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 --2- 172.21.15.47:0/241264371 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f30d4137d90 0x7f30d41c8be0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30c400e810 tx=0x7f30c400eb20 comp rx=0 tx=0).stop 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d37fe700 1 -- 172.21.15.47:0/241264371 reap_dead start 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d37fe700 1 -- 172.21.15.47:0/241264371 reap_dead start 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 shutdown_connections 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 >> 172.21.15.47:0/241264371 conn(0x7f30d40aa8b0 msgr2=0x7f30d41bf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 shutdown_connections 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 -- 172.21.15.47:0/241264371 wait complete. 2024-04-20T02:24:59.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:24:59.268+0000 7f30d96a7700 1 librados: shutdown 2024-04-20T02:24:59.316 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:24:59.316 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:25:17.167 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-04-20T02:25:17.166+0000 7f8fcc210700 -1 snap_mapper.add_oid found existing snaps mapped on 61:54c6fee4:test-rados-api-smithi179-46984-76::foo:28, removing 2024-04-20T02:25:17.168 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:25:17.167+0000 7ffb14735700 -1 snap_mapper.add_oid found existing snaps mapped on 61:54c6fee4:test-rados-api-smithi179-46984-76::foo:28, removing 2024-04-20T02:25:21.159 INFO:tasks.ceph.osd.5.smithi179.stderr:2024-04-20T02:25:21.159+0000 7fd3da6e4700 -1 snap_mapper.add_oid found existing snaps mapped on 61:87936199:test-rados-api-smithi179-46984-78::foo:29, removing 2024-04-20T02:25:21.160 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:25:21.159+0000 7ffb18f3e700 -1 snap_mapper.add_oid found existing snaps mapped on 61:87936199:test-rados-api-smithi179-46984-78::foo:29, removing 2024-04-20T02:25:46.114 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T02:25:46.114+0000 7fd47fa89700 -1 snap_mapper.add_oid found existing snaps mapped on 220:cb10628a:test-rados-api-smithi179-46984-80::foo:2, removing 2024-04-20T02:25:46.115 INFO:tasks.ceph.osd.4.smithi179.stderr:2024-04-20T02:25:46.114+0000 7f8fc4200700 -1 snap_mapper.add_oid found existing snaps mapped on 220:cb10628a:test-rados-api-smithi179-46984-80::foo:2, removing 2024-04-20T02:26:16.179 INFO:tasks.workunit.client.0.smithi179.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (17956 ms) 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (49028 ms) 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3013 ms) 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (2977 ms) 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3048 ms) 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-04-20T02:26:16.180 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3013 ms) 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4017 ms) 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38924 ms) 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17010 ms) 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4052 ms) 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-04-20T02:26:16.181 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3020 ms) 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (40085 ms) 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14927 ms) 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (4954 ms) 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4043 ms) 2024-04-20T02:26:16.182 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: waiting for scrubs... 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: done waiting 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24015 ms) 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (9991 ms) 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9004 ms) 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (8988 ms) 2024-04-20T02:26:16.183 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-04-20T02:26:29.317 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:26:29.317 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-20T02:26:29.317 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T02:26:29.318 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T02:26:29.321 INFO:tasks.ceph.mon.c:Started 2024-04-20T02:26:29.321 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-04-20T02:26:29.321 INFO:tasks.ceph.mon.g:Sent signal 19 2024-04-20T02:26:29.321 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:26:44.323 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-04-20T02:26:44.323 INFO:tasks.ceph.mon.g:Sent signal 18 2024-04-20T02:26:44.324 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:26:44.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.469+0000 7f2342d56700 10 monclient: get_monmap_and_config 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.469+0000 7f2342d56700 10 monclient: build_initial_monmap 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.470+0000 7f2342d56700 10 monclient: monmap: 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:44.471132+0000 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:44.471132+0000 2024-04-20T02:26:44.470 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:44.471 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:44.472 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.470+0000 7f2342d56700 1 Processor -- start 2024-04-20T02:26:44.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.470+0000 7f2342d56700 1 -- start start 2024-04-20T02:26:44.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.470+0000 7f2342d56700 10 monclient: init 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient: _add_conns ranks=[9,3,4,7,1,10,2,6,8,5,0] 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f233c134c70 0x7f233c135050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): picked mon.noname-a con 0x7f233c134c70 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09f3c0 0x7f233c09d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): picked mon.noname-g con 0x7f233c09f3c0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f233c09df20 0x7f233c09e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): picked mon.noname-c con 0x7f233c09df20 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233c0a0a70 con 0x7f233c134c70 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c09f0c0 con 0x7f233c09f3c0 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f233c135590 con 0x7f233c09df20 2024-04-20T02:26:44.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2342d56700 10 monclient(hunting): authenticate will time out at 2283.872850s 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f2340af2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f233c134c70 0x7f233c135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.471+0000 7f23412f3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f233c09df20 0x7f233c09e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f233c134c70 0x7f233c135050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34694/0 (socket says 172.21.15.47:34694) 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 -- 172.21.15.47:0/3354081228 learned_addr learned my addr 172.21.15.47:0/3354081228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): get_auth_request con 0x7f233c134c70 auth_method 0 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): get_auth_request con 0x7f233c09df20 auth_method 0 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09f3c0 0x7f233c09d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): get_auth_request con 0x7f233c09f3c0 auth_method 0 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient(hunting): handle_auth_done global_id 35928 payload 274 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 -- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f233c09df20 msgr2=0x7f233c09e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f233c09df20 0x7f233c09e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 -- 172.21.15.47:0/3354081228 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09f3c0 msgr2=0x7f233c09d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09f3c0 0x7f233c09d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient: _finish_hunting 0 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 monclient: found mon.noname-a 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 -- 172.21.15.47:0/3354081228 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0751e0 con 0x7f233c134c70 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f23412f3700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f233c09df20 0x7f233c09e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2340af2700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f233c134c70 0x7f233c135050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f232c00bb90 tx=0x7f232c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b158b824a9e2ab27 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 1 -- 172.21.15.47:0/3354081228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f232c010470 con 0x7f233c134c70 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: dump: 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.487 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 1 -- 172.21.15.47:0/3354081228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c0105d0 con 0x7f233c134c70 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2342d56700 5 monclient: authenticate success, global_id 35928 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2342d56700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2342d56700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 1 -- 172.21.15.47:0/3354081228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f232c010bb0 con 0x7f233c134c70 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f2342d56700 4 set_mon_vals no callback set 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:44.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.472+0000 7f233b7fe700 10 monclient: dump: 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.489 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 10 monclient: shutdown 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 -- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f233c134c70 msgr2=0x7f233c135050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f233c134c70 0x7f233c135050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f232c00bb90 tx=0x7f232c00bea0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 -- 172.21.15.47:0/3354081228 shutdown_connections 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f233c09df20 0x7f233c09e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f233c134c70 0x7f233c135050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 --2- 172.21.15.47:0/3354081228 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09f3c0 0x7f233c09d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 -- 172.21.15.47:0/3354081228 >> 172.21.15.47:0/3354081228 conn(0x7f233c0a6720 msgr2=0x7f233c0a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 -- 172.21.15.47:0/3354081228 shutdown_connections 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.473+0000 7f2342d56700 1 -- 172.21.15.47:0/3354081228 wait complete. 2024-04-20T02:26:44.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 10 monclient: build_initial_monmap 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 10 monclient: monmap: 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:44.491 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 1 librados: starting msgr at 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 1 librados: starting objecter 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 1 Processor -- start 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 1 -- start start 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 1 librados: setting wanted keys 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 1 librados: calling monclient init 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.474+0000 7f2342d56700 10 monclient: init 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:44.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient: _add_conns ranks=[3,5,2,7,6,8,4,0,1] 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09df20 0x7f233c12dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): picked mon.noname-g con 0x7f233c09df20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f233c09f3c0 0x7f233c1c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): picked mon.noname-h con 0x7f233c09f3c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 0x7f233c133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): picked mon.noname-b con 0x7f233c134c70 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c12ef40 con 0x7f233c134c70 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c12f1b0 con 0x7f233c09df20 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f233c12f490 con 0x7f233c09f3c0 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:44.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2342d56700 10 monclient(hunting): authenticate will time out at 2283.876395s 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 0x7f233c133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 0x7f233c133cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47344/0 (socket says 172.21.15.47:47344) 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 -- 172.21.15.47:0/3089874123 learned_addr learned my addr 172.21.15.47:0/3089874123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): get_auth_request con 0x7f233c134c70 auth_method 0 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09df20 0x7f233c12dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f233c09f3c0 0x7f233c1c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): get_auth_request con 0x7f233c09df20 auth_method 0 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): get_auth_request con 0x7f233c09f3c0 auth_method 0 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f233bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient(hunting): handle_auth_done global_id 24203 payload 824 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 -- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f233c09f3c0 msgr2=0x7f233c1c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f233c09f3c0 0x7f233c1c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 -- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09df20 msgr2=0x7f233c12dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09df20 0x7f233c12dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient: _finish_hunting 0 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 monclient: found mon.noname-b 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f23412f3700 1 -- 172.21.15.47:0/3089874123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334009a70 con 0x7f233c134c70 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.475+0000 7f2340af2700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09df20 0x7f233c12dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23412f3700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 0x7f233c133cb0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f233400eed0 tx=0x7f233400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 client.?.objecter ms_handle_connect 0x7f233c134c70 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2334010040 con 0x7f233c134c70 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:44.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: dump: 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.497 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: _finish_auth 0 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: _check_auth_tickets 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334014400 con 0x7f233c134c70 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 5 monclient: authenticate success, global_id 24203 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 10 monclient: _renew_subs 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2334014a20 con 0x7f233c134c70 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c00b7f0 con 0x7f233c134c70 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:44.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f233affd700 4 set_mon_vals no callback set 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f23397fa700 10 monclient: dump: 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.499 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 10 client.24203.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 10 monclient: _renew_subs 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c1c55d0 con 0x7f233c134c70 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.476+0000 7f2342d56700 1 librados: init done 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f23340145b0 con 0x7f233c134c70 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f2342d56700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f2342d56700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f2342d56700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c1c5790 con 0x7f233c134c70 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f231c03a320 0x7f231c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(422..422 src has 266..422) v4 ==== 6248+0+0 (secure 0 0 0) 0x7f233404a810 con 0x7f233c134c70 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 10 client.24203.objecter ms_dispatch 0x7f233c001550 osd_map(422..422 src has 266..422) v4 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 3 client.24203.objecter handle_osd_map got epochs [422,422] > 0 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 3 client.24203.objecter handle_osd_map decoding full epoch 422 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f2340af2700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f231c03a320 0x7f231c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f2340af2700 10 monclient: get_auth_request con 0x7f231c03a320 auth_method 0 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.477+0000 7f23397fa700 20 client.24203.objecter dump_active .. 0 homeless 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.478+0000 7f23397fa700 10 client.24203.objecter ms_handle_connect 0x7f231c03a320 2024-04-20T02:26:44.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.478+0000 7f2340af2700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f231c03a320 0x7f231c03c7e0 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 crypto rx=0x7f232c000c00 tx=0x7f232c013000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.483+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2334014860 con 0x7f233c134c70 2024-04-20T02:26:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.483+0000 7f23397fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:44.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.483+0000 7f23397fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f2342d56700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:26:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f2342d56700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f2342d56700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:44.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f233c0a6c20 con 0x7f233c134c70 2024-04-20T02:26:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f23397fa700 1 -- 172.21.15.47:0/3089874123 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f23340175a0 con 0x7f233c134c70 2024-04-20T02:26:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f23397fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:44.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.790+0000 7f23397fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:44.791 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 10 librados: watch_flush enter 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 10 librados: watch_flush exit 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 20 client.24203.objecter shutdown clearing up homeless session... 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 10 client.24203.objecter successfully canceled tick 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f231c03a320 msgr2=0x7f231c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f231c03a320 0x7f231c03c7e0 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 crypto rx=0x7f232c000c00 tx=0x7f232c013000 comp rx=0 tx=0).stop 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 10 monclient: shutdown 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 msgr2=0x7f233c133cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 0x7f233c133cb0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f233400eed0 tx=0x7f233400b390 comp rx=0 tx=0).stop 2024-04-20T02:26:44.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 shutdown_connections 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f231c03a320 0x7f231c03c7e0 unknown :-1 s=CLOSED pgs=3243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f233c134c70 0x7f233c133cb0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f233c09df20 0x7f233c12dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 --2- 172.21.15.47:0/3089874123 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f233c09f3c0 0x7f233c1c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 >> 172.21.15.47:0/3089874123 conn(0x7f233c0a6720 msgr2=0x7f233c1ba4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 shutdown_connections 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 -- 172.21.15.47:0/3089874123 wait complete. 2024-04-20T02:26:44.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.793+0000 7f2342d56700 1 librados: shutdown 2024-04-20T02:26:44.807 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:26:44.807 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:26:44.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.942+0000 7fb4cce2f700 10 monclient: get_monmap_and_config 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.942+0000 7fb4cce2f700 10 monclient: build_initial_monmap 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.942+0000 7fb4cce2f700 10 monclient: monmap: 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:44.944104+0000 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:44.944104+0000 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:44.943 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:44.944 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.943+0000 7fb4cce2f700 1 Processor -- start 2024-04-20T02:26:44.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.943+0000 7fb4cce2f700 1 -- start start 2024-04-20T02:26:44.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.943+0000 7fb4cce2f700 10 monclient: init 2024-04-20T02:26:44.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient: _add_conns ranks=[4,8,10,0,9,2,5,3,1,7,6] 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 0x7fb4c80a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): picked mon.noname-c con 0x7fb4c80a0830 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 0x7fb4c80a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): picked mon.noname-e con 0x7fb4c80a01b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb4c80a1150 0x7fb4c80a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): picked mon.noname-f con 0x7fb4c80a1150 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c809dee0 con 0x7fb4c80a1150 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8067820 con 0x7fb4c80a0830 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c809eee0 con 0x7fb4c80a01b0 2024-04-20T02:26:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4cce2f700 10 monclient(hunting): authenticate will time out at 2284.345455s 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 0x7fb4c80a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 0x7fb4c80a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 0x7fb4c80a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:52920/0 (socket says 172.21.15.47:52920) 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 0x7fb4c80a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39020/0 (socket says 172.21.15.47:39020) 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 -- 172.21.15.47:0/2721298414 learned_addr learned my addr 172.21.15.47:0/2721298414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): get_auth_request con 0x7fb4c80a01b0 auth_method 0 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): get_auth_request con 0x7fb4c80a0830 auth_method 0 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c6d9d700 1 --2- 172.21.15.47:0/2721298414 >> v2:172.21.15.179:3300/0 conn(0x7fb4c80a1150 0x7fb4c80a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c6d9d700 10 monclient(hunting): get_auth_request con 0x7fb4c80a1150 auth_method 0 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_done global_id 35405 payload 274 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 -- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 msgr2=0x7fb4c80a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 --2- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 0x7fb4c80a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 -- 172.21.15.47:0/2721298414 >> v2:172.21.15.179:3300/0 conn(0x7fb4c80a1150 msgr2=0x7fb4c80a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 --2- 172.21.15.47:0/2721298414 >> v2:172.21.15.179:3300/0 conn(0x7fb4c80a1150 0x7fb4c80a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 monclient: found mon.noname-e 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c5d9b700 1 -- 172.21.15.47:0/2721298414 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c80751e0 con 0x7fb4c80a01b0 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.944+0000 7fb4c659c700 1 --2- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 0x7fb4c80a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c5d9b700 1 --2- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 0x7fb4c80a2950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098f0 tx=0x7fb4bc009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2b25f2ee6448f400 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 1 -- 172.21.15.47:0/2721298414 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4bc011aa0 con 0x7fb4c80a01b0 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: dump: 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.960 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.960 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.960 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.960 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: _finish_auth 0 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: _check_auth_tickets 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 1 -- 172.21.15.47:0/2721298414 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc011c00 con 0x7fb4c80a01b0 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 5 monclient: authenticate success, global_id 35405 2024-04-20T02:26:44.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 1 -- 172.21.15.47:0/2721298414 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4bc011d70 con 0x7fb4c80a01b0 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 4 set_mon_vals no callback set 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4c559a700 10 monclient: dump: 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.962 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 10 monclient: shutdown 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 1 -- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 msgr2=0x7fb4c80a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.945+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 0x7fb4c80a2950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098f0 tx=0x7fb4bc009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 -- 172.21.15.47:0/2721298414 shutdown_connections 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb4c80a0830 0x7fb4c80a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/2721298414 >> v2:172.21.15.179:3300/0 conn(0x7fb4c80a1150 0x7fb4c80a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/2721298414 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a01b0 0x7fb4c80a2950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 -- 172.21.15.47:0/2721298414 >> 172.21.15.47:0/2721298414 conn(0x7fb4c80a7570 msgr2=0x7fb4c80a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 -- 172.21.15.47:0/2721298414 shutdown_connections 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.946+0000 7fb4cce2f700 1 -- 172.21.15.47:0/2721298414 wait complete. 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 10 monclient: build_initial_monmap 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 10 monclient: monmap: 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:44.964 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 1 librados: starting msgr at 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 1 librados: starting objecter 2024-04-20T02:26:44.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 1 Processor -- start 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 1 -- start start 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 1 librados: setting wanted keys 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 1 librados: calling monclient init 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.947+0000 7fb4cce2f700 10 monclient: init 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient: _add_conns ranks=[5,8,0,4,1,7,3,6,2] 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb4c80a01b0 0x7fb4c81cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): picked mon.noname-h con 0x7fb4c80a01b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a0830 0x7fb4c81bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): picked mon.noname-e con 0x7fb4c80a0830 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c80a1150 0x7fb4c81bc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): picked mon.noname-a con 0x7fb4c80a1150 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c81aad40 con 0x7fb4c80a1150 2024-04-20T02:26:44.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c809e440 con 0x7fb4c80a01b0 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c809e900 con 0x7fb4c80a0830 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4cce2f700 10 monclient(hunting): authenticate will time out at 2284.349499s 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a0830 0x7fb4c81bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c80a1150 0x7fb4c81bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c80a1150 0x7fb4c81bc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34702/0 (socket says 172.21.15.47:34702) 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a0830 0x7fb4c81bbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:52932/0 (socket says 172.21.15.47:52932) 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 1 -- 172.21.15.47:0/449659470 learned_addr learned my addr 172.21.15.47:0/449659470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): get_auth_request con 0x7fb4c80a1150 auth_method 0 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): get_auth_request con 0x7fb4c80a0830 auth_method 0 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb4c80a01b0 0x7fb4c81cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c5d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): get_auth_request con 0x7fb4c80a01b0 auth_method 0 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 10 monclient(hunting): handle_auth_done global_id 35937 payload 824 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.948+0000 7fb4c6d9d700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a0830 msgr2=0x7fb4c81bbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a0830 0x7fb4c81bbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb4c80a01b0 msgr2=0x7fb4c81cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb4c80a01b0 0x7fb4c81cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 1 monclient: found mon.noname-a 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80094a0 con 0x7fb4c80a1150 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c5d9b700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb4c80a0830 0x7fb4c81bbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c6d9d700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c80a1150 0x7fb4c81bc140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0007e60 tx=0x7fb4c000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 client.?.objecter ms_handle_connect 0x7fb4c80a1150 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4c0009960 con 0x7fb4c80a1150 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: dump: 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.970 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.971 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c0014420 con 0x7fb4c80a1150 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 5 monclient: authenticate success, global_id 35937 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 10 monclient: _renew_subs 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc009550 con 0x7fb4c80a1150 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4c0014a00 con 0x7fb4c80a1150 2024-04-20T02:26:44.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4c4d99700 4 set_mon_vals no callback set 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 10 client.35937.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4b6ffd700 10 monclient: dump: 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.973 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 10 monclient: _renew_subs 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c81ab5e0 con 0x7fb4c80a1150 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.949+0000 7fb4cce2f700 1 librados: init done 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb4c0014590 con 0x7fb4c80a1150 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4cce2f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 0x7fb4c81a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c81ab0e0 con 0x7fb4c806aac0 2024-04-20T02:26:44.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 0x7fb4c81a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient: get_auth_request con 0x7fb4c806aac0 auth_method 0 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb4a803a2f0 0x7fb4a803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(422..422 src has 266..422) v4 ==== 6248+0+0 (secure 0 0 0) 0x7fb4c004ac50 con 0x7fb4c80a1150 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 10 client.35937.objecter ms_dispatch 0x7fb4c8001550 osd_map(422..422 src has 266..422) v4 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 3 client.35937.objecter handle_osd_map got epochs [422,422] > 0 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 3 client.35937.objecter handle_osd_map decoding full epoch 422 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 20 client.35937.objecter dump_active .. 0 homeless 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 10 monclient(hunting): handle_auth_done global_id 35946 payload 824 2024-04-20T02:26:44.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4c80672f0 con 0x7fb4c806aac0 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c5d9b700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb4a803a2f0 0x7fb4a803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c5d9b700 10 monclient: get_auth_request con 0x7fb4a803a2f0 auth_method 0 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 10 client.35937.objecter ms_handle_connect 0x7fb4c806aac0 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4b6ffd700 10 client.35937.objecter resend_mon_ops 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.950+0000 7fb4c659c700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 0x7fb4c81a76d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8011b50 tx=0x7fb4b8011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4b8014440 con 0x7fb4c806aac0 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 10 monclient: dump: 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.976 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 10 client.35937.objecter ms_handle_connect 0x7fb4a803a2f0 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4c5d9b700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb4a803a2f0 0x7fb4a803c7b0 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098f0 tx=0x7fb4bc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb4b800fb10 con 0x7fb4c806aac0 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:44.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 msgr2=0x7fb4c81a76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 0x7fb4c81a76d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8011b50 tx=0x7fb4b8011f10 comp rx=0 tx=0).stop 2024-04-20T02:26:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 msgr2=0x7fb4c81a76d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:44.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.951+0000 7fb4b6ffd700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c806aac0 0x7fb4c81a76d0 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8011b50 tx=0x7fb4b8011f10 comp rx=0 tx=0).stop 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4cce2f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 0x7fb4c81a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4cce2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8068310 con 0x7fb4c81a7c10 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 0x7fb4c81a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient: get_auth_request con 0x7fb4c81a7c10 auth_method 0 2024-04-20T02:26:44.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 10 monclient(hunting): handle_auth_done global_id 35955 payload 824 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 1 -- 172.21.15.47:0/449659470 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb4c809f1c0 con 0x7fb4c81a7c10 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4c659c700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 0x7fb4c81a8020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4b801c000 tx=0x7fb4b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4b6ffd700 10 client.35937.objecter ms_handle_connect 0x7fb4c81a7c10 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4b6ffd700 10 client.35937.objecter resend_mon_ops 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4b8022300 con 0x7fb4c81a7c10 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4b6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4b6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.959+0000 7fb4b6ffd700 10 monclient: dump: 2024-04-20T02:26:44.981 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fb4b8022580 con 0x7fb4c81a7c10 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 msgr2=0x7fb4c81a8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 0x7fb4c81a8020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4b801c000 tx=0x7fb4b8004750 comp rx=0 tx=0).stop 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 msgr2=0x7fb4c81a8020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4b6ffd700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c81a7c10 0x7fb4c81a8020 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4b801c000 tx=0x7fb4b8004750 comp rx=0 tx=0).stop 2024-04-20T02:26:44.983 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:44.984 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:44.985 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:44.986 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:44.987 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:44.988 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:44.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:44.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:44.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.002 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.003 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:45.004 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:45.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 10 librados: watch_flush enter 2024-04-20T02:26:45.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 10 librados: watch_flush exit 2024-04-20T02:26:45.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 20 client.35937.objecter shutdown clearing up homeless session... 2024-04-20T02:26:45.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 10 client.35937.objecter successfully canceled tick 2024-04-20T02:26:45.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb4a803a2f0 msgr2=0x7fb4a803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb4a803a2f0 0x7fb4a803c7b0 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098f0 tx=0x7fb4bc008030 comp rx=0 tx=0).stop 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 10 monclient: shutdown 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c80a1150 msgr2=0x7fb4c81bc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 --2- 172.21.15.47:0/449659470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4c80a1150 0x7fb4c81bc140 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0007e60 tx=0x7fb4c000e3c0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4c6d9d700 1 -- 172.21.15.47:0/449659470 reap_dead start 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4c6d9d700 1 -- 172.21.15.47:0/449659470 reap_dead start 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 shutdown_connections 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 >> 172.21.15.47:0/449659470 conn(0x7fb4c80a7570 msgr2=0x7fb4c80a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 shutdown_connections 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 -- 172.21.15.47:0/449659470 wait complete. 2024-04-20T02:26:45.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:44.960+0000 7fb4cce2f700 1 librados: shutdown 2024-04-20T02:26:45.008 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:45.008 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:26:45.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:26:45.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.147+0000 7fea170c4700 10 monclient: get_monmap_and_config 2024-04-20T02:26:45.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.147+0000 7fea170c4700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.147+0000 7fea170c4700 10 monclient: monmap: 2024-04-20T02:26:45.148 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:45.149079+0000 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:45.149079+0000 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.149 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.150 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.150 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:45.150 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:45.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.148+0000 7fea170c4700 1 Processor -- start 2024-04-20T02:26:45.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.148+0000 7fea170c4700 1 -- start start 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.148+0000 7fea170c4700 10 monclient: init 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient: _add_conns ranks=[9,10,1,6,7,2,3,8,5,4,0] 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea100a50f0 0x7fea10137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): picked mon.noname-a con 0x7fea100a50f0 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea101378a0 0x7fea10138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): picked mon.noname-f con 0x7fea101378a0 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fea100a38f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea100751e0 con 0x7fea100a38f0 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea100a54d0 con 0x7fea100a50f0 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea101387b0 con 0x7fea101378a0 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea170c4700 10 monclient(hunting): authenticate will time out at 2284.550850s 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.149+0000 7fea14e60700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea100a50f0 0x7fea10137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fea101378a0 0x7fea10138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea100a50f0 0x7fea10137d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34732/0 (socket says 172.21.15.47:34732) 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 -- 172.21.15.47:0/1199436215 learned_addr learned my addr 172.21.15.47:0/1199436215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): get_auth_request con 0x7fea100a50f0 auth_method 0 2024-04-20T02:26:45.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): get_auth_request con 0x7fea101378a0 auth_method 0 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient(hunting): handle_auth_done global_id 35964 payload 274 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 -- 172.21.15.47:0/1199436215 >> v2:172.21.15.179:3300/0 conn(0x7fea101378a0 msgr2=0x7fea10138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 --2- 172.21.15.47:0/1199436215 >> v2:172.21.15.179:3300/0 conn(0x7fea101378a0 0x7fea10138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 -- 172.21.15.47:0/1199436215 >> v1:172.21.15.179:6789/0 conn(0x7fea100a38f0 legacy=0x7fea100a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1199436215 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042509311 0 0) 0x7fea100751e0 con 0x7fea100a38f0 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 monclient: found mon.noname-a 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:45.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 -- 172.21.15.47:0/1199436215 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1006da20 con 0x7fea100a50f0 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea14e60700 1 --2- 172.21.15.47:0/1199436215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea10137d30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea100a8fa0 tx=0x7fea00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f26a47ac756de29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1199436215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea00011aa0 con 0x7fea100a50f0 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: dump: 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.167 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea170c4700 5 monclient: authenticate success, global_id 35964 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea170c4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1199436215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00011c00 con 0x7fea100a50f0 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea170c4700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea170c4700 4 set_mon_vals no callback set 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 1 -- 172.21.15.47:0/1199436215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea00011d70 con 0x7fea100a50f0 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.150+0000 7fea0f7fe700 10 monclient: dump: 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.169 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 10 monclient: shutdown 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 -- 172.21.15.47:0/1199436215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 msgr2=0x7fea10137d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 --2- 172.21.15.47:0/1199436215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea10137d30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea100a8fa0 tx=0x7fea00009bd0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 -- 172.21.15.47:0/1199436215 shutdown_connections 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 --2- 172.21.15.47:0/1199436215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea10137d30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 --2- 172.21.15.47:0/1199436215 >> v2:172.21.15.179:3300/0 conn(0x7fea101378a0 0x7fea10138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 -- 172.21.15.47:0/1199436215 >> 172.21.15.47:0/1199436215 conn(0x7fea100a8510 msgr2=0x7fea100a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 -- 172.21.15.47:0/1199436215 shutdown_connections 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.151+0000 7fea170c4700 1 -- 172.21.15.47:0/1199436215 wait complete. 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 10 monclient: monmap: 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.171 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.172 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 1 librados: starting msgr at 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 1 librados: starting objecter 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 1 Processor -- start 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 1 -- start start 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 1 librados: setting wanted keys 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 1 librados: calling monclient init 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.152+0000 7fea170c4700 10 monclient: init 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient: _add_conns ranks=[6,0,7,4,1,2,3,8,5] 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea100a38f0 0x7fea10136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): picked mon.noname-d con 0x7fea100a38f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea101cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): picked mon.noname-a con 0x7fea100a50f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fea10123060 0x7fea101cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): picked mon.noname-i con 0x7fea10123060 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea101c95c0 con 0x7fea100a50f0 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea10121be0 con 0x7fea100a38f0 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fea10071e80 con 0x7fea10123060 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea170c4700 10 monclient(hunting): authenticate will time out at 2284.554391s 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea100a38f0 0x7fea10136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea101cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea101cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34744/0 (socket says 172.21.15.47:34744) 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 -- 172.21.15.47:0/3050823039 learned_addr learned my addr 172.21.15.47:0/3050823039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): get_auth_request con 0x7fea100a50f0 auth_method 0 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea100a38f0 0x7fea10136cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38864/0 (socket says 172.21.15.47:38864) 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): get_auth_request con 0x7fea100a38f0 auth_method 0 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea15661700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fea10123060 0x7fea101cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient(hunting): handle_auth_done global_id 35973 payload 824 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fea10123060 msgr2=0x7fea101cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fea10123060 0x7fea101cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea100a38f0 msgr2=0x7fea10136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea100a38f0 0x7fea10136cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 monclient: found mon.noname-a 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea0ffff700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea00009550 con 0x7fea100a50f0 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea15661700 10 monclient: get_auth_request con 0x7fea10123060 auth_method 0 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea15661700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fea10123060 0x7fea101cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.153+0000 7fea14e60700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fea100a38f0 0x7fea10136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:45.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0ffff700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea101cc030 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e9b0 tx=0x7fe9f800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 client.?.objecter ms_handle_connect 0x7fea100a50f0 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9f800cc60 con 0x7fea100a50f0 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: dump: 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.178 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 5 monclient: authenticate success, global_id 35973 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800ce20 con 0x7fea100a50f0 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9f80138e0 con 0x7fea100a50f0 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0effd700 4 set_mon_vals no callback set 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea0d7fa700 10 monclient: dump: 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.180 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 10 monclient: _renew_subs 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea10072260 con 0x7fea100a50f0 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 10 client.35973.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 10 monclient: _renew_subs 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea101240d0 con 0x7fea100a50f0 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.154+0000 7fea170c4700 1 librados: init done 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe9f8013470 con 0x7fea100a50f0 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea170c4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea170c4700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 0x7fea101ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea10071580 con 0x7fea101378a0 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f003a2f0 0x7fe9f003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(422..422 src has 266..422) v4 ==== 6248+0+0 (secure 0 0 0) 0x7fe9f804a8b0 con 0x7fea100a50f0 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 10 client.35973.objecter ms_dispatch 0x7fea10001550 osd_map(422..422 src has 266..422) v4 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 3 client.35973.objecter handle_osd_map got epochs [422,422] > 0 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 3 client.35973.objecter handle_osd_map decoding full epoch 422 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 0x7fea101ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient: get_auth_request con 0x7fea101378a0 auth_method 0 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 20 client.35973.objecter dump_active .. 0 homeless 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea15661700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f003a2f0 0x7fe9f003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea15661700 10 monclient: get_auth_request con 0x7fe9f003a2f0 auth_method 0 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 10 monclient(hunting): handle_auth_done global_id 24212 payload 824 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea100a72a0 con 0x7fea101378a0 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea14e60700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 0x7fea101ccab0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fea00009fd0 tx=0x7fea00004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 10 client.35973.objecter ms_handle_connect 0x7fea101378a0 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.155+0000 7fea0d7fa700 10 client.35973.objecter resend_mon_ops 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea00020070 con 0x7fea101378a0 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 10 monclient: dump: 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.184 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.185 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 10 client.35973.objecter ms_handle_connect 0x7fe9f003a2f0 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea15661700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f003a2f0 0x7fe9f003c7b0 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7fea10123460 tx=0x7fea08009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fea000038b0 con 0x7fea101378a0 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 msgr2=0x7fea101ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 0x7fea101ccab0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fea00009fd0 tx=0x7fea00004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 msgr2=0x7fea101ccab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.156+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea101378a0 0x7fea101ccab0 secure :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fea00009fd0 tx=0x7fea00004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea170c4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:45.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea170c4700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 0x7fea1002cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea170c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea100682f0 con 0x7fea1006aac0 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 0x7fea1002cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient: get_auth_request con 0x7fea1006aac0 auth_method 0 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 10 monclient(hunting): handle_auth_done global_id 24221 payload 824 2024-04-20T02:26:45.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 1 -- 172.21.15.47:0/3050823039 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea1006e370 con 0x7fea1006aac0 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea14e60700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 0x7fea1002cd40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fea00009130 tx=0x7fea00011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea0d7fa700 10 client.35973.objecter ms_handle_connect 0x7fea1006aac0 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.164+0000 7fea0d7fa700 10 client.35973.objecter resend_mon_ops 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fea0002c6d0 con 0x7fea1006aac0 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 10 monclient: dump: 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.190 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fea00025b20 con 0x7fea1006aac0 2024-04-20T02:26:45.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 msgr2=0x7fea1002cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 0x7fea1002cd40 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fea00009130 tx=0x7fea00011000 comp rx=0 tx=0).stop 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 msgr2=0x7fea1002cd40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea0d7fa700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fea1006aac0 0x7fea1002cd40 secure :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fea00009130 tx=0x7fea00011000 comp rx=0 tx=0).stop 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:45.192 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:45.193 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.194 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.195 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:45.196 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.197 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.198 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.200 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.204 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.205 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.207 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.209 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.211 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:45.212 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.213 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.216 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:45.217 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:45.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 10 librados: watch_flush enter 2024-04-20T02:26:45.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 10 librados: watch_flush exit 2024-04-20T02:26:45.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 20 client.35973.objecter shutdown clearing up homeless session... 2024-04-20T02:26:45.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 10 client.35973.objecter successfully canceled tick 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f003a2f0 msgr2=0x7fe9f003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe9f003a2f0 0x7fe9f003c7b0 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 crypto rx=0x7fea10123460 tx=0x7fea08009040 comp rx=0 tx=0).stop 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 10 monclient: shutdown 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 msgr2=0x7fea101cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 --2- 172.21.15.47:0/3050823039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea100a50f0 0x7fea101cc030 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e9b0 tx=0x7fe9f800ecc0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea15661700 1 -- 172.21.15.47:0/3050823039 reap_dead start 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea15661700 1 -- 172.21.15.47:0/3050823039 reap_dead start 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 shutdown_connections 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 >> 172.21.15.47:0/3050823039 conn(0x7fea100a8510 msgr2=0x7fea100a12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 shutdown_connections 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.165+0000 7fea170c4700 1 -- 172.21.15.47:0/3050823039 wait complete. 2024-04-20T02:26:45.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.166+0000 7fea170c4700 1 librados: shutdown 2024-04-20T02:26:45.221 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:45.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:26:45.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.370+0000 7f72ffe25700 10 monclient: get_monmap_and_config 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.370+0000 7f72ffe25700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.370+0000 7f72ffe25700 10 monclient: monmap: 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:45.371358+0000 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:45.371358+0000 2024-04-20T02:26:45.371 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.372 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:45.373 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:45.373 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.370+0000 7f72ffe25700 1 Processor -- start 2024-04-20T02:26:45.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 1 -- start start 2024-04-20T02:26:45.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient: init 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient: _add_conns ranks=[1,0,2,10,3,6,8,4,9,7,5] 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f72f80a07f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72f80a0010 addr v1:172.21.15.47:6789/0 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f813e380 0x7f72f8140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient(hunting): picked mon.noname-b con 0x7f72f813e380 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80751e0 con 0x7f72f80a0010 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80672f0 con 0x7f72f80a07f0 2024-04-20T02:26:45.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8067800 con 0x7f72f813e380 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72ffe25700 10 monclient(hunting): authenticate will time out at 2284.772727s 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fd3c0700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72f80a0010 0x7f72f813cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47524/0 (socket says 172.21.15.47:47524) 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f813e380 0x7f72f8140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fd3c0700 1 -- 172.21.15.47:0/3093108188 learned_addr learned my addr 172.21.15.47:0/3093108188 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): get_auth_request con 0x7f72f813e380 auth_method 0 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.371+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_done global_id 24230 payload 274 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 1 -- 172.21.15.47:0/3093108188 >> v1:172.21.15.179:6789/0 conn(0x7f72f80a07f0 legacy=0x7f72f80a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 1 -- 172.21.15.47:0/3093108188 >> v1:172.21.15.47:6789/0 conn(0x7f72f80a0010 legacy=0x7f72f813cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 1 monclient: found mon.noname-b 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 1 -- 172.21.15.47:0/3093108188 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80a12e0 con 0x7f72f813e380 2024-04-20T02:26:45.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 1 -- 172.21.15.47:0/3093108188 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365380037 0 0) 0x7f72f80751e0 con 0x7f72f80a0010 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fe3c2700 1 --2- 172.21.15.47:0/3093108188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f813e380 0x7f72f8140770 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009dc0 tx=0x7f72f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2b3443543c069c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 1 -- 172.21.15.47:0/3093108188 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72f000e070 con 0x7f72f813e380 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: dump: 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.387 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 1 -- 172.21.15.47:0/3093108188 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f00098f0 con 0x7f72f813e380 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72ffe25700 5 monclient: authenticate success, global_id 24230 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72ffe25700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:45.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72ffe25700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72ffe25700 4 set_mon_vals no callback set 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 1 -- 172.21.15.47:0/3093108188 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72f0012d10 con 0x7f72f813e380 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.372+0000 7f72fcbbf700 10 monclient: dump: 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.389 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 10 monclient: shutdown 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 -- 172.21.15.47:0/3093108188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f813e380 msgr2=0x7f72f8140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 --2- 172.21.15.47:0/3093108188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f813e380 0x7f72f8140770 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009dc0 tx=0x7f72f000b330 comp rx=0 tx=0).stop 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 -- 172.21.15.47:0/3093108188 shutdown_connections 2024-04-20T02:26:45.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 --2- 172.21.15.47:0/3093108188 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f813e380 0x7f72f8140770 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 -- 172.21.15.47:0/3093108188 >> 172.21.15.47:0/3093108188 conn(0x7f72f80a3f90 msgr2=0x7f72f806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 -- 172.21.15.47:0/3093108188 shutdown_connections 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 1 -- 172.21.15.47:0/3093108188 wait complete. 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.373+0000 7f72ffe25700 10 monclient: monmap: 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:45.391 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 librados: starting msgr at 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 librados: starting objecter 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 Processor -- start 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 -- start start 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 librados: setting wanted keys 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 librados: calling monclient init 2024-04-20T02:26:45.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient: init 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient: _add_conns ranks=[2,3,5,8,6,7,0,4,1] 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f80a0010 0x7f72f81229f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): picked mon.noname-b con 0x7f72f80a0010 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f80a07f0 0x7f72f81218c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): picked mon.noname-g con 0x7f72f80a07f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f813e380 0x7f72f8121e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): picked mon.noname-h con 0x7f72f813e380 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f809e360 con 0x7f72f80a0010 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f813d460 con 0x7f72f80a07f0 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8123930 con 0x7f72f813e380 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72ffe25700 10 monclient(hunting): authenticate will time out at 2284.775861s 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f80a0010 0x7f72f81229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f80a0010 0x7f72f81229f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47390/0 (socket says 172.21.15.47:47390) 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 1 -- 172.21.15.47:0/1887593510 learned_addr learned my addr 172.21.15.47:0/1887593510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 10 monclient(hunting): get_auth_request con 0x7f72f80a0010 auth_method 0 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.374+0000 7f72fdbc1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f80a07f0 0x7f72f81218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f813e380 0x7f72f8121e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): get_auth_request con 0x7f72f80a07f0 auth_method 0 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): get_auth_request con 0x7f72f813e380 auth_method 0 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fe3c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 10 monclient(hunting): handle_auth_done global_id 24239 payload 824 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f813e380 msgr2=0x7f72f8121e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f813e380 0x7f72f8121e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f80a07f0 msgr2=0x7f72f81218c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f80a07f0 0x7f72f81218c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 monclient: found mon.noname-b 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f0009a70 con 0x7f72f80a0010 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72fdbc1700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f80a0010 0x7f72f81229f0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f72f400c4e0 tx=0x7f72f400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 client.?.objecter ms_handle_connect 0x7f72f80a0010 2024-04-20T02:26:45.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72f4005540 con 0x7f72f80a0010 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: dump: 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.397 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f4003680 con 0x7f72f80a0010 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 5 monclient: authenticate success, global_id 24239 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 10 monclient: _renew_subs 2024-04-20T02:26:45.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f8123ef0 con 0x7f72f80a0010 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72f4003c60 con 0x7f72f80a0010 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72effff700 4 set_mon_vals no callback set 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ee7fc700 10 monclient: dump: 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.399 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 10 client.24239.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 10 monclient: _renew_subs 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f813cd20 con 0x7f72f80a0010 2024-04-20T02:26:45.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.375+0000 7f72ffe25700 1 librados: init done 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f72f40037f0 con 0x7f72f80a0010 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ffe25700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ffe25700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 0x7f72f8122340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ffe25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8068280 con 0x7f72f806aac0 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 0x7f72f8122340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient: get_auth_request con 0x7f72f806aac0 auth_method 0 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ee7fc700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(423..423 src has 266..423) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f72f4049730 con 0x7f72f80a0010 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ee7fc700 10 client.24239.objecter ms_dispatch 0x7f72f8001550 osd_map(423..423 src has 266..423) v4 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ee7fc700 3 client.24239.objecter handle_osd_map got epochs [423,423] > 0 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.376+0000 7f72ee7fc700 3 client.24239.objecter handle_osd_map decoding full epoch 423 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 20 client.24239.objecter dump_active .. 0 homeless 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_done global_id 74119 payload 824 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72fe3c2700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72fd3c0700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72f809f070 con 0x7f72f806aac0 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72fe3c2700 10 monclient: get_auth_request con 0x7f72dc03a2f0 auth_method 0 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 client.24239.objecter ms_handle_connect 0x7f72f806aac0 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 client.24239.objecter resend_mon_ops 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72fd3c0700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 0x7f72f8122340 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72f813ca50 tx=0x7f72e80018f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72e8017070 con 0x7f72f806aac0 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 monclient: dump: 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.403 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 client.24239.objecter ms_handle_connect 0x7f72dc03a2f0 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72fe3c2700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b370 tx=0x7f72f0011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f72e8012560 con 0x7f72f806aac0 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 msgr2=0x7f72f8122340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 0x7f72f8122340 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72f813ca50 tx=0x7f72e80018f0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 msgr2=0x7f72f8122340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.377+0000 7f72ee7fc700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f806aac0 0x7f72f8122340 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72f813ca50 tx=0x7f72e80018f0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72ffe25700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72ffe25700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 0x7f72f8138440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72ffe25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f806a400 con 0x7f72f8138030 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 0x7f72f8138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient: get_auth_request con 0x7f72f8138030 auth_method 0 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.385+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72fd3c0700 10 monclient(hunting): handle_auth_done global_id 74128 payload 824 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72fd3c0700 1 -- 172.21.15.47:0/1887593510 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f72f809f350 con 0x7f72f8138030 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 client.24239.objecter ms_handle_connect 0x7f72f8138030 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 client.24239.objecter resend_mon_ops 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72fd3c0700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 0x7f72f8138440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72e8014000 tx=0x7f72e8000ed0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72e8023070 con 0x7f72f8138030 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 monclient: dump: 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.408 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f72e8012e00 con 0x7f72f8138030 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:45.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 msgr2=0x7f72f8138440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 0x7f72f8138440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72e8014000 tx=0x7f72e8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 msgr2=0x7f72f8138440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.386+0000 7f72ee7fc700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f8138030 0x7f72f8138440 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72e8014000 tx=0x7f72e8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:45.410 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:45.411 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:45.412 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:45.413 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:45.414 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:45.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.416 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:45.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:45.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.425 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.426 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.428 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.429 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.430 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:45.431 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 10 librados: watch_flush enter 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 10 librados: watch_flush exit 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 20 client.24239.objecter shutdown clearing up homeless session... 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 10 client.24239.objecter successfully canceled tick 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 msgr2=0x7f72dc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b370 tx=0x7f72f0011000 comp rx=0 tx=0).stop 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 10 monclient: shutdown 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f80a0010 msgr2=0x7f72f81229f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 --2- 172.21.15.47:0/1887593510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f80a0010 0x7f72f81229f0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f72f400c4e0 tx=0x7f72f400c7f0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72fe3c2700 1 -- 172.21.15.47:0/1887593510 reap_dead start 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72fe3c2700 1 -- 172.21.15.47:0/1887593510 reap_dead start 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 shutdown_connections 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 >> 172.21.15.47:0/1887593510 conn(0x7f72f80a3f90 msgr2=0x7f72f806f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 shutdown_connections 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 -- 172.21.15.47:0/1887593510 wait complete. 2024-04-20T02:26:45.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.387+0000 7f72ffe25700 1 librados: shutdown 2024-04-20T02:26:45.434 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:45.434 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:26:45.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.573+0000 7f8869e4a700 10 monclient: get_monmap_and_config 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.573+0000 7f8869e4a700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.573+0000 7f8869e4a700 10 monclient: monmap: 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:45.574740+0000 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:45.574740+0000 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:45.576 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 1 Processor -- start 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 1 -- start start 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 10 monclient: init 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 10 monclient: _add_conns ranks=[10,2,0,1,8,5,3,9,4,7,6] 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f88640a0a70 0x7f88640a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 10 monclient(hunting): picked mon.noname-f con 0x7f88640a0a70 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 0x7f88640a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.574+0000 7f8869e4a700 10 monclient(hunting): picked mon.noname-b con 0x7f88640a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f88640a3290 addr v1:172.21.15.47:6789/0 2024-04-20T02:26:45.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88640751e0 con 0x7f88640a3290 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88640a0260 con 0x7f88640a0a70 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886409ef90 con 0x7f88640a2950 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 0x7f88640a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8869e4a700 10 monclient(hunting): authenticate will time out at 2284.976305s 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 0x7f88640a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47404/0 (socket says 172.21.15.47:47404) 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 -- 172.21.15.47:0/3459981135 learned_addr learned my addr 172.21.15.47:0/3459981135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): get_auth_request con 0x7f88640a2950 auth_method 0 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 1 --2- 172.21.15.47:0/3459981135 >> v2:172.21.15.179:3300/0 conn(0x7f88640a0a70 0x7f88640a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): get_auth_request con 0x7f88640a0a70 auth_method 0 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient(hunting): handle_auth_done global_id 24248 payload 274 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 -- 172.21.15.47:0/3459981135 >> v2:172.21.15.179:3300/0 conn(0x7f88640a0a70 msgr2=0x7f88640a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 --2- 172.21.15.47:0/3459981135 >> v2:172.21.15.179:3300/0 conn(0x7f88640a0a70 0x7f88640a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 -- 172.21.15.47:0/3459981135 >> v1:172.21.15.47:6789/0 conn(0x7f88640a3290 legacy=0x7f886413eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 monclient: found mon.noname-b 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 -- 172.21.15.47:0/3459981135 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886409f270 con 0x7f88640a2950 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.575+0000 7f8862ffd700 1 --2- 172.21.15.47:0/3459981135 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 0x7f88640a2d50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f88580098f0 tx=0x7f8858009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0d4d73767b2c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 1 -- 172.21.15.47:0/3459981135 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8858011aa0 con 0x7f88640a2950 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: dump: 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.589 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.590 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 1 -- 172.21.15.47:0/3459981135 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8858011c00 con 0x7f88640a2950 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 5 monclient: authenticate success, global_id 24248 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 1 -- 172.21.15.47:0/3459981135 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8858011d70 con 0x7f88640a2950 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 4 set_mon_vals no callback set 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:45.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f88627fc700 10 monclient: dump: 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.592 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 10 monclient: shutdown 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 -- 172.21.15.47:0/3459981135 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 msgr2=0x7f88640a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 --2- 172.21.15.47:0/3459981135 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 0x7f88640a2d50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f88580098f0 tx=0x7f8858009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 -- 172.21.15.47:0/3459981135 shutdown_connections 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 --2- 172.21.15.47:0/3459981135 >> v2:172.21.15.179:3300/0 conn(0x7f88640a0a70 0x7f88640a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 --2- 172.21.15.47:0/3459981135 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f88640a2950 0x7f88640a2d50 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 -- 172.21.15.47:0/3459981135 >> 172.21.15.47:0/3459981135 conn(0x7f88640a7570 msgr2=0x7f88640a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 -- 172.21.15.47:0/3459981135 shutdown_connections 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.576+0000 7f8869e4a700 1 -- 172.21.15.47:0/3459981135 wait complete. 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.577+0000 7f8869e4a700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.577+0000 7f8869e4a700 10 monclient: monmap: 2024-04-20T02:26:45.593 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.594 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.577+0000 7f8869e4a700 1 librados: starting msgr at 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.577+0000 7f8869e4a700 1 librados: starting objecter 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.577+0000 7f8869e4a700 1 Processor -- start 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 -- start start 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 librados: setting wanted keys 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 librados: calling monclient init 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient: init 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient: _add_conns ranks=[5,3,8,0,6,1,7,2,4] 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f88640a0a70 0x7f88641cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): picked mon.noname-h con 0x7f88640a0a70 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88640a2950 0x7f88641bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): picked mon.noname-g con 0x7f88640a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f88640a3290 0x7f88641bc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): picked mon.noname-e con 0x7f88640a3290 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88641bc990 con 0x7f88640a2950 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88641ab6e0 con 0x7f88640a0a70 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f886409e560 con 0x7f88640a3290 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8869e4a700 10 monclient(hunting): authenticate will time out at 2284.979682s 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f88640a3290 0x7f88641bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f88640a3290 0x7f88641bc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:52944/0 (socket says 172.21.15.47:52944) 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 1 -- 172.21.15.47:0/1694796485 learned_addr learned my addr 172.21.15.47:0/1694796485 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): get_auth_request con 0x7f88640a3290 auth_method 0 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8863fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f88637fe700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f88640a0a70 0x7f88641cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): get_auth_request con 0x7f88640a0a70 auth_method 0 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.578+0000 7f8862ffd700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88640a2950 0x7f88641bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8862ffd700 10 monclient(hunting): get_auth_request con 0x7f88640a2950 auth_method 0 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8862ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8862ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 10 monclient(hunting): handle_auth_done global_id 35423 payload 824 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f88640a0a70 msgr2=0x7f88641cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f88640a0a70 0x7f88641cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88640a2950 msgr2=0x7f88641bbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88640a2950 0x7f88641bbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 monclient: found mon.noname-e 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:45.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858009550 con 0x7f88640a3290 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8862ffd700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f88640a2950 0x7f88641bbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8863fff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f88640a3290 0x7f88641bc450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f885c00cb50 tx=0x7f885c00ce60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 client.?.objecter ms_handle_connect 0x7f88640a3290 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f885c003e70 con 0x7f88640a3290 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: dump: 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.599 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 5 monclient: authenticate success, global_id 35423 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c001c10 con 0x7f88640a3290 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 10 monclient: _renew_subs 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f885c00acf0 con 0x7f88640a3290 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:26:45.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886409eab0 con 0x7f88640a3290 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8861ffb700 4 set_mon_vals no callback set 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 10 client.35423.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f884bfff700 10 monclient: dump: 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.601 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 10 monclient: _renew_subs 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88641ab120 con 0x7f88640a3290 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.579+0000 7f8869e4a700 1 librados: init done 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f885c00a880 con 0x7f88640a3290 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f8869e4a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f8869e4a700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 0x7f88641a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8864068310 con 0x7f886406aac0 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 0x7f88641a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient: get_auth_request con 0x7f886406aac0 auth_method 0 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f885003a2f0 0x7f885003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(423..423 src has 266..423) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f885c04a690 con 0x7f88640a3290 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 10 client.35423.objecter ms_dispatch 0x7f8864001550 osd_map(423..423 src has 266..423) v4 2024-04-20T02:26:45.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 3 client.35423.objecter handle_osd_map got epochs [423,423] > 0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 3 client.35423.objecter handle_osd_map decoding full epoch 423 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f884bfff700 20 client.35423.objecter dump_active .. 0 homeless 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 10 monclient(hunting): handle_auth_done global_id 54573 payload 824 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f88637fe700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88640a1390 con 0x7f886406aac0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.580+0000 7f8862ffd700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f885003a2f0 0x7f885003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f8862ffd700 10 monclient: get_auth_request con 0x7f885003a2f0 auth_method 0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f88637fe700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 0x7f88641a7730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88540119d0 tx=0x7f8854011d90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 client.35423.objecter ms_handle_connect 0x7f886406aac0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 client.35423.objecter resend_mon_ops 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f88540036a0 con 0x7f886406aac0 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 monclient: dump: 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.605 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 client.35423.objecter ms_handle_connect 0x7f885003a2f0 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f8862ffd700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f885003a2f0 0x7f885003c7b0 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f88580098f0 tx=0x7f8858008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8854004240 con 0x7f886406aac0 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 msgr2=0x7f88641a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 0x7f88641a7730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88540119d0 tx=0x7f8854011d90 comp rx=0 tx=0).stop 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 msgr2=0x7f88641a7730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.581+0000 7f884bfff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f886406aac0 0x7f88641a7730 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88540119d0 tx=0x7f8854011d90 comp rx=0 tx=0).stop 2024-04-20T02:26:45.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f8869e4a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f8869e4a700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 0x7f88641a8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f8869e4a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f886406a400 con 0x7f88641a7c70 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 0x7f88641a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient: get_auth_request con 0x7f88641a7c70 auth_method 0 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 10 monclient(hunting): handle_auth_done global_id 54582 payload 824 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 1 -- 172.21.15.47:0/1694796485 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8864067cb0 con 0x7f88641a7c70 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f88637fe700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 0x7f88641a8080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f885401b000 tx=0x7f8854004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f884bfff700 10 client.35423.objecter ms_handle_connect 0x7f88641a7c70 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.589+0000 7f884bfff700 10 client.35423.objecter resend_mon_ops 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f885400fd90 con 0x7f88641a7c70 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 10 monclient: dump: 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.610 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f88540283a0 con 0x7f88641a7c70 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:45.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 msgr2=0x7f88641a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 0x7f88641a8080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f885401b000 tx=0x7f8854004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 msgr2=0x7f88641a8080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f884bfff700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f88641a7c70 0x7f88641a8080 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f885401b000 tx=0x7f8854004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:45.612 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:45.613 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:45.614 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.615 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:45.616 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.617 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.619 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.620 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.627 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:45.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.630 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.632 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.633 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:45.634 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:45.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 10 librados: watch_flush enter 2024-04-20T02:26:45.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 10 librados: watch_flush exit 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 20 client.35423.objecter shutdown clearing up homeless session... 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 10 client.35423.objecter successfully canceled tick 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f885003a2f0 msgr2=0x7f885003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f885003a2f0 0x7f885003c7b0 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 crypto rx=0x7f88580098f0 tx=0x7f8858008030 comp rx=0 tx=0).stop 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 10 monclient: shutdown 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f88640a3290 msgr2=0x7f88641bc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 1 --2- 172.21.15.47:0/1694796485 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f88640a3290 0x7f88641bc450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f885c00cb50 tx=0x7f885c00ce60 comp rx=0 tx=0).stop 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8863fff700 1 -- 172.21.15.47:0/1694796485 reap_dead start 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8863fff700 1 -- 172.21.15.47:0/1694796485 reap_dead start 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 shutdown_connections 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.590+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 >> 172.21.15.47:0/1694796485 conn(0x7f88640a7570 msgr2=0x7f88641be1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.591+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 shutdown_connections 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.591+0000 7f8869e4a700 1 -- 172.21.15.47:0/1694796485 wait complete. 2024-04-20T02:26:45.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.591+0000 7f8869e4a700 1 librados: shutdown 2024-04-20T02:26:45.638 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:45.638 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:26:45.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:26:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.776+0000 7f33ac359700 10 monclient: get_monmap_and_config 2024-04-20T02:26:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.776+0000 7f33ac359700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.776+0000 7f33ac359700 10 monclient: monmap: 2024-04-20T02:26:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.777 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:45.777995+0000 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:45.777995+0000 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:45.778 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:45.779 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.777+0000 7f33ac359700 1 Processor -- start 2024-04-20T02:26:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.777+0000 7f33ac359700 1 -- start start 2024-04-20T02:26:45.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.777+0000 7f33ac359700 10 monclient: init 2024-04-20T02:26:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient: _add_conns ranks=[6,4,1,7,9,8,5,10,0,2,3] 2024-04-20T02:26:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 0x7f33a4135d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): picked mon.noname-d con 0x7f33a40a1cc0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 0x7f33a40a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): picked mon.noname-c con 0x7f33a40a20a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f33a4099020 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40751e0 con 0x7f33a4099020 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33a409cd90 con 0x7f33a40a20a0 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33a409dad0 con 0x7f33a40a1cc0 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33ac359700 10 monclient(hunting): authenticate will time out at 2285.179249s 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 0x7f33a40a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 0x7f33a4135d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 0x7f33a40a2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39044/0 (socket says 172.21.15.47:39044) 2024-04-20T02:26:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 1 -- 172.21.15.47:0/1066957946 learned_addr learned my addr 172.21.15.47:0/1066957946 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 0x7f33a4135d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38902/0 (socket says 172.21.15.47:38902) 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): get_auth_request con 0x7f33a40a20a0 auth_method 0 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): get_auth_request con 0x7f33a40a1cc0 auth_method 0 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_done global_id 54591 payload 274 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 -- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 msgr2=0x7f33a40a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 --2- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 0x7f33a40a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 -- 172.21.15.47:0/1066957946 >> v1:172.21.15.179:6789/0 conn(0x7f33a4099020 legacy=0x7f33a4099480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 monclient: found mon.noname-d 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 -- 172.21.15.47:0/1066957946 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a409ddb0 con 0x7f33a40a1cc0 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33a98f4700 1 --2- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 0x7f33a40a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.778+0000 7f33aa0f5700 1 --2- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 0x7f33a4135d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009870 tx=0x7f33a0009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b849262333abfafc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 1 -- 172.21.15.47:0/1066957946 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a0011aa0 con 0x7f33a40a1cc0 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: dump: 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.793 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 1 -- 172.21.15.47:0/1066957946 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a0011c00 con 0x7f33a40a1cc0 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 5 monclient: authenticate success, global_id 54591 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 1 -- 172.21.15.47:0/1066957946 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a0011d70 con 0x7f33a40a1cc0 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 4 set_mon_vals no callback set 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33a90f3700 10 monclient: dump: 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.794 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 10 monclient: shutdown 2024-04-20T02:26:45.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 -- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 msgr2=0x7f33a4135d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 --2- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 0x7f33a4135d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009870 tx=0x7f33a0009b80 comp rx=0 tx=0).stop 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 -- 172.21.15.47:0/1066957946 shutdown_connections 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 --2- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a40a1cc0 0x7f33a4135d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 --2- 172.21.15.47:0/1066957946 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f33a40a20a0 0x7f33a40a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 -- 172.21.15.47:0/1066957946 >> 172.21.15.47:0/1066957946 conn(0x7f33a40a2db0 msgr2=0x7f33a40a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 -- 172.21.15.47:0/1066957946 shutdown_connections 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.779+0000 7f33ac359700 1 -- 172.21.15.47:0/1066957946 wait complete. 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 10 monclient: monmap: 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.797 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 1 librados: starting msgr at 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 1 librados: starting objecter 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 1 Processor -- start 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 1 -- start start 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 1 librados: setting wanted keys 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 1 librados: calling monclient init 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.780+0000 7f33ac359700 10 monclient: init 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient: _add_conns ranks=[6,1,0,2,4,7,8,3,5] 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 0x7f33a41c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): picked mon.noname-d con 0x7f33a4099020 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a40a1cc0 0x7f33a4134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): picked mon.noname-f con 0x7f33a40a1cc0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:45.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a40a20a0 0x7f33a41cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): picked mon.noname-a con 0x7f33a40a20a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4131840 con 0x7f33a40a20a0 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a409cf40 con 0x7f33a40a1cc0 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f33a409c9f0 con 0x7f33a4099020 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33ac359700 10 monclient(hunting): authenticate will time out at 2285.182452s 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 0x7f33a41c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 0x7f33a41c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38918/0 (socket says 172.21.15.47:38918) 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 -- 172.21.15.47:0/3975729533 learned_addr learned my addr 172.21.15.47:0/3975729533 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): get_auth_request con 0x7f33a4099020 auth_method 0 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a40a20a0 0x7f33a41cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): get_auth_request con 0x7f33a40a20a0 auth_method 0 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa8f6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33a98f4700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a40a1cc0 0x7f33a4134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient(hunting): handle_auth_done global_id 54600 payload 824 2024-04-20T02:26:45.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a40a1cc0 msgr2=0x7f33a4134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a40a1cc0 0x7f33a4134d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a40a20a0 msgr2=0x7f33a41cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33a40a20a0 0x7f33a41cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 monclient: found mon.noname-d 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33aa0f5700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394009550 con 0x7f33a4099020 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33a98f4700 10 monclient: get_auth_request con 0x7f33a40a1cc0 auth_method 0 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.781+0000 7f33a98f4700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a40a1cc0 0x7f33a4134d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33aa0f5700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 0x7f33a41c8e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33a00090b0 tx=0x7f33a0004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 client.?.objecter ms_handle_connect 0x7f33a4099020 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a0004330 con 0x7f33a4099020 2024-04-20T02:26:45.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: dump: 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.802 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a001f070 con 0x7f33a4099020 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 5 monclient: authenticate success, global_id 54600 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 10 monclient: _renew_subs 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a00094a0 con 0x7f33a4099020 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33a00044d0 con 0x7f33a4099020 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33a88f2700 4 set_mon_vals no callback set 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 10 client.54600.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f339affd700 10 monclient: dump: 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 10 monclient: _renew_subs 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a41c98f0 con 0x7f33a4099020 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.782+0000 7f33ac359700 1 librados: init done 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33ac359700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33ac359700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 0x7f33a41cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4067740 con 0x7f33a4130f10 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f33a0021030 con 0x7f33a4099020 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 0x7f33a41cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient: get_auth_request con 0x7f33a4130f10 auth_method 0 2024-04-20T02:26:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f338803a320 0x7f338803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(423..423 src has 266..423) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f33a004bbc0 con 0x7f33a4099020 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 10 client.54600.objecter ms_dispatch 0x7f33a4001550 osd_map(423..423 src has 266..423) v4 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 3 client.54600.objecter handle_osd_map got epochs [423,423] > 0 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 3 client.54600.objecter handle_osd_map decoding full epoch 423 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 10 monclient(hunting): handle_auth_done global_id 35432 payload 824 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33a40a3bd0 con 0x7f33a4130f10 2024-04-20T02:26:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 20 client.54600.objecter dump_active .. 0 homeless 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33aa8f6700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f338803a320 0x7f338803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 10 client.54600.objecter ms_handle_connect 0x7f33a4130f10 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 10 client.54600.objecter resend_mon_ops 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33a98f4700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 0x7f33a41cc8b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f33a406d4a0 tx=0x7f3394010e90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f33aa8f6700 10 monclient: get_auth_request con 0x7f338803a320 auth_method 0 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f339400ec60 con 0x7f33a4130f10 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.783+0000 7f339affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 10 monclient: dump: 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 10 client.54600.objecter ms_handle_connect 0x7f338803a320 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f33aa8f6700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f338803a320 0x7f338803c7e0 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 crypto rx=0x7f339c0012e0 tx=0x7f339c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f339400edc0 con 0x7f33a4130f10 2024-04-20T02:26:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 msgr2=0x7f33a41cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 0x7f33a41cc8b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f33a406d4a0 tx=0x7f3394010e90 comp rx=0 tx=0).stop 2024-04-20T02:26:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 msgr2=0x7f33a41cc8b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.784+0000 7f339affd700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a4130f10 0x7f33a41cc8b0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f33a406d4a0 tx=0x7f3394010e90 comp rx=0 tx=0).stop 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33ac359700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33ac359700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 0x7f33a41ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33ac359700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f33a406a400 con 0x7f33a406aac0 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 0x7f33a41ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient: get_auth_request con 0x7f33a406aac0 auth_method 0 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 10 monclient(hunting): handle_auth_done global_id 35441 payload 824 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 1 -- 172.21.15.47:0/3975729533 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f33a40a29c0 con 0x7f33a406aac0 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f339affd700 10 client.54600.objecter ms_handle_connect 0x7f33a406aac0 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f33a98f4700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 0x7f33a41ccdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f33a406d4a0 tx=0x7f3394004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f339affd700 10 client.54600.objecter resend_mon_ops 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f33940064b0 con 0x7f33a406aac0 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f339affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f339affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.792+0000 7f339affd700 10 monclient: dump: 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f3394025070 con 0x7f33a406aac0 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 msgr2=0x7f33a41ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 0x7f33a41ccdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f33a406d4a0 tx=0x7f3394004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 msgr2=0x7f33a41ccdf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f339affd700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f33a406aac0 0x7f33a41ccdf0 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f33a406d4a0 tx=0x7f3394004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:45.814 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:45.815 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:45.816 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:45.817 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:45.818 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:45.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:45.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.823 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.825 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:45.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:45.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:45.833 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:45.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:45.835 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:45.836 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 10 librados: watch_flush enter 2024-04-20T02:26:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 10 librados: watch_flush exit 2024-04-20T02:26:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 20 client.54600.objecter shutdown clearing up homeless session... 2024-04-20T02:26:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 10 client.54600.objecter successfully canceled tick 2024-04-20T02:26:45.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f338803a320 msgr2=0x7f338803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f338803a320 0x7f338803c7e0 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 crypto rx=0x7f339c0012e0 tx=0x7f339c009040 comp rx=0 tx=0).stop 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 10 monclient: shutdown 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 msgr2=0x7f33a41c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33aa8f6700 1 -- 172.21.15.47:0/3975729533 reap_dead start 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 0x7f33a41c8e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33a00090b0 tx=0x7f33a0004750 comp rx=0 tx=0).stop 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 shutdown_connections 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 --2- 172.21.15.47:0/3975729533 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f33a4099020 0x7f33a41c8e70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 >> 172.21.15.47:0/3975729533 conn(0x7f33a40a2db0 msgr2=0x7f33a41bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 shutdown_connections 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 -- 172.21.15.47:0/3975729533 wait complete. 2024-04-20T02:26:45.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.793+0000 7f33ac359700 1 librados: shutdown 2024-04-20T02:26:45.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:45.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:26:45.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.977+0000 7f5238df4700 10 monclient: get_monmap_and_config 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.977+0000 7f5238df4700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.977+0000 7f5238df4700 10 monclient: monmap: 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:45.978826+0000 2024-04-20T02:26:45.978 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:45.978826+0000 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.978+0000 7f5238df4700 1 Processor -- start 2024-04-20T02:26:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.978+0000 7f5238df4700 1 -- start start 2024-04-20T02:26:45.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.978+0000 7f5238df4700 10 monclient: init 2024-04-20T02:26:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient: _add_conns ranks=[4,2,9,3,5,1,8,7,6,10,0] 2024-04-20T02:26:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 0x7f5234071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): picked mon.noname-c con 0x7f5234071730 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f523406f020 0x7f52340a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): picked mon.noname-b con 0x7f523406f020 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52340a2db0 0x7f52340a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): picked mon.noname-a con 0x7f52340a2db0 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523409fea0 con 0x7f52340a2db0 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523409e7b0 con 0x7f523406f020 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f523406d190 con 0x7f5234071730 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5238df4700 10 monclient(hunting): authenticate will time out at 2285.380202s 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 0x7f5234071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f52340a2db0 0x7f52340a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f523406f020 0x7f52340a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 0x7f5234071b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39048/0 (socket says 172.21.15.47:39048) 2024-04-20T02:26:45.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 1 -- 172.21.15.47:0/476840180 learned_addr learned my addr 172.21.15.47:0/476840180 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): get_auth_request con 0x7f5234071730 auth_method 0 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): get_auth_request con 0x7f52340a2db0 auth_method 0 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): get_auth_request con 0x7f523406f020 auth_method 0 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient(hunting): handle_auth_done global_id 36009 payload 274 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 -- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 msgr2=0x7f5234071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 0x7f5234071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 -- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f523406f020 msgr2=0x7f52340a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f523406f020 0x7f52340a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 monclient: found mon.noname-a 2024-04-20T02:26:45.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 -- 172.21.15.47:0/476840180 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52340751e0 con 0x7f52340a2db0 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f523259c700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 0x7f5234071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5231d9b700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f523406f020 0x7f52340a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.979+0000 7f5232d9d700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52340a2db0 0x7f52340a3240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f522c009840 tx=0x7f522c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8d128eaf4bbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 1 -- 172.21.15.47:0/476840180 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522c011aa0 con 0x7f52340a2db0 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: dump: 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.994 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: _finish_auth 0 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: _check_auth_tickets 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:45.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 1 -- 172.21.15.47:0/476840180 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c011c00 con 0x7f52340a2db0 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 5 monclient: authenticate success, global_id 36009 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 1 -- 172.21.15.47:0/476840180 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522c011d70 con 0x7f52340a2db0 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 4 set_mon_vals no callback set 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f523159a700 10 monclient: dump: 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:45.996 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 10 monclient: shutdown 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 -- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52340a2db0 msgr2=0x7f52340a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:45.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52340a2db0 0x7f52340a3240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f522c009840 tx=0x7f522c009b50 comp rx=0 tx=0).stop 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 -- 172.21.15.47:0/476840180 shutdown_connections 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52340a2db0 0x7f52340a3240 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5234071730 0x7f5234071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 --2- 172.21.15.47:0/476840180 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f523406f020 0x7f52340a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 -- 172.21.15.47:0/476840180 >> 172.21.15.47:0/476840180 conn(0x7f52340a3f90 msgr2=0x7f523406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 -- 172.21.15.47:0/476840180 shutdown_connections 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.980+0000 7f5238df4700 1 -- 172.21.15.47:0/476840180 wait complete. 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.981+0000 7f5238df4700 10 monclient: build_initial_monmap 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.981+0000 7f5238df4700 10 monclient: monmap: 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:45.998 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.981+0000 7f5238df4700 1 librados: starting msgr at 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.981+0000 7f5238df4700 1 librados: starting objecter 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 Processor -- start 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 -- start start 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 librados: setting wanted keys 2024-04-20T02:26:45.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 librados: calling monclient init 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient: init 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient: _add_conns ranks=[1,8,6,4,0,3,7,2,5] 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406f020 0x7f52341cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): picked mon.noname-f con 0x7f523406f020 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5234071730 0x7f52341ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): picked mon.noname-e con 0x7f5234071730 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52340a2db0 0x7f52341241a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): picked mon.noname-d con 0x7f52340a2db0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234126650 con 0x7f523406f020 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f523409f1b0 con 0x7f52340a2db0 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5234072410 con 0x7f5234071730 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5238df4700 10 monclient(hunting): authenticate will time out at 2285.383622s 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5234071730 0x7f52341ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52340a2db0 0x7f52341241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5234071730 0x7f52341ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:52974/0 (socket says 172.21.15.47:52974) 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 1 -- 172.21.15.47:0/3291618049 learned_addr learned my addr 172.21.15.47:0/3291618049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): get_auth_request con 0x7f52340a2db0 auth_method 0 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): get_auth_request con 0x7f5234071730 auth_method 0 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406f020 0x7f52341cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): get_auth_request con 0x7f523406f020 auth_method 0 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5232d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f5231d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.982+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 10 monclient(hunting): handle_auth_done global_id 54609 payload 824 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5234071730 msgr2=0x7f52341ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5234071730 0x7f52341ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406f020 msgr2=0x7f52341cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406f020 0x7f52341cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 monclient: found mon.noname-d 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280094a0 con 0x7f52340a2db0 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5231d9b700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5234071730 0x7f52341ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5232d9d700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52340a2db0 0x7f52341241a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f523406d570 tx=0x7f522c004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 client.?.objecter ms_handle_connect 0x7f52340a2db0 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522c004410 con 0x7f52340a2db0 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: dump: 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.003 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c01e070 con 0x7f52340a2db0 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 5 monclient: authenticate success, global_id 54609 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 10 monclient: _renew_subs 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5224009b30 con 0x7f52340a2db0 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522c0045b0 con 0x7f52340a2db0 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5230d99700 4 set_mon_vals no callback set 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 10 client.54609.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5222ffd700 10 monclient: dump: 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.005 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 10 monclient: _renew_subs 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c0094a0 con 0x7f52340a2db0 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.983+0000 7f5238df4700 1 librados: init done 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5238df4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5238df4700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 0x7f52341246e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234127b20 con 0x7f523406aac0 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 0x7f52341246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f522c0036a0 con 0x7f52340a2db0 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient: get_auth_request con 0x7f523406aac0 auth_method 0 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5222ffd700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521403a2f0 0x7f521403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(423..423 src has 266..423) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f522c020070 con 0x7f52340a2db0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5222ffd700 10 client.54609.objecter ms_dispatch 0x7f5234001550 osd_map(423..423 src has 266..423) v4 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5222ffd700 3 client.54609.objecter handle_osd_map got epochs [423,423] > 0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f5222ffd700 3 client.54609.objecter handle_osd_map decoding full epoch 423 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.984+0000 7f523259c700 10 monclient(hunting): handle_auth_done global_id 54901 payload 824 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f523259c700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5234135530 con 0x7f523406aac0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5231d9b700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521403a2f0 0x7f521403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 20 client.54609.objecter dump_active .. 0 homeless 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5231d9b700 10 monclient: get_auth_request con 0x7f521403a2f0 auth_method 0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 client.54609.objecter ms_handle_connect 0x7f523406aac0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 client.54609.objecter resend_mon_ops 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f523259c700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 0x7f52341246e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f522400f3e0 tx=0x7f522400f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522400e040 con 0x7f523406aac0 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 monclient: dump: 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.009 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 client.54609.objecter ms_handle_connect 0x7f521403a2f0 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5231d9b700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521403a2f0 0x7f521403c7b0 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7f52340a2c50 tx=0x7f5228009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f522400f040 con 0x7f523406aac0 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 msgr2=0x7f52341246e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 0x7f52341246e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f522400f3e0 tx=0x7f522400f7a0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 msgr2=0x7f52341246e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.985+0000 7f5222ffd700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f523406aac0 0x7f52341246e0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f522400f3e0 tx=0x7f522400f7a0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f5238df4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f5238df4700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 0x7f5234124c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f5238df4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234068280 con 0x7f5234128440 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f523259c700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 0x7f5234124c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f523259c700 10 monclient: get_auth_request con 0x7f5234128440 auth_method 0 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f523259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f523259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.993+0000 7f523259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f523259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f523259c700 10 monclient(hunting): handle_auth_done global_id 54910 payload 824 2024-04-20T02:26:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f523259c700 1 -- 172.21.15.47:0/3291618049 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5234127ca0 con 0x7f5234128440 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f5222ffd700 10 client.54609.objecter ms_handle_connect 0x7f5234128440 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f523259c700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 0x7f5234124c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5224000f80 tx=0x7f5224004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f5222ffd700 10 client.54609.objecter resend_mon_ops 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5224023040 con 0x7f5234128440 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f5222ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f5222ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.994+0000 7f5222ffd700 10 monclient: dump: 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.014 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f5224003a20 con 0x7f5234128440 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 msgr2=0x7f5234124c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 0x7f5234124c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5224000f80 tx=0x7f5224004b40 comp rx=0 tx=0).stop 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 msgr2=0x7f5234124c20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5222ffd700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5234128440 0x7f5234124c20 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5224000f80 tx=0x7f5224004b40 comp rx=0 tx=0).stop 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:46.016 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.017 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.018 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:46.019 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.022 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.024 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:46.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.027 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:46.029 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:46.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.033 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.037 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:46.037 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 10 librados: watch_flush enter 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 10 librados: watch_flush exit 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 20 client.54609.objecter shutdown clearing up homeless session... 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 10 client.54609.objecter successfully canceled tick 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521403a2f0 msgr2=0x7f521403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f521403a2f0 0x7f521403c7b0 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7f52340a2c50 tx=0x7f5228009210 comp rx=0 tx=0).stop 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 10 monclient: shutdown 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52340a2db0 msgr2=0x7f52341241a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5232d9d700 1 -- 172.21.15.47:0/3291618049 reap_dead start 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52340a2db0 0x7f52341241a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f523406d570 tx=0x7f522c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 shutdown_connections 2024-04-20T02:26:46.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 --2- 172.21.15.47:0/3291618049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f52340a2db0 0x7f52341241a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 >> 172.21.15.47:0/3291618049 conn(0x7f52340a3f90 msgr2=0x7f5234134e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 shutdown_connections 2024-04-20T02:26:46.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 -- 172.21.15.47:0/3291618049 wait complete. 2024-04-20T02:26:46.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:45.995+0000 7f5238df4700 1 librados: shutdown 2024-04-20T02:26:46.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:46.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:26:46.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:26:46.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.178+0000 7fa38586d700 10 monclient: get_monmap_and_config 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.178+0000 7fa38586d700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.178+0000 7fa38586d700 10 monclient: monmap: 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:46.179630+0000 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:46.179630+0000 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.179 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:46.180 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:46.180 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 Processor -- start 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 -- start start 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient: init 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient: _add_conns ranks=[6,9,3,8,4,2,0,10,1,7,5] 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 0x7fa3800a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): picked mon.noname-d con 0x7fa380137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa380138170 0x7fa3800a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): picked mon.noname-a con 0x7fa380138170 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3801385b0 0x7fa3800a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): picked mon.noname-g con 0x7fa3801385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38009fa40 con 0x7fa380138170 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38009dfc0 con 0x7fa3801385b0 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800a7bc0 con 0x7fa380137d90 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa38586d700 10 monclient(hunting): authenticate will time out at 2285.580811s 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 0x7fa3800a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37e7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa380138170 0x7fa3800a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37e7fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa380138170 0x7fa3800a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34756/0 (socket says 172.21.15.47:34756) 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 0x7fa3800a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38942/0 (socket says 172.21.15.47:38942) 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37e7fc700 1 -- 172.21.15.47:0/264009466 learned_addr learned my addr 172.21.15.47:0/264009466 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37effd700 10 monclient(hunting): get_auth_request con 0x7fa380137d90 auth_method 0 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.179+0000 7fa37effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): get_auth_request con 0x7fa380138170 auth_method 0 2024-04-20T02:26:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3801385b0 0x7fa3800a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): get_auth_request con 0x7fa3801385b0 auth_method 0 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_done global_id 36018 payload 274 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 -- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 msgr2=0x7fa3800a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 0x7fa3800a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 -- 172.21.15.47:0/264009466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3801385b0 msgr2=0x7fa3800a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3801385b0 0x7fa3800a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 monclient: found mon.noname-a 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 -- 172.21.15.47:0/264009466 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800751e0 con 0x7fa380138170 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37effd700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 0x7fa3800a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37e7fc700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380138170 0x7fa3800a5c90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3740098f0 tx=0x7fa374009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6f64b3d8bbe8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 1 -- 172.21.15.47:0/264009466 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa374011aa0 con 0x7fa380138170 2024-04-20T02:26:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: dump: 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 1 -- 172.21.15.47:0/264009466 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374011c00 con 0x7fa380138170 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 1 -- 172.21.15.47:0/264009466 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa374011d70 con 0x7fa380138170 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa38586d700 5 monclient: authenticate success, global_id 36018 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa38586d700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa38586d700 4 set_mon_vals no callback set 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.180+0000 7fa37dffb700 10 monclient: dump: 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 10 monclient: shutdown 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 -- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380138170 msgr2=0x7fa3800a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380138170 0x7fa3800a5c90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3740098f0 tx=0x7fa374009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 -- 172.21.15.47:0/264009466 shutdown_connections 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3801385b0 0x7fa3800a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa380137d90 0x7fa3800a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 --2- 172.21.15.47:0/264009466 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa380138170 0x7fa3800a5c90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 -- 172.21.15.47:0/264009466 >> 172.21.15.47:0/264009466 conn(0x7fa3800aa8b0 msgr2=0x7fa3800a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 -- 172.21.15.47:0/264009466 shutdown_connections 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.181+0000 7fa38586d700 1 -- 172.21.15.47:0/264009466 wait complete. 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 10 monclient: monmap: 2024-04-20T02:26:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 1 librados: starting msgr at 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 1 librados: starting objecter 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 1 Processor -- start 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 1 -- start start 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 1 librados: setting wanted keys 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 1 librados: calling monclient init 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.182+0000 7fa38586d700 10 monclient: init 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient: _add_conns ranks=[3,7,6,4,1,0,8,2,5] 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380137d90 0x7fa3801cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): picked mon.noname-g con 0x7fa380137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa380138170 0x7fa380136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): picked mon.noname-i con 0x7fa380138170 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 0x7fa3801374b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): picked mon.noname-d con 0x7fa3801385b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801cd200 con 0x7fa380137d90 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa380123e60 con 0x7fa3801385b0 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa38009e9e0 con 0x7fa380138170 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa38586d700 10 monclient(hunting): authenticate will time out at 2285.584322s 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 0x7fa3801374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 0x7fa3801374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38958/0 (socket says 172.21.15.47:38958) 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 -- 172.21.15.47:0/4002616068 learned_addr learned my addr 172.21.15.47:0/4002616068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): get_auth_request con 0x7fa3801385b0 auth_method 0 2024-04-20T02:26:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa380138170 0x7fa380136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380137d90 0x7fa3801cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): get_auth_request con 0x7fa380138170 auth_method 0 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): get_auth_request con 0x7fa380137d90 auth_method 0 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient(hunting): handle_auth_done global_id 54627 payload 824 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa380138170 msgr2=0x7fa380136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa380138170 0x7fa380136f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380137d90 msgr2=0x7fa3801cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380137d90 0x7fa3801cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 monclient: found mon.noname-d 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700094a0 con 0x7fa3801385b0 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa37f7fe700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 0x7fa3801374b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa37800efc0 tx=0x7fa378009f60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.183+0000 7fa367fff700 10 client.?.objecter ms_handle_connect 0x7fa3801385b0 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa378009b20 con 0x7fa3801385b0 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: dump: 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37800f040 con 0x7fa3801385b0 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa37800ccc0 con 0x7fa3801385b0 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 5 monclient: authenticate success, global_id 54627 2024-04-20T02:26:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 10 monclient: _renew_subs 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa374009550 con 0x7fa3801385b0 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa37d7fa700 4 set_mon_vals no callback set 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa367fff700 10 monclient: dump: 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 10 client.54627.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 10 monclient: _renew_subs 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa380124700 con 0x7fa3801385b0 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.184+0000 7fa38586d700 1 librados: init done 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa38586d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa38586d700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 0x7fa3801208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa37800c830 con 0x7fa3801385b0 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa380068180 con 0x7fa38006aac0 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa36c03a2f0 0x7fa36c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 0x7fa3801208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient: get_auth_request con 0x7fa38006aac0 auth_method 0 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(423..423 src has 266..423) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fa37804b060 con 0x7fa3801385b0 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 10 client.54627.objecter ms_dispatch 0x7fa380001550 osd_map(423..423 src has 266..423) v4 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 3 client.54627.objecter handle_osd_map got epochs [423,423] > 0 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 3 client.54627.objecter handle_osd_map decoding full epoch 423 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 20 client.54627.objecter dump_active .. 0 homeless 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37e7fc700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa36c03a2f0 0x7fa36c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37e7fc700 10 monclient: get_auth_request con 0x7fa36c03a2f0 auth_method 0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 10 monclient(hunting): handle_auth_done global_id 22440 payload 824 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa38009f760 con 0x7fa38006aac0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37effd700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 0x7fa3801208d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fa370011bd0 tx=0x7fa370011f90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 10 client.54627.objecter ms_handle_connect 0x7fa38006aac0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 10 client.54627.objecter resend_mon_ops 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa367fff700 10 client.54627.objecter ms_handle_connect 0x7fa36c03a2f0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.185+0000 7fa37e7fc700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa36c03a2f0 0x7fa36c03c7b0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7fa374000c00 tx=0x7fa374008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa370014440 con 0x7fa38006aac0 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:26:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 10 monclient: dump: 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.209 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa37000fa90 con 0x7fa38006aac0 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 msgr2=0x7fa3801208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 0x7fa3801208d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fa370011bd0 tx=0x7fa370011f90 comp rx=0 tx=0).stop 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 msgr2=0x7fa3801208d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.186+0000 7fa367fff700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa38006aac0 0x7fa3801208d0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa38586d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa38586d700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 0x7fa380121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa38586d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38006a400 con 0x7fa380120e10 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 0x7fa380121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient: get_auth_request con 0x7fa380120e10 auth_method 0 2024-04-20T02:26:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 10 monclient(hunting): handle_auth_done global_id 22449 payload 824 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.194+0000 7fa37effd700 1 -- 172.21.15.47:0/4002616068 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa380124380 con 0x7fa380120e10 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa37effd700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 0x7fa380121220 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fa370011bd0 tx=0x7fa370004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 client.54627.objecter ms_handle_connect 0x7fa380120e10 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 client.54627.objecter resend_mon_ops 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa3700229f0 con 0x7fa380120e10 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 monclient: dump: 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.213 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.214 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fa370022580 con 0x7fa380120e10 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 msgr2=0x7fa380121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 0x7fa380121220 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fa370011bd0 tx=0x7fa370004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 msgr2=0x7fa380121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.195+0000 7fa367fff700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa380120e10 0x7fa380121220 secure :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fa370011bd0 tx=0x7fa370004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:46.215 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:46.216 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:46.217 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:46.218 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:46.219 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.220 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.221 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.223 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.224 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.226 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.227 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.229 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:46.230 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.232 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:46.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.235 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:46.236 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 10 librados: watch_flush enter 2024-04-20T02:26:46.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 10 librados: watch_flush exit 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 20 client.54627.objecter shutdown clearing up homeless session... 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 10 client.54627.objecter successfully canceled tick 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa36c03a2f0 msgr2=0x7fa36c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa36c03a2f0 0x7fa36c03c7b0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7fa374000c00 tx=0x7fa374008030 comp rx=0 tx=0).stop 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 10 monclient: shutdown 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 msgr2=0x7fa3801374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa37f7fe700 1 -- 172.21.15.47:0/4002616068 reap_dead start 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 0x7fa3801374b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa37800efc0 tx=0x7fa378009f60 comp rx=0 tx=0).stop 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 shutdown_connections 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 --2- 172.21.15.47:0/4002616068 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa3801385b0 0x7fa3801374b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 >> 172.21.15.47:0/4002616068 conn(0x7fa3800aa8b0 msgr2=0x7fa3800a5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 shutdown_connections 2024-04-20T02:26:46.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 -- 172.21.15.47:0/4002616068 wait complete. 2024-04-20T02:26:46.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.196+0000 7fa38586d700 1 librados: shutdown 2024-04-20T02:26:46.239 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:46.239 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:26:46.239 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:26:46.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.379+0000 7f4e32754700 10 monclient: get_monmap_and_config 2024-04-20T02:26:46.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.379+0000 7f4e32754700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.379+0000 7f4e32754700 10 monclient: monmap: 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:46.380339+0000 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:46.380339+0000 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.380 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.381 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.381 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.381 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:46.381 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:46.381 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 Processor -- start 2024-04-20T02:26:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 -- start start 2024-04-20T02:26:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient: init 2024-04-20T02:26:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient: _add_conns ranks=[5,8,10,0,1,2,7,6,4,9,3] 2024-04-20T02:26:46.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): picked mon.noname-h con 0x7f4e2c09fa80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 0x7f4e2c0a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): picked mon.noname-e con 0x7f4e2c09fe60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0a1f10 0x7f4e2c0a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): picked mon.noname-f con 0x7f4e2c0a1f10 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c0a0780 con 0x7f4e2c0a1f10 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c09df90 con 0x7f4e2c09fa80 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c09dcb0 con 0x7f4e2c09fe60 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e32754700 10 monclient(hunting): authenticate will time out at 2285.781682s 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 0x7f4e2c0a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 0x7f4e2c0a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:52978/0 (socket says 172.21.15.47:52978) 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/935237654 learned_addr learned my addr 172.21.15.47:0/935237654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): get_auth_request con 0x7f4e2c09fe60 auth_method 0 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e30cf1700 1 --2- 172.21.15.47:0/935237654 >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0a1f10 0x7f4e2c0a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.380+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): get_auth_request con 0x7f4e2c0a1f10 auth_method 0 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2bfff700 10 monclient(hunting): get_auth_request con 0x7f4e2c09fa80 auth_method 0 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e30cf1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_done global_id 35459 payload 274 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/935237654 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 msgr2=0x7f4e2c135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c135df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/935237654 >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0a1f10 msgr2=0x7f4e2c0a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 --2- 172.21.15.47:0/935237654 >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0a1f10 0x7f4e2c0a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 monclient: found mon.noname-e 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/935237654 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c0751e0 con 0x7f4e2c09fe60 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2b7fe700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 0x7f4e2c0a0240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e200098f0 tx=0x7f4e20009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2f435f1e23268c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 1 -- 172.21.15.47:0/935237654 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e20011aa0 con 0x7f4e2c09fe60 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:46.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: dump: 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.398 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e32754700 5 monclient: authenticate success, global_id 35459 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e32754700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 1 -- 172.21.15.47:0/935237654 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20011c00 con 0x7f4e2c09fe60 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e32754700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:46.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e32754700 4 set_mon_vals no callback set 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 1 -- 172.21.15.47:0/935237654 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e20011d70 con 0x7f4e2c09fe60 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.381+0000 7f4e2affd700 10 monclient: dump: 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.400 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 10 monclient: shutdown 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 -- 172.21.15.47:0/935237654 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 msgr2=0x7f4e2c0a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 0x7f4e2c0a0240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e200098f0 tx=0x7f4e20009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 -- 172.21.15.47:0/935237654 shutdown_connections 2024-04-20T02:26:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 --2- 172.21.15.47:0/935237654 >> v2:172.21.15.179:3300/0 conn(0x7f4e2c0a1f10 0x7f4e2c0a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 --2- 172.21.15.47:0/935237654 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4e2c09fe60 0x7f4e2c0a0240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 -- 172.21.15.47:0/935237654 >> 172.21.15.47:0/935237654 conn(0x7f4e2c0a3000 msgr2=0x7f4e2c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 -- 172.21.15.47:0/935237654 shutdown_connections 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 1 -- 172.21.15.47:0/935237654 wait complete. 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.382+0000 7f4e32754700 10 monclient: monmap: 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.402 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 1 librados: starting msgr at 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 1 librados: starting objecter 2024-04-20T02:26:46.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 1 Processor -- start 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 1 -- start start 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 1 librados: setting wanted keys 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 1 librados: calling monclient init 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.383+0000 7f4e32754700 10 monclient: init 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient: _add_conns ranks=[5,2,3,7,8,4,0,1,6] 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): picked mon.noname-h con 0x7f4e2c09fa80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 0x7f4e2c09a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): picked mon.noname-b con 0x7f4e2c09fe60 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4e2c0a1f10 0x7f4e2c09aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): picked mon.noname-g con 0x7f4e2c0a1f10 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:46.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c130710 con 0x7f4e2c09fe60 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c130960 con 0x7f4e2c0a1f10 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c09d010 con 0x7f4e2c09fa80 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e32754700 10 monclient(hunting): authenticate will time out at 2285.785141s 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 0x7f4e2c09a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 0x7f4e2c09a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47408/0 (socket says 172.21.15.47:47408) 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/2214743274 learned_addr learned my addr 172.21.15.47:0/2214743274 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): get_auth_request con 0x7f4e2c09fe60 auth_method 0 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): get_auth_request con 0x7f4e2c09fa80 auth_method 0 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4e2c0a1f10 0x7f4e2c09aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): get_auth_request con 0x7f4e2c0a1f10 auth_method 0 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e30cf1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient(hunting): handle_auth_done global_id 24266 payload 824 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 msgr2=0x7f4e2c072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c09fa80 0x7f4e2c072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4e2c0a1f10 msgr2=0x7f4e2c09aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4e2c0a1f10 0x7f4e2c09aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 monclient: found mon.noname-b 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20009550 con 0x7f4e2c09fe60 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e2b7fe700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 0x7f4e2c09a500 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009f50 tx=0x7f4e20004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e28ff9700 10 client.?.objecter ms_handle_connect 0x7f4e2c09fe60 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e28ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e20004490 con 0x7f4e2c09fe60 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e28ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e28ff9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.384+0000 7f4e28ff9700 10 monclient: dump: 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.408 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 5 monclient: authenticate success, global_id 24266 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 monclient: _renew_subs 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20003b10 con 0x7f4e2c09fe60 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c09cb50 con 0x7f4e2c09fe60 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e20003c80 con 0x7f4e2c09fe60 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e28ff9700 10 monclient: dump: 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.411 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e2a7fc700 4 set_mon_vals no callback set 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 client.24266.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 monclient: _renew_subs 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:46.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c09bca0 con 0x7f4e2c09fe60 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 1 librados: init done 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 0x7f4e2c09af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.385+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c130af0 con 0x7f4e2c06aac0 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4e20021070 con 0x7f4e2c09fe60 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 0x7f4e2c09af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient: get_auth_request con 0x7f4e2c06aac0 auth_method 0 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e1803a2f0 0x7f4e1803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(424..424 src has 266..424) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f4e2004bbf0 con 0x7f4e2c09fe60 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 10 client.24266.objecter ms_dispatch 0x7f4e2c001550 osd_map(424..424 src has 266..424) v4 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 3 client.24266.objecter handle_osd_map got epochs [424,424] > 0 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 3 client.24266.objecter handle_osd_map decoding full epoch 424 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_done global_id 64382 payload 824 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e2c135130 con 0x7f4e2c06aac0 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 20 client.24266.objecter dump_active .. 0 homeless 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e30cf1700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e1803a2f0 0x7f4e1803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e30cf1700 10 monclient: get_auth_request con 0x7f4e1803a2f0 auth_method 0 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 10 client.24266.objecter ms_handle_connect 0x7f4e2c06aac0 2024-04-20T02:26:46.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 10 client.24266.objecter resend_mon_ops 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 0x7f4e2c09af80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c011620 tx=0x7f4e1c0119e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e1c0043a0 con 0x7f4e2c06aac0 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.386+0000 7f4e28ff9700 10 monclient: dump: 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.415 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 10 client.24266.objecter ms_handle_connect 0x7f4e1803a2f0 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e30cf1700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e1803a2f0 0x7f4e1803c7b0 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c072cc0 tx=0x7f4e2400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4e1c004500 con 0x7f4e2c06aac0 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:46.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 msgr2=0x7f4e2c09af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 0x7f4e2c09af80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c011620 tx=0x7f4e1c0119e0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 msgr2=0x7f4e2c09af80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.387+0000 7f4e28ff9700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06aac0 0x7f4e2c09af80 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c011620 tx=0x7f4e1c0119e0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e32754700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e32754700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 0x7f4e2c06f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e32754700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c068180 con 0x7f4e2c06ef30 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 0x7f4e2c06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient: get_auth_request con 0x7f4e2c06ef30 auth_method 0 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 10 monclient(hunting): handle_auth_done global_id 64391 payload 824 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.395+0000 7f4e2bfff700 1 -- 172.21.15.47:0/2214743274 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4e2c09e550 con 0x7f4e2c06ef30 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 client.24266.objecter ms_handle_connect 0x7f4e2c06ef30 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 client.24266.objecter resend_mon_ops 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e2bfff700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 0x7f4e2c06f340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c011620 tx=0x7f4e1c004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e1c0158d0 con 0x7f4e2c06ef30 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 monclient: dump: 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.420 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.421 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f4e1c015460 con 0x7f4e2c06ef30 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 msgr2=0x7f4e2c06f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 0x7f4e2c06f340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c011620 tx=0x7f4e1c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 msgr2=0x7f4e2c06f340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e28ff9700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f4e2c06ef30 0x7f4e2c06f340 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c011620 tx=0x7f4e1c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:46.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:46.423 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.424 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.425 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.426 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.427 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:46.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.429 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.433 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:46.436 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:46.437 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.438 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.440 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.445 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.446 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:46.446 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e32754700 10 librados: watch_flush enter 2024-04-20T02:26:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e32754700 10 librados: watch_flush exit 2024-04-20T02:26:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e32754700 20 client.24266.objecter shutdown clearing up homeless session... 2024-04-20T02:26:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.396+0000 7f4e32754700 10 client.24266.objecter successfully canceled tick 2024-04-20T02:26:46.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e1803a2f0 msgr2=0x7f4e1803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e1803a2f0 0x7f4e1803c7b0 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c072cc0 tx=0x7f4e2400b040 comp rx=0 tx=0).stop 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 10 monclient: shutdown 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 msgr2=0x7f4e2c09a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 0x7f4e2c09a500 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009f50 tx=0x7f4e20004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e30cf1700 1 -- 172.21.15.47:0/2214743274 reap_dead start 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 shutdown_connections 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 --2- 172.21.15.47:0/2214743274 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4e2c09fe60 0x7f4e2c09a500 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 >> 172.21.15.47:0/2214743274 conn(0x7f4e2c0a3000 msgr2=0x7f4e2c133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 shutdown_connections 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 -- 172.21.15.47:0/2214743274 wait complete. 2024-04-20T02:26:46.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.397+0000 7f4e32754700 1 librados: shutdown 2024-04-20T02:26:46.450 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:46.450 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:26:46.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:26:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.588+0000 7fcc98f6b700 10 monclient: get_monmap_and_config 2024-04-20T02:26:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.588+0000 7fcc98f6b700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.588+0000 7fcc98f6b700 10 monclient: monmap: 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:46.590069+0000 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:46.590069+0000 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.589+0000 7fcc98f6b700 1 Processor -- start 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.589+0000 7fcc98f6b700 1 -- start start 2024-04-20T02:26:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.589+0000 7fcc98f6b700 10 monclient: init 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient: _add_conns ranks=[5,10,7,6,2,3,0,9,4,8,1] 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 0x7fcc94071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): picked mon.noname-h con 0x7fcc94071630 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcc94071f50 0x7fcc940a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): picked mon.noname-f con 0x7fcc94071f50 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:46.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc940a2e90 0x7fcc940a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): picked mon.noname-i con 0x7fcc940a2e90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc94072720 con 0x7fcc94071f50 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940a0820 con 0x7fcc94071630 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9409e890 con 0x7fcc940a2e90 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc98f6b700 10 monclient(hunting): authenticate will time out at 2285.991441s 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcc94071f50 0x7fcc940a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 0x7fcc94071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc940a2e90 0x7fcc940a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcc94071f50 0x7fcc940a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:60910/0 (socket says 172.21.15.47:60910) 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 0x7fcc94071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:60258/0 (socket says 172.21.15.47:60258) 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 1 -- 172.21.15.47:0/3820827019 learned_addr learned my addr 172.21.15.47:0/3820827019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc940a2e90 0x7fcc940a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:35580/0 (socket says 172.21.15.47:35580) 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): get_auth_request con 0x7fcc94071f50 auth_method 0 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): get_auth_request con 0x7fcc940a2e90 auth_method 0 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): get_auth_request con 0x7fcc94071630 auth_method 0 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.590+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_done global_id 54928 payload 274 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 -- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc940a2e90 msgr2=0x7fcc940a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc940a2e90 0x7fcc940a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 -- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 msgr2=0x7fcc94071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 0x7fcc94071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 monclient: found mon.noname-f 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 -- 172.21.15.47:0/3820827019 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc940751e0 con 0x7fcc94071f50 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9259c700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 0x7fcc94071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc94071f50 0x7fcc940a2950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc880098f0 tx=0x7fcc88009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06809f45b2e1261 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 1 -- 172.21.15.47:0/3820827019 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc88011aa0 con 0x7fcc94071f50 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: dump: 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.606 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 1 -- 172.21.15.47:0/3820827019 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88011c00 con 0x7fcc94071f50 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc98f6b700 5 monclient: authenticate success, global_id 54928 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc98f6b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 1 -- 172.21.15.47:0/3820827019 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc88011d70 con 0x7fcc94071f50 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc9159a700 10 monclient: dump: 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.608 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.609 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc98f6b700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.591+0000 7fcc98f6b700 4 set_mon_vals no callback set 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 10 monclient: shutdown 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 -- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc94071f50 msgr2=0x7fcc940a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc94071f50 0x7fcc940a2950 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc880098f0 tx=0x7fcc88009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 -- 172.21.15.47:0/3820827019 shutdown_connections 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc940a2e90 0x7fcc940a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc94071630 0x7fcc94071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/3820827019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc94071f50 0x7fcc940a2950 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 -- 172.21.15.47:0/3820827019 >> 172.21.15.47:0/3820827019 conn(0x7fcc940a7570 msgr2=0x7fcc940a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 -- 172.21.15.47:0/3820827019 shutdown_connections 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.592+0000 7fcc98f6b700 1 -- 172.21.15.47:0/3820827019 wait complete. 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 10 monclient: monmap: 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.611 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 1 librados: starting msgr at 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 1 librados: starting objecter 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 1 Processor -- start 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 1 -- start start 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 1 librados: setting wanted keys 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 1 librados: calling monclient init 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.593+0000 7fcc98f6b700 10 monclient: init 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient: _add_conns ranks=[0,2,4,6,5,3,7,1,8] 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94071630 0x7fcc941c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): picked mon.noname-a con 0x7fcc94071630 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc94071f50 0x7fcc941bcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): picked mon.noname-b con 0x7fcc94071f50 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:46.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc940a2e90 0x7fcc941bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): picked mon.noname-c con 0x7fcc940a2e90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941bd7e0 con 0x7fcc94071630 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94072380 con 0x7fcc94071f50 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941c9a60 con 0x7fcc940a2e90 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc98f6b700 10 monclient(hunting): authenticate will time out at 2285.995200s 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94071630 0x7fcc941c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94071630 0x7fcc941c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34762/0 (socket says 172.21.15.47:34762) 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 1 -- 172.21.15.47:0/926262960 learned_addr learned my addr 172.21.15.47:0/926262960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): get_auth_request con 0x7fcc94071630 auth_method 0 2024-04-20T02:26:46.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc94071f50 0x7fcc941bcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): get_auth_request con 0x7fcc94071f50 auth_method 0 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc940a2e90 0x7fcc941bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): get_auth_request con 0x7fcc940a2e90 auth_method 0 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient(hunting): handle_auth_done global_id 74155 payload 824 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc94071f50 msgr2=0x7fcc941bcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc94071f50 0x7fcc941bcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94071630 msgr2=0x7fcc941c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94071630 0x7fcc941c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 monclient: found mon.noname-c 2024-04-20T02:26:46.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc840094a0 con 0x7fcc940a2e90 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc94071f50 0x7fcc941bcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc9259c700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc94071630 0x7fcc941c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc92d9d700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc940a2e90 0x7fcc941bd2a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00ebd0 tx=0x7fcc8c00eee0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc82ffd700 10 client.?.objecter ms_handle_connect 0x7fcc940a2e90 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.594+0000 7fcc82ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc8c00cc70 con 0x7fcc940a2e90 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: dump: 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00f040 con 0x7fcc940a2e90 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 5 monclient: authenticate success, global_id 74155 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc8c00cdf0 con 0x7fcc940a2e90 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 10 monclient: _renew_subs 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc90d99700 4 set_mon_vals no callback set 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc88009550 con 0x7fcc940a2e90 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc82ffd700 10 monclient: dump: 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.618 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.619 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 10 client.74155.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 10 monclient: _renew_subs 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc941aa160 con 0x7fcc940a2e90 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.595+0000 7fcc98f6b700 1 librados: init done 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc98f6b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 0x7fcc941aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94068310 con 0x7fcc9406aac0 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcc8c00c800 con 0x7fcc940a2e90 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc7403a2f0 0x7fcc7403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 0x7fcc941aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient: get_auth_request con 0x7fcc9406aac0 auth_method 0 2024-04-20T02:26:46.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(424..424 src has 266..424) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fcc8c04ac50 con 0x7fcc940a2e90 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 10 client.74155.objecter ms_dispatch 0x7fcc94001550 osd_map(424..424 src has 266..424) v4 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 3 client.74155.objecter handle_osd_map got epochs [424,424] > 0 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 3 client.74155.objecter handle_osd_map decoding full epoch 424 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc7403a2f0 0x7fcc7403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc91d9b700 10 monclient: get_auth_request con 0x7fcc7403a2f0 auth_method 0 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 20 client.74155.objecter dump_active .. 0 homeless 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 10 monclient(hunting): handle_auth_done global_id 54610 payload 824 2024-04-20T02:26:46.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc940a4480 con 0x7fcc9406aac0 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 10 client.74155.objecter ms_handle_connect 0x7fcc9406aac0 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc9259c700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 0x7fcc941aa320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400f4a0 tx=0x7fcc8400f860 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.596+0000 7fcc82ffd700 10 client.74155.objecter resend_mon_ops 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 10 client.74155.objecter ms_handle_connect 0x7fcc7403a2f0 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc91d9b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc7403a2f0 0x7fcc7403c7b0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7fcc88000c00 tx=0x7fcc88008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc840044a0 con 0x7fcc9406aac0 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 10 monclient: dump: 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.622 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcc84004030 con 0x7fcc9406aac0 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 msgr2=0x7fcc941aa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 0x7fcc941aa320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400f4a0 tx=0x7fcc8400f860 comp rx=0 tx=0).stop 2024-04-20T02:26:46.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 msgr2=0x7fcc941aa320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.597+0000 7fcc82ffd700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc9406aac0 0x7fcc941aa320 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400f4a0 tx=0x7fcc8400f860 comp rx=0 tx=0).stop 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc98f6b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 0x7fcc941cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc98f6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9406a400 con 0x7fcc941cc3e0 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 0x7fcc941cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient: get_auth_request con 0x7fcc941cc3e0 auth_method 0 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.605+0000 7fcc9259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc9259c700 10 monclient(hunting): handle_auth_done global_id 54619 payload 824 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc9259c700 1 -- 172.21.15.47:0/926262960 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc9406f090 con 0x7fcc941cc3e0 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc9259c700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 0x7fcc941cc7f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009720 tx=0x7fcc84004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 client.74155.objecter ms_handle_connect 0x7fcc941cc3e0 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 client.74155.objecter resend_mon_ops 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc84014c30 con 0x7fcc941cc3e0 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 monclient: dump: 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.627 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7fcc840147c0 con 0x7fcc941cc3e0 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:46.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 msgr2=0x7fcc941cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 0x7fcc941cc7f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009720 tx=0x7fcc84004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 msgr2=0x7fcc941cc7f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.606+0000 7fcc82ffd700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc941cc3e0 0x7fcc941cc7f0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009720 tx=0x7fcc84004750 comp rx=0 tx=0).stop 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:46.629 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.630 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.631 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:46.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:46.633 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.635 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:46.638 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:46.640 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:46.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:46.646 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.648 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.649 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:46.650 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:46.651 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:46.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 10 librados: watch_flush enter 2024-04-20T02:26:46.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 10 librados: watch_flush exit 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 20 client.74155.objecter shutdown clearing up homeless session... 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 10 client.74155.objecter successfully canceled tick 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc7403a2f0 msgr2=0x7fcc7403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc7403a2f0 0x7fcc7403c7b0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7fcc88000c00 tx=0x7fcc88008030 comp rx=0 tx=0).stop 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 10 monclient: shutdown 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc940a2e90 msgr2=0x7fcc941bd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc940a2e90 0x7fcc941bd2a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00ebd0 tx=0x7fcc8c00eee0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc92d9d700 1 -- 172.21.15.47:0/926262960 reap_dead start 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 shutdown_connections 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 --2- 172.21.15.47:0/926262960 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc940a2e90 0x7fcc941bd2a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 >> 172.21.15.47:0/926262960 conn(0x7fcc940a7570 msgr2=0x7fcc94067730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 shutdown_connections 2024-04-20T02:26:46.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 -- 172.21.15.47:0/926262960 wait complete. 2024-04-20T02:26:46.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.607+0000 7fcc98f6b700 1 librados: shutdown 2024-04-20T02:26:46.654 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:46.654 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:26:46.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:26:46.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.792+0000 7f20f132b700 10 monclient: get_monmap_and_config 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.792+0000 7f20f132b700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.792+0000 7f20f132b700 10 monclient: monmap: 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:46.793635+0000 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:46.793635+0000 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.793 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:46.794 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 Processor -- start 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 -- start start 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient: init 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient: _add_conns ranks=[2,6,10,8,4,0,3,1,7,9,5] 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 0x7f20ec0a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): picked mon.noname-b con 0x7f20ec0a4c80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a0d20 0x7f20ec0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): picked mon.noname-d con 0x7f20ec0a0d20 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f20ec0a55a0 0x7f20ec137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): picked mon.noname-f con 0x7f20ec0a55a0 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0a0000 con 0x7f20ec0a55a0 2024-04-20T02:26:46.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0a6a10 con 0x7f20ec0a4c80 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec09e440 con 0x7f20ec0a0d20 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20f132b700 10 monclient(hunting): authenticate will time out at 2286.194665s 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 0x7f20ec0a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a0d20 0x7f20ec0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 0x7f20ec0a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47432/0 (socket says 172.21.15.47:47432) 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 1 -- 172.21.15.47:0/1304110773 learned_addr learned my addr 172.21.15.47:0/1304110773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 10 monclient(hunting): get_auth_request con 0x7f20ec0a0d20 auth_method 0 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 10 monclient(hunting): get_auth_request con 0x7f20ec0a4c80 auth_method 0 2024-04-20T02:26:46.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20eaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.793+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eb7fe700 1 --2- 172.21.15.47:0/1304110773 >> v2:172.21.15.179:3300/0 conn(0x7f20ec0a55a0 0x7f20ec137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eb7fe700 10 monclient(hunting): get_auth_request con 0x7f20ec0a55a0 auth_method 0 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 10 monclient(hunting): handle_auth_done global_id 24284 payload 274 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 -- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a0d20 msgr2=0x7f20ec0a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 --2- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a0d20 0x7f20ec0a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 -- 172.21.15.47:0/1304110773 >> v2:172.21.15.179:3300/0 conn(0x7f20ec0a55a0 msgr2=0x7f20ec137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 --2- 172.21.15.47:0/1304110773 >> v2:172.21.15.179:3300/0 conn(0x7f20ec0a55a0 0x7f20ec137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 monclient: found mon.noname-b 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 -- 172.21.15.47:0/1304110773 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0751e0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20ea7fc700 1 --2- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a0d20 0x7f20ec0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eb7fe700 1 --2- 172.21.15.47:0/1304110773 >> v2:172.21.15.179:3300/0 conn(0x7f20ec0a55a0 0x7f20ec137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20eaffd700 1 --2- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 0x7f20ec0a07e0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0098f0 tx=0x7f20dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce88b939aa515b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 1 -- 172.21.15.47:0/1304110773 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20dc011aa0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:46.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: dump: 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 1 -- 172.21.15.47:0/1304110773 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc011c00 con 0x7f20ec0a4c80 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20f132b700 5 monclient: authenticate success, global_id 24284 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20f132b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20f132b700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 1 -- 172.21.15.47:0/1304110773 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20dc011d70 con 0x7f20ec0a4c80 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20f132b700 4 set_mon_vals no callback set 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.794+0000 7f20e9ffb700 10 monclient: dump: 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.808 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 10 monclient: shutdown 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:46.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 -- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 msgr2=0x7f20ec0a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 --2- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 0x7f20ec0a07e0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0098f0 tx=0x7f20dc009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 -- 172.21.15.47:0/1304110773 shutdown_connections 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 --2- 172.21.15.47:0/1304110773 >> v2:172.21.15.179:3300/0 conn(0x7f20ec0a55a0 0x7f20ec137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 --2- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20ec0a4c80 0x7f20ec0a07e0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 --2- 172.21.15.47:0/1304110773 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a0d20 0x7f20ec0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 -- 172.21.15.47:0/1304110773 >> 172.21.15.47:0/1304110773 conn(0x7f20ec0aa280 msgr2=0x7f20ec0aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 -- 172.21.15.47:0/1304110773 shutdown_connections 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 1 -- 172.21.15.47:0/1304110773 wait complete. 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 10 monclient: build_initial_monmap 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.795+0000 7f20f132b700 10 monclient: monmap: 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:46.810 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 librados: starting msgr at 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 librados: starting objecter 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 Processor -- start 2024-04-20T02:26:46.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 -- start start 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 librados: setting wanted keys 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 librados: calling monclient init 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 10 monclient: init 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 10 monclient: _add_conns ranks=[4,6,8,3,5,7,2,1,0] 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20ec0a0d20 0x7f20ec1cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 10 monclient(hunting): picked mon.noname-c con 0x7f20ec0a0d20 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 0x7f20ec124320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 10 monclient(hunting): picked mon.noname-d con 0x7f20ec0a4c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.796+0000 7f20f132b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f20ec0a55a0 0x7f20ec124860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient(hunting): picked mon.noname-e con 0x7f20ec0a55a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1263c0 con 0x7f20ec0a0d20 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec126a80 con 0x7f20ec0a4c80 2024-04-20T02:26:46.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec09dd20 con 0x7f20ec0a55a0 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient(hunting): authenticate will time out at 2286.198034s 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 0x7f20ec124320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20ec0a0d20 0x7f20ec1cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 0x7f20ec124320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38982/0 (socket says 172.21.15.47:38982) 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20ec0a0d20 0x7f20ec1cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39074/0 (socket says 172.21.15.47:39074) 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 -- 172.21.15.47:0/3170972229 learned_addr learned my addr 172.21.15.47:0/3170972229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): get_auth_request con 0x7f20ec0a4c80 auth_method 0 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f20ec0a55a0 0x7f20ec124860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): get_auth_request con 0x7f20ec0a0d20 auth_method 0 2024-04-20T02:26:46.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): get_auth_request con 0x7f20ec0a55a0 auth_method 0 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20eb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient(hunting): handle_auth_done global_id 54645 payload 824 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 -- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f20ec0a55a0 msgr2=0x7f20ec124860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f20ec0a55a0 0x7f20ec124860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 -- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20ec0a0d20 msgr2=0x7f20ec1cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20ec0a0d20 0x7f20ec1cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 monclient: found mon.noname-d 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 -- 172.21.15.47:0/3170972229 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00094a0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20ea7fc700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 0x7f20ec124320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20e000ebb0 tx=0x7f20e000eec0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 client.?.objecter ms_handle_connect 0x7f20ec0a4c80 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20e000ccc0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: dump: 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.816 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000ce20 con 0x7f20ec0a4c80 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 5 monclient: authenticate success, global_id 54645 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient: _renew_subs 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc009550 con 0x7f20ec0a4c80 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20e00138e0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20d37fe700 10 monclient: dump: 2024-04-20T02:26:46.817 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:46.818 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.797+0000 7f20e97fa700 4 set_mon_vals no callback set 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.798+0000 7f20f132b700 10 client.54645.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.798+0000 7f20f132b700 10 monclient: _renew_subs 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.798+0000 7f20f132b700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.798+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec125eb0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.798+0000 7f20f132b700 1 librados: init done 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.798+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f20e0013470 con 0x7f20ec0a4c80 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20f132b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20f132b700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20f132b700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec0008d0 con 0x7f20ec0a4c80 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20d37fe700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f20d803a2f0 0x7f20d803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(424..424 src has 266..424) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f20e004a960 con 0x7f20ec0a4c80 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20d37fe700 10 client.54645.objecter ms_dispatch 0x7f20ec001550 osd_map(424..424 src has 266..424) v4 2024-04-20T02:26:46.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20d37fe700 3 client.54645.objecter handle_osd_map got epochs [424,424] > 0 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20d37fe700 3 client.54645.objecter handle_osd_map decoding full epoch 424 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20d37fe700 20 client.54645.objecter dump_active .. 0 homeless 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20eaffd700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f20d803a2f0 0x7f20d803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.799+0000 7f20eaffd700 10 monclient: get_auth_request con 0x7f20d803a2f0 auth_method 0 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.800+0000 7f20d37fe700 10 client.54645.objecter ms_handle_connect 0x7f20d803a2f0 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.800+0000 7f20eaffd700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f20d803a2f0 0x7f20d803c7b0 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000c00 tx=0x7f20dc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.804+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f20e0013720 con 0x7f20ec0a4c80 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.804+0000 7f20d37fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:46.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:46.804+0000 7f20d37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.115+0000 7f20f132b700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:26:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.115+0000 7f20f132b700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:26:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.115+0000 7f20f132b700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:47.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.115+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f20ec127aa0 con 0x7f20ec0a4c80 2024-04-20T02:26:47.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.117+0000 7f20d37fe700 1 -- 172.21.15.47:0/3170972229 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f20e001b3b0 con 0x7f20ec0a4c80 2024-04-20T02:26:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.117+0000 7f20d37fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:26:47.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.117+0000 7f20d37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:47.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=10,kv=11,logm=62} crc {auth=3340494316,config=692619927,health=3851054018,kv=1530195301,logm=504098240}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=10,kv=9,logm=64} crc {auth=3340494316,config=692619927,health=3830892973,kv=2178996138,logm=2066272648}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=3340494316,config=692619927,health=2677938672,kv=1530195301,logm=1114501116}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=15,config=2,health=12,kv=10,logm=61} crc {auth=620115301,config=692619927,health=4290917231,kv=435135402,logm=2074991511}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=15,config=2,health=10,kv=11,logm=62} crc {auth=3340494316,config=692619927,health=3132313865,kv=1530195301,logm=1357068255}) 2024-04-20T02:26:47.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=13,config=2,health=12,kv=10,logm=63} crc {auth=45332002,config=692619927,health=2677938672,kv=435135402,logm=1847859301}) 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=3340494316,config=692619927,health=2677938672,kv=1530195301,logm=1782820630}) 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=10,kv=10,logm=64} crc {auth=1157641345,config=692619927,health=2840447782,kv=3421593098,logm=1353480386}) 2024-04-20T02:26:47.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.117+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=1480903455,config=692619927,health=2677938672,kv=1530195301,logm=955901644}) 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3554964814}) 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2425474026}) 2024-04-20T02:26:47.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2631660836}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2616928874}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3054024771}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1048551845}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3041169989}) 2024-04-20T02:26:47.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1569199874}) 2024-04-20T02:26:47.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.120+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2292943306}) 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.120+0000 7f20f132b700 10 librados: watch_flush enter 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 10 librados: watch_flush exit 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 20 client.54645.objecter shutdown clearing up homeless session... 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 10 client.54645.objecter successfully canceled tick 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f20d803a2f0 msgr2=0x7f20d803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f20d803a2f0 0x7f20d803c7b0 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000c00 tx=0x7f20dc008030 comp rx=0 tx=0).stop 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 10 monclient: shutdown 2024-04-20T02:26:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 msgr2=0x7f20ec124320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 0x7f20ec124320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20e000ebb0 tx=0x7f20e000eec0 comp rx=0 tx=0).stop 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 shutdown_connections 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f20d803a2f0 0x7f20d803c7b0 unknown :-1 s=CLOSED pgs=3253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f20ec0a55a0 0x7f20ec124860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20ec0a0d20 0x7f20ec1cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 --2- 172.21.15.47:0/3170972229 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f20ec0a4c80 0x7f20ec124320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 >> 172.21.15.47:0/3170972229 conn(0x7f20ec0aa280 msgr2=0x7f20ec0a75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 shutdown_connections 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 -- 172.21.15.47:0/3170972229 wait complete. 2024-04-20T02:26:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:47.122+0000 7f20f132b700 1 librados: shutdown 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3890513473}) 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3058203534}) 2024-04-20T02:26:47.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2772888734}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=906211671}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=782269387}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1836847620}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=692633096}) 2024-04-20T02:26:47.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3422546114}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2774855699}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1548515685}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=183491550}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=595956662}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1988213589}) 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1138768988}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2244639302}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1245615149}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3211437062}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=961860873}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1469514175}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4078113409}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4006647468}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4066034318}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4138896655}) 2024-04-20T02:26:47.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3099971210}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2433864293}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1435577591}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1266400841}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3411446029}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3253419484}) 2024-04-20T02:26:47.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=485324217}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=727920654}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=874741063}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3378606690}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1177552942}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=106448836}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=205292634}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1624217244}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1449428575}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2311317220}) 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1497795138}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3288891917}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1448037183}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1228032668}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=854127403}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1891677666}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2556988796}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=587613729}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=974453681}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3556930571}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1998500746}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1731557723}) 2024-04-20T02:26:47.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=978376486}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=58093933}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=620820312}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2171043686}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=849614828}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=305229720}) 2024-04-20T02:26:47.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1933688258}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=69241575}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2097588849}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=846109766}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3258744689}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3611047472}) 2024-04-20T02:26:47.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.138+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1997055213}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203933309}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2731582760}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1386018788}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2262051804}) 2024-04-20T02:26:47.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2877847550}) 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2544204154}) 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3016783264}) 2024-04-20T02:26:47.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=40760148}) 2024-04-20T02:26:47.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.140+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.140+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2666010517}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=111020553}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1626778363}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=31734686}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1316657931}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1459302745}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=6503038}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083789580}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3280134926}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1326916152}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2860923260}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1276369432}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2049564745}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536820210}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1439673505}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.142+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.143+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=901269300}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.143+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.143+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225054019}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.143+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3448464020}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3810784757}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2231754839}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2144579304}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2309497739}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2377494162}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4244737091}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2973994640}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277374109}) 2024-04-20T02:26:47.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.144+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694782978}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2231285650}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977242867}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3643341891}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1509787713}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1766301466}) 2024-04-20T02:26:47.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=383304854}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4268634026}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3095962191}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=485987524}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1491725379}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=788859066}) 2024-04-20T02:26:47.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3500846723}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2777145349}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1564715586}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1561965022}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2313824028}) 2024-04-20T02:26:47.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3376482991}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4119872913}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583024593}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1812528519}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=89969859}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1813864756}) 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3092631775}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802684584}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3908666945}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=844383880}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3916226102}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159177913}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4072830593}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3054795195}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3954003115}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3081539488}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3385056422}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1291966664}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1251257337}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3739926612}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2846599655}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2947766814}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2374862427}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=243218572}) 2024-04-20T02:26:47.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370254909}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3699767120}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=489227803}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2450678905}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.153+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1009921249}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=786978342}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4080628407}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2749307977}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3834714132}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1678209987}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3022581655}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1702158717}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912900785}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=934494601}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2584372160}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3492618729}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=466178143}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=192083514}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1478391585}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1445094332}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1974984429}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2690180042}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.156+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=476106018}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=273511530}) 2024-04-20T02:26:47.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=343786681}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3109075015}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2221216656}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3595858647}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2849790229}) 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3555070557}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2952536978}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3139851477}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3944177187}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=72775826}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1805686539}) 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=161314960}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4114992432}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1742136863}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=917651861}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558198147}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1252744195}) 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3776455797}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1207022387}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=292453948}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4121501098}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3030724319}) 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3214919737}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1454192257}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291519248}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3323738489}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1153297555}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=316459464}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2452480108}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=572707829}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3548719568}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=313753222}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=502107880}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2057635982}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.163+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2250168674}) 2024-04-20T02:26:47.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=611467435}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3394446933}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4108929110}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4208320881}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4052661192}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1993763786}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2251002290}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1162619952}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2045975957}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2137038098}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961140450}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1146526760}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2498131987}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2693063365}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2746587448}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1794058626}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2087298213}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1054707345}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1605309966}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=304468006}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4164021368}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3860177547}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4234567044}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2119209385}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2906359419}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1289400108}) 2024-04-20T02:26:47.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.168+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2155513138}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=367457330}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=119026948}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2520673155}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=392600677}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=71821353}) 2024-04-20T02:26:47.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2542275649}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=541352532}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322360228}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3743591525}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2265678373}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3438086953}) 2024-04-20T02:26:47.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2832371321}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1604564037}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=209348779}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1418296227}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=799534486}) 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919344531}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3022906048}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.172+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.172+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=626639130}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3885193010}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1622956974}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=275386176}) 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=553279895}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2732014315}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3182375015}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389137679}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3496588545}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4018182894}) 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1650313184}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3520426071}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3823568272}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1061132266}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=555432092}) 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=60984892}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078789829}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3181880369}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594630928}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1199044430}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1126993291}) 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2711988348}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4106269404}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=455018495}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1238001982}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3262478626}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2996651514}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=974886085}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2295463130}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2155588663}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=147231412}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3251480027}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2534295681}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=252438815}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484870087}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=19214919}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1722719538}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=122523856}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2885928268}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=900526902}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3743356800}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2398614761}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3531424582}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119586621}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.179+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=164991438}) 2024-04-20T02:26:47.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3392384873}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3946679440}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2578487129}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2996412678}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514818504}) 2024-04-20T02:26:47.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503237667}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3274915940}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2836868554}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=566994845}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3919370180}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1091687770}) 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3544248622}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2698536597}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=61741956}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2953199170}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=670967214}) 2024-04-20T02:26:47.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3858417081}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3815753731}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3224851024}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4223668547}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1384576011}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=327173708}) 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3884224865}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3687489275}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2433491643}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2169782162}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1417926755}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.185+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.185+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.185+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=791039999}) 2024-04-20T02:26:47.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.185+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3507034663}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519959028}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4222126805}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=724498941}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2046056610}) 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2501993389}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1838960053}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1949235632}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2542609032}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1981021143}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3607927525}) 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.320 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4204609050}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1092944991}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4243558986}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3938281918}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3229373648}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.188+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3200592206}) 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.321 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1531688601}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100458376}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2424824910}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3904553213}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4220346937}) 2024-04-20T02:26:47.322 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4251260622}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3186020670}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=721454835}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.190+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3997007304}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541438656}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1152640795}) 2024-04-20T02:26:47.323 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=846295458}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2607171705}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14114941}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=480465438}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=694601181}) 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.324 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=856864295}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.192+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2494573666}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462541987}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919430555}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3644037984}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=89085683}) 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.325 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=550154823}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2770760968}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4087775892}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697713381}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=945009036}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4251497934}) 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.326 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2828157185}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=479372013}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2772915791}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2084017333}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=839950045}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.327 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2178714065}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3524842787}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2568650105}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3482866308}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2697904429}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3912627805}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=908970027}) 2024-04-20T02:26:47.328 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1400918337}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3295248483}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4253552053}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3562018648}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1865997182}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36779888}) 2024-04-20T02:26:47.329 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3125751175}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1473958943}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1059566021}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=313380570}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1249423292}) 2024-04-20T02:26:47.330 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2700531570}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702056609}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3419753681}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2390133195}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320266433}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4274115557}) 2024-04-20T02:26:47.331 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=148491984}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2425641577}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1609115583}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3511101614}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280085675}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.201+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2153425660}) 2024-04-20T02:26:47.332 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=691896448}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3084298315}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2383006926}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1724721634}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4121145908}) 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.333 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1637810770}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=934851373}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818008662}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3397161657}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1559800564}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3659374260}) 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.334 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3551280446}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3359479308}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2768650693}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1663149872}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.204+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.205+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.205+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3094238767}) 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.205+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.335 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.205+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062698724}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.205+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3101271049}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028442350}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3747319914}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=711626663}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2933368940}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.336 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3411141792}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1240579275}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1291322888}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2271113933}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.206+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423035821}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3961495484}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3125422075}) 2024-04-20T02:26:47.337 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=813380827}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4178846298}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=435966406}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3115882340}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.338 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4285530592}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3460124333}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.208+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1054885491}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1950175870}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=417436509}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=603093450}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.339 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2320163723}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=830563479}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1534001708}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=675959022}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3555110625}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3705335641}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.340 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1950554430}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2270281239}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2327444104}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1421132321}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.211+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=222451073}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2434622344}) 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.341 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1969676749}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3030996984}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.212+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3141860362}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2577976744}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3947393072}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1893725376}) 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.342 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4032982347}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=349157937}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1283714268}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=283078865}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2709935077}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.214+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4190988786}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3917867038}) 2024-04-20T02:26:47.343 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=741123089}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=912980390}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1953322405}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3218344567}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2913174479}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.344 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2764917355}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3717486776}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.215+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2462427751}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=755989607}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=534925998}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=409505882}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.345 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2349204128}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4093538351}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4141280347}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2054878301}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3907512405}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.217+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3375132269}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.346 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=651541689}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=888112514}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1026462509}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.218+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1793989039}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1535030020}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2067022388}) 2024-04-20T02:26:47.347 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=541769007}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779818601}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.219+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3672376891}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968521604}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3552231619}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1330223950}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.348 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4252415442}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.220+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3149490282}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=721434281}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2737305136}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2580613464}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.221+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3065536877}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638456444}) 2024-04-20T02:26:47.349 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267680}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3359542376}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2064184537}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3013151022}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.350 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3896171512}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.222+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2251277256}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.223+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.223+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1122663739}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.223+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3166555567}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3355643013}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001962086}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.351 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1203390505}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2617807207}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2311941312}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4122709045}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3110835506}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4212724766}) 2024-04-20T02:26:47.352 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.224+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3686431041}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959195288}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2014714865}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3681830405}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1724390285}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1684540241}) 2024-04-20T02:26:47.353 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3250653205}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2234209656}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2824739966}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.226+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=560254746}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1149240908}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2724499393}) 2024-04-20T02:26:47.354 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2798750812}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3913266953}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1991950537}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.227+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3812151676}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1917146697}) 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.355 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262256189}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.228+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2347609953}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977378742}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=102748227}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4208532991}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1112064761}) 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.356 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2781111133}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4247682657}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3405302555}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2035284336}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.229+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2840905396}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1063794264}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.357 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3064576858}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2258188238}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2603062338}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2004845106}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1203152165}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.358 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2042768181}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=62853998}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.231+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2594912517}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3491942997}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=956139072}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3991136747}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.359 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=772990987}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1847893930}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=520712374}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1137151113}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=170787692}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.232+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=559747884}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1233988947}) 2024-04-20T02:26:47.360 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3150851844}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3370076761}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1712068228}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4160806426}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2711480029}) 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.361 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4220789184}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=200526089}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.234+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2260200139}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.235+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.235+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.235+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=313271075}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.235+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.235+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.235+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1642531888}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1664452110}) 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.362 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=91931589}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1988468120}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1666498451}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1931752569}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4027448587}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.236+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3381465025}) 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.363 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2488509380}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1966723558}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3454498282}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2611276101}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2371692325}) 2024-04-20T02:26:47.364 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1799207742}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4271546789}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2097306314}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.238+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4037745029}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3297377772}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1752646828}) 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.365 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3786985965}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1056568119}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2466533432}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2468519487}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=668295364}) 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.366 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4131839389}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=471112490}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=132694680}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=720299838}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1666385129}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2780622243}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.367 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2242399335}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=921958134}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1501364794}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170630702}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1124152737}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=476098215}) 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.368 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=916012884}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1950272309}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2512382572}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=258483885}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=200243504}) 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.369 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1816827515}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391084286}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1878815541}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.257+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.257+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.257+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3247299611}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.257+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=304809514}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898020167}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.370 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3069957460}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4139785795}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1291586893}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3126740437}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3366264576}) 2024-04-20T02:26:47.371 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=615888590}) 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687825113}) 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509754500}) 2024-04-20T02:26:47.406 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1112742496}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2657641005}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1876321331}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.291+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3009455402}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2788529487}) 2024-04-20T02:26:47.407 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859247474}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=92244921}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2337903554}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=400610423}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=43529993}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2894258287}) 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.408 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2524730168}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4093925050}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2550814212}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.296+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3268293933}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.296+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3212320808}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1646985158}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.409 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=702492335}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3075628558}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1697258093}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=742453687}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=992265677}) 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.410 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4042131379}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3225993712}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1983844387}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1757741833}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1610252199}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=642489806}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.411 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3030949908}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2216998846}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2531799014}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1902994626}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053263630}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647422118}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.412 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=192753936}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=973439138}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4015664608}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=222679208}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1002865814}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.301+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2185298019}) 2024-04-20T02:26:47.413 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3861018697}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536256882}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2885394184}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705393216}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3171351977}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4276582430}) 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.414 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3638458727}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1544223781}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4270557848}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2925152553}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3309070952}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2858030223}) 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.415 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1018426556}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1869431220}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2925598767}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3290099981}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3237619333}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.416 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=412116674}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3411485607}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622224785}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1141426308}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3315374225}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2769921481}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=501329142}) 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.417 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2671331504}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3761249196}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=818655748}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4016986278}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2792817109}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1734265590}) 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.418 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1060904722}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4030618199}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2598232389}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3684218262}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3160810738}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.419 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=62644263}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145841435}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261086271}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557390958}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3193684303}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3487252042}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2261257496}) 2024-04-20T02:26:47.420 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3433744600}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1727527045}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3714654920}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1361060390}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2927936037}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1049405992}) 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.421 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=193594008}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1323207097}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1972933292}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2031711121}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2744996986}) 2024-04-20T02:26:47.422 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4025315147}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=781527271}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252756519}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1741017986}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2405985032}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=133566804}) 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1512680531}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2279745177}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3992087533}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=225439925}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=610975387}) 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808219778}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1620913382}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=3} crc {logm=2603683344,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=3} crc {logm=4281637138,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2006388495}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=54,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=6} crc {logm=96176079,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=53,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=3} crc {logm=1948321353,mdsmap=3585761405,mgr=2537084382,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2979802314,monmap=3385036552,osd_metadata=3566517105,osd_pg_creating=447882695,osd_snap=2085319540}) 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=56,mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=1} crc {logm=678618472,mdsmap=3585761405,mgr=565164128,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517105,osd_pg_creating=447882695,osd_snap=2462050620}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=57,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=2782994805,mdsmap=3585761405,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3169875456,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=55,mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=3661044822,mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1918622166,mon_config_key=1169884507,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=447882695,osd_snap=2866008682}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=56,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=11,monmap=3,osd_metadata=5} crc {logm=1749342013,mdsmap=2394902515,mgr=1417024152,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2868358642,monmap=3385036551,osd_metadata=2946886609}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=3,osd_metadata=4} crc {logm=2333024506,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3403331883,mon_config_key=2979802314,monmap=3385036551,osd_metadata=3303912138}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=32,osdmap=66} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=375014854,osdmap=2246055667}) 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=32,osdmap=66} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=719017941,osdmap=2741317854}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=1750332813,osdmap=1745734420}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=3834342011,osdmap=770768200}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.315+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=32,osdmap=66} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4020111860,osdmap=2987425005}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=32,osdmap=66} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=2155506562,osdmap=3971199464}) 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=30,osdmap=68} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=1489849191,osdmap=2397358827}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=31,osdmap=67} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=4037983085,osdmap=1031214988}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_metadata=1,osd_pg_creating=1,osd_snap=33,osdmap=65} crc {osd_metadata=2300547217,osd_pg_creating=447882695,osd_snap=829996834,osdmap=1638944561}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=925892707}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=438044663}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2059621569}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=29684378}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1319864332}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2798863176}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3461663832}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2148547301}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1173078656}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=81} crc {osdmap=4231658458}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=81} crc {osdmap=1367073084}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=2581495789}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=82} crc {osdmap=2019749382}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=77} crc {osdmap=2276507997}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=80} crc {osdmap=456328041}) 2024-04-20T02:26:47.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=80} crc {osdmap=2800177608}) 2024-04-20T02:26:47.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:26:47.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=81} crc {osdmap=918596640}) 2024-04-20T02:26:47.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:26:47.329+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=79} crc {osdmap=3650796208}) 2024-04-20T02:26:48.158 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:26:48.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:26:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.312+0000 7f365b1ca700 10 monclient: get_monmap_and_config 2024-04-20T02:26:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.312+0000 7f365b1ca700 10 monclient: build_initial_monmap 2024-04-20T02:26:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.312+0000 7f365b1ca700 10 monclient: monmap: 2024-04-20T02:26:48.312 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:48.313421+0000 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:48.313421+0000 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:48.313 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:48.314 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 Processor -- start 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 -- start start 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient: init 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient: _add_conns ranks=[0,5,7,6,8,9,10,4,3,1,2] 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f365409df00 addr v1:172.21.15.47:6789/0 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3654099890 0x7f36540a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): picked mon.noname-h con 0x7f3654099890 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f365413d6a0 0x7f365413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): picked mon.noname-i con 0x7f365413d6a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36540751e0 con 0x7f365409df00 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f365409ea50 con 0x7f3654099890 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3654099c70 con 0x7f365413d6a0 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f365b1ca700 10 monclient(hunting): authenticate will time out at 2287.714838s 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f3658f66700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f365409df00 0x7f36540a1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47540/0 (socket says 172.21.15.47:47540) 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.313+0000 7f3658f66700 1 -- 172.21.15.47:0/2822487619 learned_addr learned my addr 172.21.15.47:0/2822487619 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3654099890 0x7f36540a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): get_auth_request con 0x7f3654099890 auth_method 0 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f365413d6a0 0x7f365413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): get_auth_request con 0x7f365413d6a0 auth_method 0 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient(hunting): handle_auth_done global_id 54628 payload 274 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 -- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3654099890 msgr2=0x7f36540a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3654099890 0x7f36540a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 -- 172.21.15.47:0/2822487619 >> v1:172.21.15.47:6789/0 conn(0x7f365409df00 legacy=0x7f36540a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient: _finish_hunting 0 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 monclient: found mon.noname-i 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:26:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 -- 172.21.15.47:0/2822487619 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365409ed30 con 0x7f365413d6a0 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3653fff700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3654099890 0x7f36540a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f3659767700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f365413d6a0 0x7f365413fa90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f364c009dc0 tx=0x7f364c00b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ba91260e34740a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 1 -- 172.21.15.47:0/2822487619 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f364c00e070 con 0x7f365413d6a0 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: dump: 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 1 -- 172.21.15.47:0/2822487619 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c0098f0 con 0x7f365413d6a0 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f365b1ca700 5 monclient: authenticate success, global_id 54628 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f365b1ca700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 1 -- 172.21.15.47:0/2822487619 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f364c0128a0 con 0x7f365413d6a0 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f365b1ca700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f365b1ca700 4 set_mon_vals no callback set 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.314+0000 7f36537fe700 10 monclient: dump: 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 10 monclient: shutdown 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 -- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f365413d6a0 msgr2=0x7f365413fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f365413d6a0 0x7f365413fa90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f364c009dc0 tx=0x7f364c00b330 comp rx=0 tx=0).stop 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 -- 172.21.15.47:0/2822487619 shutdown_connections 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f365413d6a0 0x7f365413fa90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 --2- 172.21.15.47:0/2822487619 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3654099890 0x7f36540a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 -- 172.21.15.47:0/2822487619 >> 172.21.15.47:0/2822487619 conn(0x7f36540a6720 msgr2=0x7f36540a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 -- 172.21.15.47:0/2822487619 shutdown_connections 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.315+0000 7f365b1ca700 1 -- 172.21.15.47:0/2822487619 wait complete. 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 10 monclient: build_initial_monmap 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 10 monclient: monmap: 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 1 librados: starting msgr at 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 1 librados: starting objecter 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 1 Processor -- start 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 1 -- start start 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 1 librados: setting wanted keys 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 1 librados: calling monclient init 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.316+0000 7f365b1ca700 10 monclient: init 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient: _add_conns ranks=[1,6,0,5,3,8,2,4,7] 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 0x7f36540709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): picked mon.noname-f con 0x7f3654099890 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 0x7f3654070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): picked mon.noname-d con 0x7f365409df00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f365413d6a0 0x7f3654071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): picked mon.noname-a con 0x7f365413d6a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36540a0410 con 0x7f365413d6a0 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365409e560 con 0x7f3654099890 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3654090ea0 con 0x7f365409df00 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f365b1ca700 10 monclient(hunting): authenticate will time out at 2287.718634s 2024-04-20T02:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 0x7f3654070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 0x7f36540709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 0x7f3654070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38998/0 (socket says 172.21.15.47:38998) 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 0x7f36540709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:60928/0 (socket says 172.21.15.47:60928) 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 1 -- 172.21.15.47:0/980863012 learned_addr learned my addr 172.21.15.47:0/980863012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): get_auth_request con 0x7f365409df00 auth_method 0 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): get_auth_request con 0x7f3654099890 auth_method 0 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3653fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.317+0000 7f3658f66700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f365413d6a0 0x7f3654071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): get_auth_request con 0x7f365413d6a0 auth_method 0 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3659767700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 10 monclient(hunting): handle_auth_done global_id 54946 payload 824 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 -- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 msgr2=0x7f3654070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 0x7f3654070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 -- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f365413d6a0 msgr2=0x7f3654071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f365413d6a0 0x7f3654071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 10 monclient: _finish_hunting 0 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 monclient: found mon.noname-f 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 -- 172.21.15.47:0/980863012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36400094a0 con 0x7f3654099890 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3653fff700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 0x7f3654070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f3658f66700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 0x7f36540709f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3648001eb0 tx=0x7f364800c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 client.?.objecter ms_handle_connect 0x7f3654099890 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3648005db0 con 0x7f3654099890 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: dump: 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: _finish_auth 0 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: _check_auth_tickets 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36480070d0 con 0x7f3654099890 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 5 monclient: authenticate success, global_id 54946 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36480076b0 con 0x7f3654099890 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36527fc700 4 set_mon_vals no callback set 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f36517fa700 10 monclient: dump: 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 10 monclient: _renew_subs 2024-04-20T02:26:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c009a70 con 0x7f3654099890 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 10 client.54946.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 10 monclient: _renew_subs 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3654072180 con 0x7f3654099890 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.318+0000 7f365b1ca700 1 librados: init done 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3648005940 con 0x7f3654099890 2024-04-20T02:26:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f363803a2f0 0x7f363803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(426..426 src has 266..426) v4 ==== 5876+0+0 (secure 0 0 0) 0x7f3648049650 con 0x7f3654099890 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 10 client.54946.objecter ms_dispatch 0x7f3654001550 osd_map(426..426 src has 266..426) v4 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 3 client.54946.objecter handle_osd_map got epochs [426,426] > 0 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 3 client.54946.objecter handle_osd_map decoding full epoch 426 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f36517fa700 20 client.54946.objecter dump_active .. 0 homeless 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f365b1ca700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f365b1ca700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f365b1ca700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365406c330 con 0x7f3654099890 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f3653fff700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f363803a2f0 0x7f363803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.320+0000 7f3653fff700 10 monclient: get_auth_request con 0x7f363803a2f0 auth_method 0 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.321+0000 7f36517fa700 10 client.54946.objecter ms_handle_connect 0x7f363803a2f0 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.321+0000 7f3653fff700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f363803a2f0 0x7f363803c7b0 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 crypto rx=0x7f36540706f0 tx=0x7f3640009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.328+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f364801a360 con 0x7f3654099890 2024-04-20T02:26:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.328+0000 7f36517fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.328+0000 7f36517fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:48.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.649+0000 7f365b1ca700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:26:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.649+0000 7f365b1ca700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.649+0000 7f365b1ca700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.649+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f365406c7a0 con 0x7f3654099890 2024-04-20T02:26:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.650+0000 7f36517fa700 1 -- 172.21.15.47:0/980863012 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f3648017020 con 0x7f3654099890 2024-04-20T02:26:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.650+0000 7f36517fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:48.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.650+0000 7f36517fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:48.651 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 10 librados: watch_flush enter 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 10 librados: watch_flush exit 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 20 client.54946.objecter shutdown clearing up homeless session... 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 10 client.54946.objecter successfully canceled tick 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f363803a2f0 msgr2=0x7f363803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f363803a2f0 0x7f363803c7b0 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 crypto rx=0x7f36540706f0 tx=0x7f3640009210 comp rx=0 tx=0).stop 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 10 monclient: shutdown 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:48.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 msgr2=0x7f36540709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 0x7f36540709f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3648001eb0 tx=0x7f364800c6e0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 shutdown_connections 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f365413d6a0 0x7f3654071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f363803a2f0 0x7f363803c7b0 unknown :-1 s=CLOSED pgs=3254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3654099890 0x7f36540709f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 --2- 172.21.15.47:0/980863012 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f365409df00 0x7f3654070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 >> 172.21.15.47:0/980863012 conn(0x7f36540a6720 msgr2=0x7f365409b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 shutdown_connections 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 -- 172.21.15.47:0/980863012 wait complete. 2024-04-20T02:26:48.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.652+0000 7f365b1ca700 1 librados: shutdown 2024-04-20T02:26:48.666 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:26:48.666 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:26:48.666 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:26:48.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:26:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.801+0000 7f2e1ef56700 10 monclient: get_monmap_and_config 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.801+0000 7f2e1ef56700 10 monclient: build_initial_monmap 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.801+0000 7f2e1ef56700 10 monclient: monmap: 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:48.802689+0000 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:48.802689+0000 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:48.803 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 Processor -- start 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 -- start start 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient: init 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient: _add_conns ranks=[8,6,4,1,2,7,5,9,3,10,0] 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2e180a0a70 0x7f2e180a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): picked mon.noname-e con 0x7f2e180a0a70 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 0x7f2e180a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): picked mon.noname-d con 0x7f2e180a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:48.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2e180a3290 0x7f2e1813eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): picked mon.noname-c con 0x7f2e180a3290 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180a0260 con 0x7f2e180a3290 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1809ef90 con 0x7f2e180a2950 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1809f270 con 0x7f2e180a0a70 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.802+0000 7f2e1ef56700 10 monclient(hunting): authenticate will time out at 2288.203933s 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 0x7f2e180a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2e180a3290 0x7f2e1813eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2e180a0a70 0x7f2e180a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 0x7f2e180a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:39012/0 (socket says 172.21.15.47:39012) 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 -- 172.21.15.47:0/3862880049 learned_addr learned my addr 172.21.15.47:0/3862880049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): get_auth_request con 0x7f2e180a2950 auth_method 0 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): get_auth_request con 0x7f2e180a3290 auth_method 0 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): get_auth_request con 0x7f2e180a0a70 auth_method 0 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient(hunting): handle_auth_done global_id 54663 payload 274 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 -- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2e180a0a70 msgr2=0x7f2e180a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2e180a0a70 0x7f2e180a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 -- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2e180a3290 msgr2=0x7f2e1813eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2e180a3290 0x7f2e1813eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 monclient: found mon.noname-d 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:48.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 -- 172.21.15.47:0/3862880049 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e180751e0 con 0x7f2e180a2950 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ccf2700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2e180a0a70 0x7f2e180a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e17fff700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 0x7f2e180a2d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009920 tx=0x7f2e04009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=42fb638d809d7ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 1 -- 172.21.15.47:0/3862880049 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e04011aa0 con 0x7f2e180a2950 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: dump: 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.820 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 1 -- 172.21.15.47:0/3862880049 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e04011c00 con 0x7f2e180a2950 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ef56700 5 monclient: authenticate success, global_id 54663 2024-04-20T02:26:48.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ef56700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ef56700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 1 -- 172.21.15.47:0/3862880049 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e04011d70 con 0x7f2e180a2950 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e1ef56700 4 set_mon_vals no callback set 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.803+0000 7f2e177fe700 10 monclient: dump: 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.822 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 10 monclient: shutdown 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 msgr2=0x7f2e180a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 0x7f2e180a2d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009920 tx=0x7f2e04009c30 comp rx=0 tx=0).stop 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3862880049 shutdown_connections 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2e180a3290 0x7f2e1813eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2e180a0a70 0x7f2e180a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3862880049 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e180a2950 0x7f2e180a2d50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3862880049 >> 172.21.15.47:0/3862880049 conn(0x7f2e180a7570 msgr2=0x7f2e180a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3862880049 shutdown_connections 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.804+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3862880049 wait complete. 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 10 monclient: build_initial_monmap 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 10 monclient: monmap: 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:48.824 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 1 librados: starting msgr at 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 1 librados: starting objecter 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 1 Processor -- start 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 1 -- start start 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 1 librados: setting wanted keys 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 1 librados: calling monclient init 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.805+0000 7f2e1ef56700 10 monclient: init 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:48.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient: _add_conns ranks=[3,1,2,5,4,0,6,8,7] 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e180a0a70 0x7f2e181cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): picked mon.noname-g con 0x7f2e180a0a70 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e180a2950 0x7f2e181bab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): picked mon.noname-f con 0x7f2e180a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e180a3290 0x7f2e181bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): picked mon.noname-b con 0x7f2e180a3290 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181cd110 con 0x7f2e180a2950 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181a8b10 con 0x7f2e180a3290 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1809eab0 con 0x7f2e180a0a70 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:48.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ef56700 10 monclient(hunting): authenticate will time out at 2288.207357s 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e180a3290 0x7f2e181bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e180a3290 0x7f2e181bb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47460/0 (socket says 172.21.15.47:47460) 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 -- 172.21.15.47:0/3728404171 learned_addr learned my addr 172.21.15.47:0/3728404171 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): get_auth_request con 0x7f2e180a3290 auth_method 0 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e180a2950 0x7f2e181bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e180a0a70 0x7f2e181cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): get_auth_request con 0x7f2e180a2950 auth_method 0 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): get_auth_request con 0x7f2e180a0a70 auth_method 0 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e17fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient(hunting): handle_auth_done global_id 24293 payload 824 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e180a0a70 msgr2=0x7f2e181cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2e180a0a70 0x7f2e181cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e180a2950 msgr2=0x7f2e181bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e180a2950 0x7f2e181bab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient: _finish_hunting 0 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 monclient: found mon.noname-b 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.806+0000 7f2e1d4f3700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c0094a0 con 0x7f2e180a3290 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1d4f3700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e180a3290 0x7f2e181bb0a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000efc0 tx=0x7f2e1000c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 client.?.objecter ms_handle_connect 0x7f2e180a3290 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e1000cd40 con 0x7f2e180a3290 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: dump: 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.829 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: _finish_auth 0 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: _check_auth_tickets 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 5 monclient: authenticate success, global_id 24293 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1000f040 con 0x7f2e180a3290 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e1000cec0 con 0x7f2e180a3290 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 10 monclient: _renew_subs 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e04009550 con 0x7f2e180a3290 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e16ffd700 4 set_mon_vals no callback set 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e157fa700 10 monclient: dump: 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.831 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 10 client.24293.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 10 monclient: _renew_subs 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181aa030 con 0x7f2e180a3290 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.807+0000 7f2e1ef56700 1 librados: init done 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ef56700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 0x7f2e181a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18068310 con 0x7f2e1806aac0 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 0x7f2e181a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient: get_auth_request con 0x7f2e1806aac0 auth_method 0 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2e1000c8d0 con 0x7f2e180a3290 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_done global_id 36045 payload 824 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e180a1390 con 0x7f2e1806aac0 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e157fa700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2dfc03a2f0 0x7f2dfc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(426..426 src has 266..426) v4 ==== 5876+0+0 (secure 0 0 0) 0x7f2e1004ac60 con 0x7f2e180a3290 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e157fa700 10 client.24293.objecter ms_dispatch 0x7f2e18001550 osd_map(426..426 src has 266..426) v4 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e157fa700 3 client.24293.objecter handle_osd_map got epochs [426,426] > 0 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e157fa700 3 client.24293.objecter handle_osd_map decoding full epoch 426 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.808+0000 7f2e1ccf2700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 0x7f2e181a66b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011b50 tx=0x7f2e0c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 20 client.24293.objecter dump_active .. 0 homeless 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 client.24293.objecter ms_handle_connect 0x7f2e1806aac0 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 client.24293.objecter resend_mon_ops 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e17fff700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2dfc03a2f0 0x7f2dfc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e17fff700 10 monclient: get_auth_request con 0x7f2dfc03a2f0 auth_method 0 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e0c014440 con 0x7f2e1806aac0 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 monclient: dump: 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.835 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2e0c00fb10 con 0x7f2e1806aac0 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 msgr2=0x7f2e181a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 0x7f2e181a66b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011b50 tx=0x7f2e0c011f10 comp rx=0 tx=0).stop 2024-04-20T02:26:48.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 msgr2=0x7f2e181a66b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e1806aac0 0x7f2e181a66b0 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011b50 tx=0x7f2e0c011f10 comp rx=0 tx=0).stop 2024-04-20T02:26:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e157fa700 10 client.24293.objecter ms_handle_connect 0x7f2dfc03a2f0 2024-04-20T02:26:48.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.809+0000 7f2e17fff700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2dfc03a2f0 0x7f2dfc03c7b0 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009920 tx=0x7f2e04008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ef56700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 0x7f2e181a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ef56700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e18072a10 con 0x7f2e181a6bf0 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 0x7f2e181a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient: get_auth_request con 0x7f2e181a6bf0 auth_method 0 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:48.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 10 monclient(hunting): handle_auth_done global_id 36054 payload 824 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 1 -- 172.21.15.47:0/3728404171 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2e18067cb0 con 0x7f2e181a6bf0 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e157fa700 10 client.24293.objecter ms_handle_connect 0x7f2e181a6bf0 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e157fa700 10 client.24293.objecter resend_mon_ops 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e1ccf2700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 0x7f2e181a7000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c01c000 tx=0x7f2e0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e0c022300 con 0x7f2e181a6bf0 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e157fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.817+0000 7f2e157fa700 10 monclient: dump: 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f2e0c022580 con 0x7f2e181a6bf0 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 msgr2=0x7f2e181a7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 0x7f2e181a7000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c01c000 tx=0x7f2e0c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 msgr2=0x7f2e181a7000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e157fa700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e181a6bf0 0x7f2e181a7000 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c01c000 tx=0x7f2e0c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:48.842 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:48.843 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:48.844 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:48.845 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:48.846 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:48.847 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:48.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.850 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:48.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.856 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:48.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.859 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:48.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:48.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:48.862 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:48.863 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 10 librados: watch_flush enter 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 10 librados: watch_flush exit 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 20 client.24293.objecter shutdown clearing up homeless session... 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 10 client.24293.objecter successfully canceled tick 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2dfc03a2f0 msgr2=0x7f2dfc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2dfc03a2f0 0x7f2dfc03c7b0 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009920 tx=0x7f2e04008030 comp rx=0 tx=0).stop 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 10 monclient: shutdown 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e180a3290 msgr2=0x7f2e181bb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 --2- 172.21.15.47:0/3728404171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e180a3290 0x7f2e181bb0a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000efc0 tx=0x7f2e1000c0c0 comp rx=0 tx=0).stop 2024-04-20T02:26:48.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1d4f3700 1 -- 172.21.15.47:0/3728404171 reap_dead start 2024-04-20T02:26:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1d4f3700 1 -- 172.21.15.47:0/3728404171 reap_dead start 2024-04-20T02:26:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 shutdown_connections 2024-04-20T02:26:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 >> 172.21.15.47:0/3728404171 conn(0x7f2e180a7570 msgr2=0x7f2e181bd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 shutdown_connections 2024-04-20T02:26:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 -- 172.21.15.47:0/3728404171 wait complete. 2024-04-20T02:26:48.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:48.818+0000 7f2e1ef56700 1 librados: shutdown 2024-04-20T02:26:48.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:48.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:26:48.866 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.004+0000 7f27a2005700 10 monclient: get_monmap_and_config 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.004+0000 7f27a2005700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.004+0000 7f27a2005700 10 monclient: monmap: 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:49.006047+0000 2024-04-20T02:26:49.005 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:49.006047+0000 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.006 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:49.010 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:49.010 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.005+0000 7f27a2005700 1 Processor -- start 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.005+0000 7f27a2005700 1 -- start start 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.005+0000 7f27a2005700 10 monclient: init 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient: _add_conns ranks=[10,5,3,7,8,1,6,0,9,2,4] 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f279c0a1510 0x7f279c0a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): picked mon.noname-f con 0x7f279c0a1510 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f279c09f9a0 0x7f279c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): picked mon.noname-h con 0x7f279c09f9a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09fe00 0x7f279c0a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): picked mon.noname-g con 0x7f279c09fe00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279c0720a0 con 0x7f279c0a1510 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c070640 con 0x7f279c09fe00 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f279c071d30 con 0x7f279c09f9a0 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f27a2005700 10 monclient(hunting): authenticate will time out at 2288.407253s 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f279c09f9a0 0x7f279c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f279c0a1510 0x7f279c0a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f279c09f9a0 0x7f279c0a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:60274/0 (socket says 172.21.15.47:60274) 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 1 -- 172.21.15.47:0/3569383108 learned_addr learned my addr 172.21.15.47:0/3569383108 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): get_auth_request con 0x7f279c0a1510 auth_method 0 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): get_auth_request con 0x7f279c09f9a0 auth_method 0 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09fe00 0x7f279c0a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): get_auth_request con 0x7f279c09fe00 auth_method 0 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient(hunting): handle_auth_done global_id 54964 payload 274 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 -- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f279c09f9a0 msgr2=0x7f279c0a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f279c09f9a0 0x7f279c0a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 -- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09fe00 msgr2=0x7f279c0a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09fe00 0x7f279c0a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 monclient: found mon.noname-f 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279b7fe700 1 -- 172.21.15.47:0/3569383108 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0751e0 con 0x7f279c0a1510 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.006+0000 7f279bfff700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09fe00 0x7f279c0a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279b7fe700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 0x7f279c0a18f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f278c009840 tx=0x7f278c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74571654b19bb327 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 1 -- 172.21.15.47:0/3569383108 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f278c011aa0 con 0x7f279c0a1510 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: dump: 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 1 -- 172.21.15.47:0/3569383108 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c011c00 con 0x7f279c0a1510 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 5 monclient: authenticate success, global_id 54964 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 1 -- 172.21.15.47:0/3569383108 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f278c011d70 con 0x7f279c0a1510 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 4 set_mon_vals no callback set 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f279a7fc700 10 monclient: dump: 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.030 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 10 monclient: shutdown 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 -- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 msgr2=0x7f279c0a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 0x7f279c0a18f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f278c009840 tx=0x7f278c009b50 comp rx=0 tx=0).stop 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 -- 172.21.15.47:0/3569383108 shutdown_connections 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 0x7f279c0a18f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09fe00 0x7f279c0a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 --2- 172.21.15.47:0/3569383108 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f279c09f9a0 0x7f279c0a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.007+0000 7f27a2005700 1 -- 172.21.15.47:0/3569383108 >> 172.21.15.47:0/3569383108 conn(0x7f279c0a8510 msgr2=0x7f279c0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.008+0000 7f27a2005700 1 -- 172.21.15.47:0/3569383108 shutdown_connections 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.008+0000 7f27a2005700 1 -- 172.21.15.47:0/3569383108 wait complete. 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.008+0000 7f27a2005700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.008+0000 7f27a2005700 10 monclient: monmap: 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:49.032 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.008+0000 7f27a2005700 1 librados: starting msgr at 2024-04-20T02:26:49.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.008+0000 7f27a2005700 1 librados: starting objecter 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 Processor -- start 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 -- start start 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 librados: setting wanted keys 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 librados: calling monclient init 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient: init 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient: _add_conns ranks=[3,2,1,6,5,4,7,8,0] 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09f9a0 0x7f279c11fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): picked mon.noname-g con 0x7f279c09f9a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c09fe00 0x7f279c120070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): picked mon.noname-b con 0x7f279c09fe00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 0x7f279c1205b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): picked mon.noname-f con 0x7f279c0a1510 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c121e60 con 0x7f279c0a1510 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c122f20 con 0x7f279c09fe00 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c071040 con 0x7f279c09f9a0 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.009+0000 7f27a2005700 10 monclient(hunting): authenticate will time out at 2288.410908s 2024-04-20T02:26:49.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c09fe00 0x7f279c120070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c09fe00 0x7f279c120070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47464/0 (socket says 172.21.15.47:47464) 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 -- 172.21.15.47:0/4023942603 learned_addr learned my addr 172.21.15.47:0/4023942603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): get_auth_request con 0x7f279c09fe00 auth_method 0 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09f9a0 0x7f279c11fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): get_auth_request con 0x7f279c09f9a0 auth_method 0 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 0x7f279c1205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): get_auth_request con 0x7f279c0a1510 auth_method 0 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient(hunting): handle_auth_done global_id 24302 payload 824 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09f9a0 msgr2=0x7f279c11fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f279c09f9a0 0x7f279c11fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 msgr2=0x7f279c1205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f279c0a1510 0x7f279c1205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 monclient: found mon.noname-b 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794009a70 con 0x7f279c09fe00 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f279affd700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c09fe00 0x7f279c120070 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f278400e8e0 tx=0x7f278400ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 client.?.objecter ms_handle_connect 0x7f279c09fe00 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2784013af0 con 0x7f279c09fe00 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:49.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: dump: 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.039 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 5 monclient: authenticate success, global_id 24302 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784013cb0 con 0x7f279c09fe00 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2784013e20 con 0x7f279c09fe00 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 10 monclient: _renew_subs 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c0094a0 con 0x7f279c09fe00 2024-04-20T02:26:49.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2799ffb700 4 set_mon_vals no callback set 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f2793fff700 10 monclient: dump: 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.041 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 10 client.24302.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 10 monclient: _renew_subs 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279c1237c0 con 0x7f279c09fe00 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.010+0000 7f27a2005700 1 librados: init done 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f27a2005700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f27a2005700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 0x7f279c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0682f0 con 0x7f279c06aac0 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 0x7f279c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient: get_auth_request con 0x7f279c06aac0 auth_method 0 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2784013680 con 0x7f279c09fe00 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.011+0000 7f279b7fe700 10 monclient(hunting): handle_auth_done global_id 24311 payload 824 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f279b7fe700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f279c0a1e30 con 0x7f279c06aac0 2024-04-20T02:26:49.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f277c03a320 0x7f277c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(426..426 src has 266..426) v4 ==== 5876+0+0 (secure 0 0 0) 0x7f278404a560 con 0x7f279c09fe00 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 client.24302.objecter ms_dispatch 0x7f279c001550 osd_map(426..426 src has 266..426) v4 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 3 client.24302.objecter handle_osd_map got epochs [426,426] > 0 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 3 client.24302.objecter handle_osd_map decoding full epoch 426 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 20 client.24302.objecter dump_active .. 0 homeless 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 client.24302.objecter ms_handle_connect 0x7f279c06aac0 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 client.24302.objecter resend_mon_ops 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f279b7fe700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 0x7f279c09aa30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f278c009fd0 tx=0x7f278c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f279bfff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f277c03a320 0x7f277c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f279bfff700 10 monclient: get_auth_request con 0x7f277c03a320 auth_method 0 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f278c004570 con 0x7f279c06aac0 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:49.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 monclient: dump: 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 client.24302.objecter ms_handle_connect 0x7f277c03a320 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f279bfff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f277c03a320 0x7f277c03c7e0 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 crypto rx=0x7f2794011580 tx=0x7f279400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f278c0036a0 con 0x7f279c06aac0 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 msgr2=0x7f279c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 0x7f279c09aa30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f278c009fd0 tx=0x7f278c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 msgr2=0x7f279c09aa30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.012+0000 7f2793fff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c06aac0 0x7f279c09aa30 secure :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f278c009fd0 tx=0x7f278c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f27a2005700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:49.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f27a2005700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 0x7f279c09af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f27a2005700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c06a400 con 0x7f279c123160 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 0x7f279c09af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient: get_auth_request con 0x7f279c123160 auth_method 0 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.020+0000 7f279b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f279b7fe700 10 monclient(hunting): handle_auth_done global_id 24320 payload 824 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f279b7fe700 1 -- 172.21.15.47:0/4023942603 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f279c070960 con 0x7f279c123160 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f279b7fe700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 0x7f279c09af70 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f278c009790 tx=0x7f278c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 client.24302.objecter ms_handle_connect 0x7f279c123160 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 client.24302.objecter resend_mon_ops 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f278c009320 con 0x7f279c123160 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 monclient: dump: 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.048 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f278c01ab00 con 0x7f279c123160 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 msgr2=0x7f279c09af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 0x7f279c09af70 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f278c009790 tx=0x7f278c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 msgr2=0x7f279c09af70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.021+0000 7f2793fff700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c123160 0x7f279c09af70 secure :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f278c009790 tx=0x7f278c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:49.050 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:49.051 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.052 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:49.053 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:49.054 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:49.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.058 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:49.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.060 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:49.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:49.064 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.067 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:49.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:49.072 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:49.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 10 librados: watch_flush enter 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 10 librados: watch_flush exit 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 20 client.24302.objecter shutdown clearing up homeless session... 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 10 client.24302.objecter successfully canceled tick 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f277c03a320 msgr2=0x7f277c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f277c03a320 0x7f277c03c7e0 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 crypto rx=0x7f2794011580 tx=0x7f279400b040 comp rx=0 tx=0).stop 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 10 monclient: shutdown 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c09fe00 msgr2=0x7f279c120070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 --2- 172.21.15.47:0/4023942603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f279c09fe00 0x7f279c120070 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f278400e8e0 tx=0x7f278400ebf0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f279bfff700 1 -- 172.21.15.47:0/4023942603 reap_dead start 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f279bfff700 1 -- 172.21.15.47:0/4023942603 reap_dead start 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 shutdown_connections 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 >> 172.21.15.47:0/4023942603 conn(0x7f279c0a8510 msgr2=0x7f279c0a4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 shutdown_connections 2024-04-20T02:26:49.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 -- 172.21.15.47:0/4023942603 wait complete. 2024-04-20T02:26:49.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.022+0000 7f27a2005700 1 librados: shutdown 2024-04-20T02:26:49.076 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:49.076 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:26:49.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.242+0000 7f03a6b16700 10 monclient: get_monmap_and_config 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.242+0000 7f03a6b16700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.242+0000 7f03a6b16700 10 monclient: monmap: 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:49.243788+0000 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:49.243788+0000 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:49.243 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:49.244 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.243+0000 7f03a6b16700 1 Processor -- start 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.243+0000 7f03a6b16700 1 -- start start 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.243+0000 7f03a6b16700 10 monclient: init 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient: _add_conns ranks=[10,9,4,0,3,8,1,5,6,7,2] 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03a00a2790 0x7f03a00a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): picked mon.noname-f con 0x7f03a00a2790 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03a0067bc0 0x7f03a00a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): picked mon.noname-a con 0x7f03a0067bc0 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0137940 0x7f03a0137df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): picked mon.noname-c con 0x7f03a0137940 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:49.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00a35f0 con 0x7f03a0067bc0 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a009fff0 con 0x7f03a00a2790 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03a00672f0 con 0x7f03a0137940 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a6b16700 10 monclient(hunting): authenticate will time out at 2288.645215s 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03a0067bc0 0x7f03a00a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0137940 0x7f03a0137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03a0067bc0 0x7f03a00a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34804/0 (socket says 172.21.15.47:34804) 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 -- 172.21.15.47:0/1853556461 learned_addr learned my addr 172.21.15.47:0/1853556461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): get_auth_request con 0x7f03a0067bc0 auth_method 0 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): get_auth_request con 0x7f03a0137940 auth_method 0 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a48b2700 1 --2- 172.21.15.47:0/1853556461 >> v2:172.21.15.179:3300/0 conn(0x7f03a00a2790 0x7f03a00a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a48b2700 10 monclient(hunting): get_auth_request con 0x7f03a00a2790 auth_method 0 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a48b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a48b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a48b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient(hunting): handle_auth_done global_id 36063 payload 274 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 -- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0137940 msgr2=0x7f03a0137df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 --2- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0137940 0x7f03a0137df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 -- 172.21.15.47:0/1853556461 >> v2:172.21.15.179:3300/0 conn(0x7f03a00a2790 msgr2=0x7f03a00a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 --2- 172.21.15.47:0/1853556461 >> v2:172.21.15.179:3300/0 conn(0x7f03a00a2790 0x7f03a00a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 monclient: found mon.noname-a 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:49.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 -- 172.21.15.47:0/1853556461 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00751e0 con 0x7f03a0067bc0 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a50b3700 1 --2- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0137940 0x7f03a0137df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f03a48b2700 1 --2- 172.21.15.47:0/1853556461 >> v2:172.21.15.179:3300/0 conn(0x7f03a00a2790 0x7f03a00a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.244+0000 7f039ffff700 1 --2- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0067bc0 0x7f03a00a30b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f03a006d560 tx=0x7f038c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c02a642140c529 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 1 -- 172.21.15.47:0/1853556461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f038c011aa0 con 0x7f03a0067bc0 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: dump: 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.259 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 1 -- 172.21.15.47:0/1853556461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c011c00 con 0x7f03a0067bc0 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 5 monclient: authenticate success, global_id 36063 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:49.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 1 -- 172.21.15.47:0/1853556461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f038c011d70 con 0x7f03a0067bc0 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 4 set_mon_vals no callback set 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f039f7fe700 10 monclient: dump: 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.261 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 10 monclient: shutdown 2024-04-20T02:26:49.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 -- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0067bc0 msgr2=0x7f03a00a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 --2- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0067bc0 0x7f03a00a30b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f03a006d560 tx=0x7f038c009bd0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 -- 172.21.15.47:0/1853556461 shutdown_connections 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 --2- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0137940 0x7f03a0137df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 --2- 172.21.15.47:0/1853556461 >> v2:172.21.15.179:3300/0 conn(0x7f03a00a2790 0x7f03a00a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 --2- 172.21.15.47:0/1853556461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0067bc0 0x7f03a00a30b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 -- 172.21.15.47:0/1853556461 >> 172.21.15.47:0/1853556461 conn(0x7f03a00a3eb0 msgr2=0x7f03a006c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 -- 172.21.15.47:0/1853556461 shutdown_connections 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.245+0000 7f03a6b16700 1 -- 172.21.15.47:0/1853556461 wait complete. 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 10 monclient: monmap: 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:49.263 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 1 librados: starting msgr at 2024-04-20T02:26:49.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 1 librados: starting objecter 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 1 Processor -- start 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 1 -- start start 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 1 librados: setting wanted keys 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 1 librados: calling monclient init 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.246+0000 7f03a6b16700 10 monclient: init 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient: _add_conns ranks=[3,0,1,5,6,7,8,2,4] 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03a00a2790 0x7f03a01217e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): picked mon.noname-g con 0x7f03a00a2790 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 0x7f03a0121d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): picked mon.noname-a con 0x7f03a0137940 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a0122260 0x7f03a01ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): picked mon.noname-f con 0x7f03a0122260 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:49.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0071f30 con 0x7f03a0137940 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a01228e0 con 0x7f03a0122260 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a009e380 con 0x7f03a00a2790 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a6b16700 10 monclient(hunting): authenticate will time out at 2288.648643s 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 0x7f03a0121d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 0x7f03a0121d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34820/0 (socket says 172.21.15.47:34820) 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 1 -- 172.21.15.47:0/3407005807 learned_addr learned my addr 172.21.15.47:0/3407005807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): get_auth_request con 0x7f03a0137940 auth_method 0 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f039ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a50b3700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a0122260 0x7f03a01ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a48b2700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03a00a2790 0x7f03a01217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a50b3700 10 monclient(hunting): get_auth_request con 0x7f03a0122260 auth_method 0 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a50b3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a50b3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a50b3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a48b2700 10 monclient(hunting): get_auth_request con 0x7f03a00a2790 auth_method 0 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a48b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a48b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.247+0000 7f03a48b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a50b3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a50b3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a50b3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 10 monclient(hunting): handle_auth_done global_id 36072 payload 824 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03a00a2790 msgr2=0x7f03a01217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03a00a2790 0x7f03a01217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a0122260 msgr2=0x7f03a01ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03a0122260 0x7f03a01ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 monclient: found mon.noname-a 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:49.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398009a70 con 0x7f03a0137940 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039ffff700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 0x7f03a0121d20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f038c013000 tx=0x7f038c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 client.?.objecter ms_handle_connect 0x7f03a0137940 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f038c004480 con 0x7f03a0137940 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: dump: 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c003b10 con 0x7f03a0137940 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 5 monclient: authenticate success, global_id 36072 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 10 monclient: _renew_subs 2024-04-20T02:26:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c009550 con 0x7f03a0137940 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f038c003c80 con 0x7f03a0137940 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039effd700 4 set_mon_vals no callback set 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f039d7fa700 10 monclient: dump: 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 10 client.36072.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 10 monclient: _renew_subs 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a0124cb0 con 0x7f03a0137940 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.248+0000 7f03a6b16700 1 librados: init done 2024-04-20T02:26:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a6b16700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a6b16700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 0x7f03a01cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0124e40 con 0x7f03a0067bc0 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 0x7f03a01cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient: get_auth_request con 0x7f03a0067bc0 auth_method 0 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f038c021070 con 0x7f03a0137940 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f039d7fa700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f038403a2f0 0x7f038403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 10 monclient(hunting): handle_auth_done global_id 74191 payload 824 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03a00a4c80 con 0x7f03a0067bc0 2024-04-20T02:26:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(427..427 src has 266..427) v4 ==== 6248+0+0 (secure 0 0 0) 0x7f038c04bb40 con 0x7f03a0137940 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f039d7fa700 10 client.36072.objecter ms_dispatch 0x7f03a0001550 osd_map(427..427 src has 266..427) v4 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f039d7fa700 3 client.36072.objecter handle_osd_map got epochs [427,427] > 0 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f039d7fa700 3 client.36072.objecter handle_osd_map decoding full epoch 427 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a50b3700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f038403a2f0 0x7f038403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.249+0000 7f03a48b2700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 0x7f03a01cd2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0394010820 tx=0x7f0394010be0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f03a50b3700 10 monclient: get_auth_request con 0x7f038403a2f0 auth_method 0 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 20 client.36072.objecter dump_active .. 0 homeless 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 client.36072.objecter ms_handle_connect 0x7f03a0067bc0 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 client.36072.objecter resend_mon_ops 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f039400ec60 con 0x7f03a0067bc0 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 monclient: dump: 2024-04-20T02:26:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.275 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 client.36072.objecter ms_handle_connect 0x7f038403a2f0 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f03a50b3700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f038403a2f0 0x7f038403c7b0 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 crypto rx=0x7f03980114e0 tx=0x7f039800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f039400edc0 con 0x7f03a0067bc0 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 msgr2=0x7f03a01cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 0x7f03a01cd2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0394010820 tx=0x7f0394010be0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 msgr2=0x7f03a01cd2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.250+0000 7f039d7fa700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0067bc0 0x7f03a01cd2e0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0394010820 tx=0x7f0394010be0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a6b16700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a6b16700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 0x7f03a01cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a6b16700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0068230 con 0x7f03a0124410 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 0x7f03a01cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient: get_auth_request con 0x7f03a0124410 auth_method 0 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 10 monclient(hunting): handle_auth_done global_id 74200 payload 824 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.258+0000 7f03a48b2700 1 -- 172.21.15.47:0/3407005807 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03a009f000 con 0x7f03a0124410 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 client.36072.objecter ms_handle_connect 0x7f03a0124410 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a48b2700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 0x7f03a01cd820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0394010820 tx=0x7f0394004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 client.36072.objecter resend_mon_ops 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0394021da0 con 0x7f03a0124410 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 monclient: dump: 2024-04-20T02:26:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f0394021930 con 0x7f03a0124410 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 msgr2=0x7f03a01cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 0x7f03a01cd820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0394010820 tx=0x7f0394004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 msgr2=0x7f03a01cd820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f039d7fa700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03a0124410 0x7f03a01cd820 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0394010820 tx=0x7f0394004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:26:49.282 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.283 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:49.284 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:49.285 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.286 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.287 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.290 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.291 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.292 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:49.294 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.297 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.300 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.301 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.302 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.302 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.302 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:49.302 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a6b16700 10 librados: watch_flush enter 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a6b16700 10 librados: watch_flush exit 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a6b16700 20 client.36072.objecter shutdown clearing up homeless session... 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a6b16700 10 client.36072.objecter successfully canceled tick 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f038403a2f0 msgr2=0x7f038403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.259+0000 7f03a6b16700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f038403a2f0 0x7f038403c7b0 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 crypto rx=0x7f03980114e0 tx=0x7f039800b040 comp rx=0 tx=0).stop 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 10 monclient: shutdown 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 msgr2=0x7f03a0121d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 0x7f03a0121d20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f038c013000 tx=0x7f038c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a50b3700 1 -- 172.21.15.47:0/3407005807 reap_dead start 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 shutdown_connections 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 --2- 172.21.15.47:0/3407005807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03a0137940 0x7f03a0121d20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 >> 172.21.15.47:0/3407005807 conn(0x7f03a00a3eb0 msgr2=0x7f03a00a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 shutdown_connections 2024-04-20T02:26:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 -- 172.21.15.47:0/3407005807 wait complete. 2024-04-20T02:26:49.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.260+0000 7f03a6b16700 1 librados: shutdown 2024-04-20T02:26:49.305 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:49.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:26:49.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.446+0000 7fe688af6700 10 monclient: get_monmap_and_config 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.446+0000 7fe688af6700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.446+0000 7fe688af6700 10 monclient: monmap: 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:49.447940+0000 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:49.447940+0000 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.447 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:49.448 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.447+0000 7fe688af6700 1 Processor -- start 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.447+0000 7fe688af6700 1 -- start start 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.447+0000 7fe688af6700 10 monclient: init 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient: _add_conns ranks=[1,7,5,9,2,6,8,4,10,3,0] 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe684137d90 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6840a09e0 0x7fe68413cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): picked mon.noname-i con 0x7fe6840a09e0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe68413d1c0 0x7fe68413f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): picked mon.noname-h con 0x7fe68413d1c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840751e0 con 0x7fe684137d90 2024-04-20T02:26:49.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe68409fa40 con 0x7fe68413d1c0 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe68409dfc0 con 0x7fe6840a09e0 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe688af6700 10 monclient(hunting): authenticate will time out at 2288.849240s 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe68413d1c0 0x7fe68413f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6840a09e0 0x7fe68413cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe68259c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fe684137d90 0x7fe684138170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:33046/0 (socket says 172.21.15.47:33046) 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe68259c700 1 -- 172.21.15.47:0/622952790 learned_addr learned my addr 172.21.15.47:0/622952790 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): get_auth_request con 0x7fe6840a09e0 auth_method 0 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): get_auth_request con 0x7fe68413d1c0 auth_method 0 2024-04-20T02:26:49.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient(hunting): handle_auth_done global_id 64427 payload 274 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 1 -- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6840a09e0 msgr2=0x7fe68413cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe68159a700 1 -- 172.21.15.47:0/622952790 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541104551 0 0) 0x7fe6840751e0 con 0x7fe684137d90 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 1 --2- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6840a09e0 0x7fe68413cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 1 -- 172.21.15.47:0/622952790 >> v1:172.21.15.179:6789/0 conn(0x7fe684137d90 legacy=0x7fe684138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 1 monclient: found mon.noname-h 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe682d9d700 1 -- 172.21.15.47:0/622952790 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684138880 con 0x7fe68413d1c0 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe681d9b700 1 --2- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6840a09e0 0x7fe68413cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.448+0000 7fe68159a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe682d9d700 1 --2- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe68413d1c0 0x7fe68413f5b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009dc0 tx=0x7fe67c00b330 comp rx=0 tx=0).ready entity=mon.5 client_cookie=6af7c42334fe5115 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 1 -- 172.21.15.47:0/622952790 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67c00e070 con 0x7fe68413d1c0 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: dump: 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.462 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 5 monclient: authenticate success, global_id 64427 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 1 -- 172.21.15.47:0/622952790 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c0098f0 con 0x7fe68413d1c0 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 4 set_mon_vals no callback set 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 1 -- 172.21.15.47:0/622952790 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67c0128a0 con 0x7fe68413d1c0 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe68159a700 10 monclient: dump: 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.464 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 10 monclient: shutdown 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 1 -- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe68413d1c0 msgr2=0x7fe68413f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 1 --2- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe68413d1c0 0x7fe68413f5b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009dc0 tx=0x7fe67c00b330 comp rx=0 tx=0).stop 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 1 -- 172.21.15.47:0/622952790 shutdown_connections 2024-04-20T02:26:49.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.449+0000 7fe688af6700 1 --2- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe68413d1c0 0x7fe68413f5b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.450+0000 7fe688af6700 1 --2- 172.21.15.47:0/622952790 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6840a09e0 0x7fe68413cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.450+0000 7fe688af6700 1 -- 172.21.15.47:0/622952790 >> 172.21.15.47:0/622952790 conn(0x7fe6840aa8b0 msgr2=0x7fe6840a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.450+0000 7fe688af6700 1 -- 172.21.15.47:0/622952790 shutdown_connections 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.450+0000 7fe688af6700 1 -- 172.21.15.47:0/622952790 wait complete. 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.450+0000 7fe688af6700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 10 monclient: monmap: 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:49.466 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 1 librados: starting msgr at 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 1 librados: starting objecter 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 1 Processor -- start 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 1 -- start start 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 1 librados: setting wanted keys 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 1 librados: calling monclient init 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.451+0000 7fe688af6700 10 monclient: init 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient: _add_conns ranks=[6,4,0,1,2,5,3,7,8] 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6840a09e0 0x7fe68413b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): picked mon.noname-d con 0x7fe6840a09e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe684137d90 0x7fe6841cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): picked mon.noname-c con 0x7fe684137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe68413d1c0 0x7fe6841cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): picked mon.noname-a con 0x7fe68413d1c0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68409ea50 con 0x7fe68413d1c0 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe68413c170 con 0x7fe684137d90 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe68409dc20 con 0x7fe6840a09e0 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe688af6700 10 monclient(hunting): authenticate will time out at 2288.853421s 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6840a09e0 0x7fe68413b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe684137d90 0x7fe6841cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6840a09e0 0x7fe68413b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:39018/0 (socket says 172.21.15.47:39018) 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe684137d90 0x7fe6841cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39132/0 (socket says 172.21.15.47:39132) 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 1 -- 172.21.15.47:0/227769822 learned_addr learned my addr 172.21.15.47:0/227769822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe68413d1c0 0x7fe6841cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): get_auth_request con 0x7fe6840a09e0 auth_method 0 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): get_auth_request con 0x7fe684137d90 auth_method 0 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): get_auth_request con 0x7fe68413d1c0 auth_method 0 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient(hunting): handle_auth_done global_id 74209 payload 824 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6840a09e0 msgr2=0x7fe68413b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6840a09e0 0x7fe68413b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe68413d1c0 msgr2=0x7fe6841cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe68413d1c0 0x7fe6841cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 monclient: found mon.noname-c 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe681d9b700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780094a0 con 0x7fe684137d90 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe68259c700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6840a09e0 0x7fe68413b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.452+0000 7fe682d9d700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe68413d1c0 0x7fe6841cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe681d9b700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe684137d90 0x7fe6841cc3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e9b0 tx=0x7fe67800ecc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 client.?.objecter ms_handle_connect 0x7fe684137d90 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67800ee50 con 0x7fe684137d90 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: dump: 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.471 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67800ccc0 con 0x7fe684137d90 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 5 monclient: authenticate success, global_id 74209 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 10 monclient: _renew_subs 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c009a70 con 0x7fe684137d90 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67800ce30 con 0x7fe684137d90 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe672ffd700 10 monclient: dump: 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.473 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe680d99700 4 set_mon_vals no callback set 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 10 client.74209.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:49.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 10 monclient: _renew_subs 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe684121c90 con 0x7fe684137d90 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.453+0000 7fe688af6700 1 librados: init done 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe688af6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe688af6700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 0x7fe684122ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe684068180 con 0x7fe68406aac0 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 0x7fe684122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient: get_auth_request con 0x7fe68406aac0 auth_method 0 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 10 monclient(hunting): handle_auth_done global_id 54681 payload 824 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6840a7bc0 con 0x7fe68406aac0 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe67800c850 con 0x7fe684137d90 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.454+0000 7fe68259c700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 0x7fe684122ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe674007300 tx=0x7fe6740076c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66403a2f0 0x7fe66403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 client.74209.objecter ms_handle_connect 0x7fe68406aac0 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 client.74209.objecter resend_mon_ops 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(427..427 src has 266..427) v4 ==== 6248+0+0 (secure 0 0 0) 0x7fe67804a800 con 0x7fe684137d90 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 client.74209.objecter ms_dispatch 0x7fe684001550 osd_map(427..427 src has 266..427) v4 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 3 client.74209.objecter handle_osd_map got epochs [427,427] > 0 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 3 client.74209.objecter handle_osd_map decoding full epoch 427 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 20 client.74209.objecter dump_active .. 0 homeless 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe67400fca0 con 0x7fe68406aac0 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:49.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 monclient: dump: 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.477 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe67400fe00 con 0x7fe68406aac0 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 msgr2=0x7fe684122ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 0x7fe684122ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe674007300 tx=0x7fe6740076c0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 msgr2=0x7fe684122ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe68406aac0 0x7fe684122ac0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe674007300 tx=0x7fe6740076c0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe682d9d700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66403a2f0 0x7fe66403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe682d9d700 10 monclient: get_auth_request con 0x7fe66403a2f0 auth_method 0 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe672ffd700 10 client.74209.objecter ms_handle_connect 0x7fe66403a2f0 2024-04-20T02:26:49.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.455+0000 7fe682d9d700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66403a2f0 0x7fe66403c7b0 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 crypto rx=0x7fe6840a1280 tx=0x7fe67c024040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe688af6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:49.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe688af6700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 0x7fe684135f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe688af6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe68406a400 con 0x7fe6841cce40 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 0x7fe684135f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient: get_auth_request con 0x7fe6841cce40 auth_method 0 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 10 monclient(hunting): handle_auth_done global_id 54690 payload 824 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 1 -- 172.21.15.47:0/227769822 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe6840a0000 con 0x7fe6841cce40 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe672ffd700 10 client.74209.objecter ms_handle_connect 0x7fe6841cce40 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe672ffd700 10 client.74209.objecter resend_mon_ops 2024-04-20T02:26:49.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe68259c700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 0x7fe684135f90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe674007300 tx=0x7fe674000ed0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe674022070 con 0x7fe6841cce40 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe672ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe672ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.463+0000 7fe672ffd700 10 monclient: dump: 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.482 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fe6740133f0 con 0x7fe6841cce40 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 msgr2=0x7fe684135f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 0x7fe684135f90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe674007300 tx=0x7fe674000ed0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 msgr2=0x7fe684135f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe672ffd700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6841cce40 0x7fe684135f90 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe674007300 tx=0x7fe674000ed0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:49.484 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.485 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.486 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:49.487 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.488 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:49.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.491 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:49.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:49.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.499 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.502 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.503 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:49.504 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 10 librados: watch_flush enter 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 10 librados: watch_flush exit 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 20 client.74209.objecter shutdown clearing up homeless session... 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 10 client.74209.objecter successfully canceled tick 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66403a2f0 msgr2=0x7fe66403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66403a2f0 0x7fe66403c7b0 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 crypto rx=0x7fe6840a1280 tx=0x7fe67c024040 comp rx=0 tx=0).stop 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 10 monclient: shutdown 2024-04-20T02:26:49.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe684137d90 msgr2=0x7fe6841cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 --2- 172.21.15.47:0/227769822 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe684137d90 0x7fe6841cc3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e9b0 tx=0x7fe67800ecc0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe682d9d700 1 -- 172.21.15.47:0/227769822 reap_dead start 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe682d9d700 1 -- 172.21.15.47:0/227769822 reap_dead start 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 shutdown_connections 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 >> 172.21.15.47:0/227769822 conn(0x7fe6840aa8b0 msgr2=0x7fe6840a3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 shutdown_connections 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 -- 172.21.15.47:0/227769822 wait complete. 2024-04-20T02:26:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.464+0000 7fe688af6700 1 librados: shutdown 2024-04-20T02:26:49.507 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:49.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:26:49.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:26:49.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.645+0000 7f63753c0700 10 monclient: get_monmap_and_config 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.645+0000 7f63753c0700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.645+0000 7f63753c0700 10 monclient: monmap: 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:49.646565+0000 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:49.646565+0000 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.646 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:49.647 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 1 Processor -- start 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 1 -- start start 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 10 monclient: init 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 10 monclient: _add_conns ranks=[8,3,1,0,2,5,9,6,7,10,4] 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 0x7f63700a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 10 monclient(hunting): picked mon.noname-e con 0x7f6370137d90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6370138170 0x7f63700a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.646+0000 7f63753c0700 10 monclient(hunting): picked mon.noname-g con 0x7f6370138170 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f63701385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:49.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63700751e0 con 0x7f63701385b0 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637009fa40 con 0x7f6370138170 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f637009dfc0 con 0x7f6370137d90 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 10 monclient(hunting): authenticate will time out at 2289.048068s 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 0x7f63700a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6370138170 0x7f63700a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 0x7f63700a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:53014/0 (socket says 172.21.15.47:53014) 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636f7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f63701385b0 0x7f63700a61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:33050/0 (socket says 172.21.15.47:33050) 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 -- 172.21.15.47:0/4236508422 learned_addr learned my addr 172.21.15.47:0/4236508422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): get_auth_request con 0x7f6370137d90 auth_method 0 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): get_auth_request con 0x7f6370138170 auth_method 0 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient(hunting): handle_auth_done global_id 35486 payload 274 2024-04-20T02:26:49.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 -- 172.21.15.47:0/4236508422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6370138170 msgr2=0x7f63700a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 --2- 172.21.15.47:0/4236508422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6370138170 0x7f63700a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 1 -- 172.21.15.47:0/4236508422 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554016702 0 0) 0x7f63700751e0 con 0x7f63701385b0 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 -- 172.21.15.47:0/4236508422 >> v1:172.21.15.179:6789/0 conn(0x7f63701385b0 legacy=0x7f63700a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 monclient: found mon.noname-e 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 -- 172.21.15.47:0/4236508422 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63700a7bc0 con 0x7f6370137d90 2024-04-20T02:26:49.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636e7fc700 1 --2- 172.21.15.47:0/4236508422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6370138170 0x7f63700a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636effd700 1 --2- 172.21.15.47:0/4236508422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 0x7f63700a09e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e80 tx=0x7f636800b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f38d4548314daac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 1 -- 172.21.15.47:0/4236508422 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f636800e070 con 0x7f6370137d90 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: dump: 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.663 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 1 -- 172.21.15.47:0/4236508422 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636800bd80 con 0x7f6370137d90 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 5 monclient: authenticate success, global_id 35486 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f63753c0700 4 set_mon_vals no callback set 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 1 -- 172.21.15.47:0/4236508422 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6368012860 con 0x7f6370137d90 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.647+0000 7f636dffb700 10 monclient: dump: 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.665 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 10 monclient: shutdown 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 -- 172.21.15.47:0/4236508422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 msgr2=0x7f63700a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 --2- 172.21.15.47:0/4236508422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 0x7f63700a09e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e80 tx=0x7f636800b330 comp rx=0 tx=0).stop 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 -- 172.21.15.47:0/4236508422 shutdown_connections 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 --2- 172.21.15.47:0/4236508422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370137d90 0x7f63700a09e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 --2- 172.21.15.47:0/4236508422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6370138170 0x7f63700a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 -- 172.21.15.47:0/4236508422 >> 172.21.15.47:0/4236508422 conn(0x7f63700aa8b0 msgr2=0x7f63700a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 -- 172.21.15.47:0/4236508422 shutdown_connections 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.648+0000 7f63753c0700 1 -- 172.21.15.47:0/4236508422 wait complete. 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.649+0000 7f63753c0700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.649+0000 7f63753c0700 10 monclient: monmap: 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.667 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.649+0000 7f63753c0700 1 librados: starting msgr at 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.649+0000 7f63753c0700 1 librados: starting objecter 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 Processor -- start 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 -- start start 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 librados: setting wanted keys 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 librados: calling monclient init 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient: init 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient: _add_conns ranks=[5,2,1,6,3,8,4,7,0] 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6370137d90 0x7f63701cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): picked mon.noname-h con 0x7f6370137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6370138170 0x7f6370136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): picked mon.noname-b con 0x7f6370138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63701385b0 0x7f63701373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): picked mon.noname-f con 0x7f63701385b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370137900 con 0x7f63701385b0 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6370124050 con 0x7f6370138170 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f637009ea50 con 0x7f6370137d90 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f63753c0700 10 monclient(hunting): authenticate will time out at 2289.051752s 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6370138170 0x7f6370136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6370138170 0x7f6370136e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47496/0 (socket says 172.21.15.47:47496) 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 1 -- 172.21.15.47:0/2405220186 learned_addr learned my addr 172.21.15.47:0/2405220186 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 10 monclient(hunting): get_auth_request con 0x7f6370138170 auth_method 0 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636effd700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6370137d90 0x7f63701cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.650+0000 7f636f7fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63701385b0 0x7f63701373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): get_auth_request con 0x7f6370137d90 auth_method 0 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): get_auth_request con 0x7f63701385b0 auth_method 0 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 10 monclient(hunting): handle_auth_done global_id 24329 payload 824 2024-04-20T02:26:49.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6370137d90 msgr2=0x7f63701cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6370137d90 0x7f63701cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63701385b0 msgr2=0x7f63701373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63701385b0 0x7f63701373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 monclient: found mon.noname-b 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640094a0 con 0x7f6370138170 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636e7fc700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6370138170 0x7f6370136e80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f636400ebe0 tx=0x7f636400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 client.?.objecter ms_handle_connect 0x7f6370138170 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f636400cc60 con 0x7f6370138170 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:49.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: dump: 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.673 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 5 monclient: authenticate success, global_id 24329 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636400cdc0 con 0x7f6370138170 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 10 monclient: _renew_subs 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6368009b30 con 0x7f6370138170 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63640138e0 con 0x7f6370138170 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:49.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 10 client.24329.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63577fe700 10 monclient: dump: 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.675 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f636d7fa700 4 set_mon_vals no callback set 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 10 monclient: _renew_subs 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6370122260 con 0x7f6370138170 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.651+0000 7f63753c0700 1 librados: init done 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f63753c0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f63753c0700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 0x7f63701208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6370068180 con 0x7f637006aac0 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 0x7f63701208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient: get_auth_request con 0x7f637006aac0 auth_method 0 2024-04-20T02:26:49.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6364013470 con 0x7f6370138170 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.652+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f636effd700 10 monclient(hunting): handle_auth_done global_id 35495 payload 824 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f636effd700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f637009f760 con 0x7f637006aac0 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f635c03a2f0 0x7f635c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(427..427 src has 266..427) v4 ==== 6248+0+0 (secure 0 0 0) 0x7f636404a890 con 0x7f6370138170 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 client.24329.objecter ms_dispatch 0x7f6370001550 osd_map(427..427 src has 266..427) v4 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 3 client.24329.objecter handle_osd_map got epochs [427,427] > 0 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 3 client.24329.objecter handle_osd_map decoding full epoch 427 2024-04-20T02:26:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f636effd700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 0x7f63701208d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e80 tx=0x7f6368004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f636f7fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f635c03a2f0 0x7f635c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f636f7fe700 10 monclient: get_auth_request con 0x7f635c03a2f0 auth_method 0 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 20 client.24329.objecter dump_active .. 0 homeless 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 client.24329.objecter ms_handle_connect 0x7f637006aac0 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 client.24329.objecter resend_mon_ops 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63680047f0 con 0x7f637006aac0 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 monclient: dump: 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6368004950 con 0x7f637006aac0 2024-04-20T02:26:49.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 msgr2=0x7f63701208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 0x7f63701208d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e80 tx=0x7f6368004ad0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 msgr2=0x7f63701208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f637006aac0 0x7f63701208d0 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e80 tx=0x7f6368004ad0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f63577fe700 10 client.24329.objecter ms_handle_connect 0x7f635c03a2f0 2024-04-20T02:26:49.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.653+0000 7f636f7fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f635c03a2f0 0x7f635c03c7b0 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 crypto rx=0x7f6360000f60 tx=0x7f6360004a50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f63753c0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f63753c0700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 0x7f6370121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f63753c0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f637006a400 con 0x7f6370120e10 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 0x7f6370121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient: get_auth_request con 0x7f6370120e10 auth_method 0 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 10 monclient(hunting): handle_auth_done global_id 35504 payload 824 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.661+0000 7f636effd700 1 -- 172.21.15.47:0/2405220186 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6370121760 con 0x7f6370120e10 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 client.24329.objecter ms_handle_connect 0x7f6370120e10 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 client.24329.objecter resend_mon_ops 2024-04-20T02:26:49.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f636effd700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 0x7f6370121220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6368022000 tx=0x7f6368011000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f636802acc0 con 0x7f6370120e10 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 monclient: dump: 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.684 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f636802ae20 con 0x7f6370120e10 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:49.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:49.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 msgr2=0x7f6370121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 0x7f6370121220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6368022000 tx=0x7f6368011000 comp rx=0 tx=0).stop 2024-04-20T02:26:49.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 msgr2=0x7f6370121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63577fe700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6370120e10 0x7f6370121220 secure :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6368022000 tx=0x7f6368011000 comp rx=0 tx=0).stop 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:49.687 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.688 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.689 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.691 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:49.692 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:49.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.698 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.700 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.701 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.703 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.706 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:49.708 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63753c0700 10 librados: watch_flush enter 2024-04-20T02:26:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63753c0700 10 librados: watch_flush exit 2024-04-20T02:26:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63753c0700 20 client.24329.objecter shutdown clearing up homeless session... 2024-04-20T02:26:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.662+0000 7f63753c0700 10 client.24329.objecter successfully canceled tick 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f635c03a2f0 msgr2=0x7f635c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f635c03a2f0 0x7f635c03c7b0 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 crypto rx=0x7f6360000f60 tx=0x7f6360004a50 comp rx=0 tx=0).stop 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 10 monclient: shutdown 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6370138170 msgr2=0x7f6370136e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 --2- 172.21.15.47:0/2405220186 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6370138170 0x7f6370136e80 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f636400ebe0 tx=0x7f636400eef0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f636f7fe700 1 -- 172.21.15.47:0/2405220186 reap_dead start 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f636f7fe700 1 -- 172.21.15.47:0/2405220186 reap_dead start 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 shutdown_connections 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 >> 172.21.15.47:0/2405220186 conn(0x7f63700aa8b0 msgr2=0x7f63701bf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 shutdown_connections 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 -- 172.21.15.47:0/2405220186 wait complete. 2024-04-20T02:26:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.663+0000 7f63753c0700 1 librados: shutdown 2024-04-20T02:26:49.712 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:49.712 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:26:49.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.850+0000 7f81b5b07700 10 monclient: get_monmap_and_config 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.850+0000 7f81b5b07700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.850+0000 7f81b5b07700 10 monclient: monmap: 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:49.851853+0000 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:49.851853+0000 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.851 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:49.852 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.851+0000 7f81b5b07700 1 Processor -- start 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.851+0000 7f81b5b07700 1 -- start start 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.851+0000 7f81b5b07700 10 monclient: init 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.851+0000 7f81b5b07700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.851+0000 7f81b5b07700 10 monclient: _add_conns ranks=[8,10,4,5,0,9,2,6,7,1,3] 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.851+0000 7f81b5b07700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 0x7f81b0135e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): picked mon.noname-e con 0x7f81b009f9e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f81b009fdc0 0x7f81b00a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): picked mon.noname-f con 0x7f81b009fdc0 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b00a1f10 0x7f81b00a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): picked mon.noname-c con 0x7f81b00a1f10 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b0136350 con 0x7f81b009fdc0 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00a0740 con 0x7f81b00a1f10 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f81b009e520 con 0x7f81b009f9e0 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81b5b07700 10 monclient(hunting): authenticate will time out at 2289.253073s 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 0x7f81b0135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b00a1f10 0x7f81b00a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 0x7f81b0135e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:53046/0 (socket says 172.21.15.47:53046) 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 -- 172.21.15.47:0/3214932454 learned_addr learned my addr 172.21.15.47:0/3214932454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): get_auth_request con 0x7f81b009f9e0 auth_method 0 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): get_auth_request con 0x7f81b00a1f10 auth_method 0 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81aeffd700 1 --2- 172.21.15.47:0/3214932454 >> v2:172.21.15.179:3300/0 conn(0x7f81b009fdc0 0x7f81b00a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81aeffd700 10 monclient(hunting): get_auth_request con 0x7f81b009fdc0 auth_method 0 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81aeffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81aeffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient(hunting): handle_auth_done global_id 35513 payload 274 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 -- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b00a1f10 msgr2=0x7f81b00a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b00a1f10 0x7f81b00a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 -- 172.21.15.47:0/3214932454 >> v2:172.21.15.179:3300/0 conn(0x7f81b009fdc0 msgr2=0x7f81b00a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3214932454 >> v2:172.21.15.179:3300/0 conn(0x7f81b009fdc0 0x7f81b00a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 monclient: found mon.noname-e 2024-04-20T02:26:49.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 -- 172.21.15.47:0/3214932454 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00751e0 con 0x7f81b009f9e0 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81affff700 1 --2- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b00a1f10 0x7f81b00a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81aeffd700 1 --2- 172.21.15.47:0/3214932454 >> v2:172.21.15.179:3300/0 conn(0x7f81b009fdc0 0x7f81b00a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 0x7f81b0135e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098f0 tx=0x7f81a0009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ae6c8863f52d5903 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81ae7fc700 1 -- 172.21.15.47:0/3214932454 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a0011aa0 con 0x7f81b009f9e0 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.852+0000 7f81ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: dump: 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.868 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 5 monclient: authenticate success, global_id 35513 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 1 -- 172.21.15.47:0/3214932454 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0011c00 con 0x7f81b009f9e0 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 4 set_mon_vals no callback set 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 1 -- 172.21.15.47:0/3214932454 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a0011d70 con 0x7f81b009f9e0 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81ae7fc700 10 monclient: dump: 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.870 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 10 monclient: shutdown 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 -- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 msgr2=0x7f81b0135e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 0x7f81b0135e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098f0 tx=0x7f81a0009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 -- 172.21.15.47:0/3214932454 shutdown_connections 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b00a1f10 0x7f81b00a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3214932454 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f81b009f9e0 0x7f81b0135e10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3214932454 >> v2:172.21.15.179:3300/0 conn(0x7f81b009fdc0 0x7f81b00a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 -- 172.21.15.47:0/3214932454 >> 172.21.15.47:0/3214932454 conn(0x7f81b00a3000 msgr2=0x7f81b00a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 -- 172.21.15.47:0/3214932454 shutdown_connections 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.853+0000 7f81b5b07700 1 -- 172.21.15.47:0/3214932454 wait complete. 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.854+0000 7f81b5b07700 10 monclient: build_initial_monmap 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.854+0000 7f81b5b07700 10 monclient: monmap: 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:49.872 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.854+0000 7f81b5b07700 1 librados: starting msgr at 2024-04-20T02:26:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.854+0000 7f81b5b07700 1 librados: starting objecter 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.854+0000 7f81b5b07700 1 Processor -- start 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 -- start start 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 librados: setting wanted keys 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 librados: calling monclient init 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient: init 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient: _add_conns ranks=[3,4,0,6,7,1,8,5,2] 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81b009f9e0 0x7f81b0132c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): picked mon.noname-g con 0x7f81b009f9e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b009fdc0 0x7f81b01331b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): picked mon.noname-c con 0x7f81b009fdc0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b00a1f10 0x7f81b012b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): picked mon.noname-a con 0x7f81b00a1f10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01336f0 con 0x7f81b00a1f10 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0072ee0 con 0x7f81b009f9e0 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01cb210 con 0x7f81b009fdc0 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81b5b07700 10 monclient(hunting): authenticate will time out at 2289.256571s 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b009fdc0 0x7f81b01331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b009fdc0 0x7f81b01331b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39146/0 (socket says 172.21.15.47:39146) 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b00a1f10 0x7f81b012b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 1 -- 172.21.15.47:0/3115357944 learned_addr learned my addr 172.21.15.47:0/3115357944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): get_auth_request con 0x7f81b009fdc0 auth_method 0 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): get_auth_request con 0x7f81b00a1f10 auth_method 0 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81affff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81b009f9e0 0x7f81b0132c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81af7fe700 10 monclient(hunting): get_auth_request con 0x7f81b009f9e0 auth_method 0 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81af7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81af7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.855+0000 7f81aeffd700 10 monclient(hunting): handle_auth_done global_id 74227 payload 824 2024-04-20T02:26:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81b009f9e0 msgr2=0x7f81b0132c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81b009f9e0 0x7f81b0132c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b00a1f10 msgr2=0x7f81b012b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81b00a1f10 0x7f81b012b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 10 monclient: _finish_hunting 0 2024-04-20T02:26:49.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 monclient: found mon.noname-c 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8009a70 con 0x7f81b009fdc0 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f81b009f9e0 0x7f81b0132c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81aeffd700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b009fdc0 0x7f81b01331b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e7e0 tx=0x7f81a400eba0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 client.?.objecter ms_handle_connect 0x7f81b009fdc0 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a4013880 con 0x7f81b009fdc0 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: dump: 2024-04-20T02:26:49.878 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.879 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: _finish_auth 0 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: _check_auth_tickets 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4013a00 con 0x7f81b009fdc0 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 5 monclient: authenticate success, global_id 74227 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 10 monclient: _renew_subs 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a0009550 con 0x7f81b009fdc0 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a4012370 con 0x7f81b009fdc0 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f8197fff700 10 monclient: dump: 2024-04-20T02:26:49.880 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.881 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81adffb700 4 set_mon_vals no callback set 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 10 client.74227.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 10 monclient: _renew_subs 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b012de70 con 0x7f81b009fdc0 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.856+0000 7f81b5b07700 1 librados: init done 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81b5b07700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 0x7f81b012bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0068210 con 0x7f81b006aac0 2024-04-20T02:26:49.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f81a4013410 con 0x7f81b009fdc0 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f819c03a320 0x7f819c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 0x7f81b012bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient: get_auth_request con 0x7f81b006aac0 auth_method 0 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(427..427 src has 266..427) v4 ==== 6248+0+0 (secure 0 0 0) 0x7f81a404a240 con 0x7f81b009fdc0 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 10 client.74227.objecter ms_dispatch 0x7f81b0001550 osd_map(427..427 src has 266..427) v4 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 3 client.74227.objecter handle_osd_map got epochs [427,427] > 0 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 3 client.74227.objecter handle_osd_map decoding full epoch 427 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f8197fff700 20 client.74227.objecter dump_active .. 0 homeless 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81affff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f819c03a320 0x7f819c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81affff700 10 monclient: get_auth_request con 0x7f819c03a320 auth_method 0 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 10 monclient(hunting): handle_auth_done global_id 55036 payload 824 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.857+0000 7f81af7fe700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81b00a0a00 con 0x7f81b006aac0 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 client.74227.objecter ms_handle_connect 0x7f81b006aac0 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 0x7f81b012bb00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81a0000c00 tx=0x7f81a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 client.74227.objecter resend_mon_ops 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 client.74227.objecter ms_handle_connect 0x7f819c03a320 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f81affff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f819c03a320 0x7f819c03c7e0 secure :-1 s=READY pgs=3260 cs=0 l=1 rev1=1 crypto rx=0x7f81b009cde0 tx=0x7f81a800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a0004430 con 0x7f81b006aac0 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 monclient: dump: 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.884 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f81a0004590 con 0x7f81b006aac0 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:49.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 msgr2=0x7f81b012bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 0x7f81b012bb00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81a0000c00 tx=0x7f81a0004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 msgr2=0x7f81b012bb00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.858+0000 7f8197fff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b006aac0 0x7f81b012bb00 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81a0000c00 tx=0x7f81a0004750 comp rx=0 tx=0).stop 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81b5b07700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 0x7f81b012c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81b5b07700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b006a400 con 0x7f81b01ca7a0 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 0x7f81b012c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient: get_auth_request con 0x7f81b01ca7a0 auth_method 0 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.866+0000 7f81af7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:49.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f81af7fe700 10 monclient(hunting): handle_auth_done global_id 55045 payload 824 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f81af7fe700 1 -- 172.21.15.47:0/3115357944 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f81b00a3d60 con 0x7f81b01ca7a0 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 client.74227.objecter ms_handle_connect 0x7f81b01ca7a0 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f81af7fe700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 0x7f81b012c040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81b0099fa0 tx=0x7f81a0011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 client.74227.objecter resend_mon_ops 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a002cdb0 con 0x7f81b01ca7a0 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 monclient: dump: 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:49.889 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f81a002c940 con 0x7f81b01ca7a0 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 msgr2=0x7f81b012c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 0x7f81b012c040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81b0099fa0 tx=0x7f81a0011000 comp rx=0 tx=0).stop 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 msgr2=0x7f81b012c040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.867+0000 7f8197fff700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b01ca7a0 0x7f81b012c040 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81b0099fa0 tx=0x7f81a0011000 comp rx=0 tx=0).stop 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:49.891 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.893 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:49.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:49.895 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:49.897 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.898 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:49.900 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.901 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:49.906 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:49.907 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:49.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.909 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:49.910 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 10 librados: watch_flush enter 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 10 librados: watch_flush exit 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 20 client.74227.objecter shutdown clearing up homeless session... 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 10 client.74227.objecter successfully canceled tick 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f819c03a320 msgr2=0x7f819c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f819c03a320 0x7f819c03c7e0 secure :-1 s=READY pgs=3260 cs=0 l=1 rev1=1 crypto rx=0x7f81b009cde0 tx=0x7f81a800b040 comp rx=0 tx=0).stop 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 10 monclient: shutdown 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b009fdc0 msgr2=0x7f81b01331b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 --2- 172.21.15.47:0/3115357944 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f81b009fdc0 0x7f81b01331b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e7e0 tx=0x7f81a400eba0 comp rx=0 tx=0).stop 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81affff700 1 -- 172.21.15.47:0/3115357944 reap_dead start 2024-04-20T02:26:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81affff700 1 -- 172.21.15.47:0/3115357944 reap_dead start 2024-04-20T02:26:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 shutdown_connections 2024-04-20T02:26:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 >> 172.21.15.47:0/3115357944 conn(0x7f81b00a3000 msgr2=0x7f81b009e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 shutdown_connections 2024-04-20T02:26:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 -- 172.21.15.47:0/3115357944 wait complete. 2024-04-20T02:26:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:49.868+0000 7f81b5b07700 1 librados: shutdown 2024-04-20T02:26:49.914 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:49.914 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:26:49.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:26:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.053+0000 7fdcc91dd700 10 monclient: get_monmap_and_config 2024-04-20T02:26:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.053+0000 7fdcc91dd700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.053+0000 7fdcc91dd700 10 monclient: monmap: 2024-04-20T02:26:50.053 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:50.054351+0000 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:50.054351+0000 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.054 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:50.055 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:50.055 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.053+0000 7fdcc91dd700 1 Processor -- start 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.053+0000 7fdcc91dd700 1 -- start start 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.053+0000 7fdcc91dd700 10 monclient: init 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient: _add_conns ranks=[2,7,3,10,4,9,5,6,8,0,1] 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 0x7fdcc4071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): picked mon.noname-b con 0x7fdcc4071630 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdcc4071f50 0x7fdcc40a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): picked mon.noname-i con 0x7fdcc4071f50 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc40a2e90 0x7fdcc40a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): picked mon.noname-g con 0x7fdcc40a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4072720 con 0x7fdcc4071630 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40a0820 con 0x7fdcc40a2e90 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc409e890 con 0x7fdcc4071f50 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc91dd700 10 monclient(hunting): authenticate will time out at 2289.455486s 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 0x7fdcc4071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 0x7fdcc4071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47510/0 (socket says 172.21.15.47:47510) 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/4143555555 learned_addr learned my addr 172.21.15.47:0/4143555555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): get_auth_request con 0x7fdcc4071630 auth_method 0 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc259c700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdcc4071f50 0x7fdcc40a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc359e700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc40a2e90 0x7fdcc40a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc259c700 10 monclient(hunting): get_auth_request con 0x7fdcc4071f50 auth_method 0 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc359e700 10 monclient(hunting): get_auth_request con 0x7fdcc40a2e90 auth_method 0 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc359e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc359e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc359e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_done global_id 24338 payload 274 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdcc4071f50 msgr2=0x7fdcc40a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdcc4071f50 0x7fdcc40a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.054+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc40a2e90 msgr2=0x7fdcc40a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc40a2e90 0x7fdcc40a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 1 monclient: found mon.noname-b 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/4143555555 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40751e0 con 0x7fdcc4071630 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc2d9d700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 0x7fdcc4071a10 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009920 tx=0x7fdcb4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e335d6603b01c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 1 -- 172.21.15.47:0/4143555555 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb4009e70 con 0x7fdcc4071630 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: dump: 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 5 monclient: authenticate success, global_id 24338 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 1 -- 172.21.15.47:0/4143555555 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb400bcd0 con 0x7fdcc4071630 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 4 set_mon_vals no callback set 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 1 -- 172.21.15.47:0/4143555555 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb400be40 con 0x7fdcc4071630 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc1d9b700 10 monclient: dump: 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 10 monclient: shutdown 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 -- 172.21.15.47:0/4143555555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 msgr2=0x7fdcc4071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 0x7fdcc4071a10 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009920 tx=0x7fdcb4009c30 comp rx=0 tx=0).stop 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 -- 172.21.15.47:0/4143555555 shutdown_connections 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc40a2e90 0x7fdcc40a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdcc4071630 0x7fdcc4071a10 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/4143555555 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdcc4071f50 0x7fdcc40a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.055+0000 7fdcc91dd700 1 -- 172.21.15.47:0/4143555555 >> 172.21.15.47:0/4143555555 conn(0x7fdcc40a7570 msgr2=0x7fdcc40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.056+0000 7fdcc91dd700 1 -- 172.21.15.47:0/4143555555 shutdown_connections 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.056+0000 7fdcc91dd700 1 -- 172.21.15.47:0/4143555555 wait complete. 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.056+0000 7fdcc91dd700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.056+0000 7fdcc91dd700 10 monclient: monmap: 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.073 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 1 librados: starting msgr at 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 1 librados: starting objecter 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 1 Processor -- start 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 1 -- start start 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 1 librados: setting wanted keys 2024-04-20T02:26:50.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 1 librados: calling monclient init 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.057+0000 7fdcc91dd700 10 monclient: init 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient: _add_conns ranks=[3,5,1,4,0,8,7,6,2] 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc4071630 0x7fdcc41bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): picked mon.noname-g con 0x7fdcc4071630 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdcc4071f50 0x7fdcc41bd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): picked mon.noname-h con 0x7fdcc4071f50 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcc40a2e90 0x7fdcc41a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): picked mon.noname-f con 0x7fdcc40a2e90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41a9ba0 con 0x7fdcc40a2e90 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41a80b0 con 0x7fdcc4071630 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4072380 con 0x7fdcc4071f50 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc91dd700 10 monclient(hunting): authenticate will time out at 2289.459143s 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc4071630 0x7fdcc41bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc4071630 0x7fdcc41bcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:50898/0 (socket says 172.21.15.47:50898) 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/710572891 learned_addr learned my addr 172.21.15.47:0/710572891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): get_auth_request con 0x7fdcc4071630 auth_method 0 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdcc4071f50 0x7fdcc41bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcc40a2e90 0x7fdcc41a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): get_auth_request con 0x7fdcc4071f50 auth_method 0 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): get_auth_request con 0x7fdcc40a2e90 auth_method 0 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc359e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient(hunting): handle_auth_done global_id 22530 payload 824 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdcc4071f50 msgr2=0x7fdcc41bd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fdcc4071f50 0x7fdcc41bd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcc40a2e90 msgr2=0x7fdcc41a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdcc40a2e90 0x7fdcc41a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 monclient: found mon.noname-g 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb4009550 con 0x7fdcc4071630 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcc2d9d700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc4071630 0x7fdcc41bcd20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b3c0 tx=0x7fdcb4004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcbb7fe700 10 client.?.objecter ms_handle_connect 0x7fdcc4071630 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.058+0000 7fdcbb7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb400bb60 con 0x7fdcc4071630 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:26:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: dump: 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.079 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb400be40 con 0x7fdcc4071630 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcb401b430 con 0x7fdcc4071630 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 5 monclient: authenticate success, global_id 22530 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc159a700 4 set_mon_vals no callback set 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcbb7fe700 10 monclient: dump: 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 10 monclient: _renew_subs 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc409f200 con 0x7fdcc4071630 2024-04-20T02:26:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 10 client.22530.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 10 monclient: _renew_subs 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc41a8980 con 0x7fdcc4071630 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.059+0000 7fdcc91dd700 1 librados: init done 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc91dd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 0x7fdcc41a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4075760 con 0x7fdcc406aac0 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fdcb4021020 con 0x7fdcc4071630 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 0x7fdcc41a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient: get_auth_request con 0x7fdcc406aac0 auth_method 0 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdca403a2f0 0x7fdca403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(427..427 src has 266..427) v4 ==== 6248+0+0 (secure 0 0 0) 0x7fdcb404b670 con 0x7fdcc4071630 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 10 client.22530.objecter ms_dispatch 0x7fdcc4001550 osd_map(427..427 src has 266..427) v4 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 3 client.22530.objecter handle_osd_map got epochs [427,427] > 0 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 3 client.22530.objecter handle_osd_map decoding full epoch 427 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 20 client.22530.objecter dump_active .. 0 homeless 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 10 monclient(hunting): handle_auth_done global_id 22539 payload 824 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdcc40a4480 con 0x7fdcc406aac0 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc359e700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdca403a2f0 0x7fdca403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc359e700 10 monclient: get_auth_request con 0x7fdca403a2f0 auth_method 0 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 10 client.22530.objecter ms_handle_connect 0x7fdcc406aac0 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 10 client.22530.objecter resend_mon_ops 2024-04-20T02:26:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcc259c700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 0x7fdcc41a6bf0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fdcc409d8d0 tx=0x7fdcac011b90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcac00c070 con 0x7fdcc406aac0 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.060+0000 7fdcbb7fe700 10 monclient: dump: 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 10 client.22530.objecter ms_handle_connect 0x7fdca403a2f0 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcc359e700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdca403a2f0 0x7fdca403c7b0 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 crypto rx=0x7fdcc41a8e10 tx=0x7fdcbc009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdcac0157d0 con 0x7fdcc406aac0 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 msgr2=0x7fdcc41a6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 0x7fdcc41a6bf0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fdcc409d8d0 tx=0x7fdcac011b90 comp rx=0 tx=0).stop 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 msgr2=0x7fdcc41a6bf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.061+0000 7fdcbb7fe700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc406aac0 0x7fdcc41a6bf0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc91dd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 0x7fdcc41a7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc91dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4068310 con 0x7fdcc41a7130 2024-04-20T02:26:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 0x7fdcc41a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient: get_auth_request con 0x7fdcc41a7130 auth_method 0 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 10 monclient(hunting): handle_auth_done global_id 22548 payload 824 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 1 -- 172.21.15.47:0/710572891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdcc406f090 con 0x7fdcc41a7130 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcc259c700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 0x7fdcc41a7560 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000c00 tx=0x7fdcac0047c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcbb7fe700 10 client.22530.objecter ms_handle_connect 0x7fdcc41a7130 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.069+0000 7fdcbb7fe700 10 client.22530.objecter resend_mon_ops 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdcac023040 con 0x7fdcc41a7130 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:26:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 10 monclient: dump: 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.089 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fdcac015e30 con 0x7fdcc41a7130 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 msgr2=0x7fdcc41a7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 0x7fdcc41a7560 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000c00 tx=0x7fdcac0047c0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 msgr2=0x7fdcc41a7560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcbb7fe700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc41a7130 0x7fdcc41a7560 secure :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fdcac000c00 tx=0x7fdcac0047c0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:50.090 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:50.091 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:50.092 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:50.093 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.094 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:50.095 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:50.096 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:50.099 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:50.102 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.105 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:50.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.108 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:26:50.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.110 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.110 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:50.110 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:50.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 10 librados: watch_flush enter 2024-04-20T02:26:50.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 10 librados: watch_flush exit 2024-04-20T02:26:50.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 20 client.22530.objecter shutdown clearing up homeless session... 2024-04-20T02:26:50.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 10 client.22530.objecter successfully canceled tick 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdca403a2f0 msgr2=0x7fdca403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdca403a2f0 0x7fdca403c7b0 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 crypto rx=0x7fdcc41a8e10 tx=0x7fdcbc009780 comp rx=0 tx=0).stop 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 10 monclient: shutdown 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc4071630 msgr2=0x7fdcc41bcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 --2- 172.21.15.47:0/710572891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdcc4071630 0x7fdcc41bcd20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b3c0 tx=0x7fdcb4004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc359e700 1 -- 172.21.15.47:0/710572891 reap_dead start 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc359e700 1 -- 172.21.15.47:0/710572891 reap_dead start 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 shutdown_connections 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 >> 172.21.15.47:0/710572891 conn(0x7fdcc40a7570 msgr2=0x7fdcc4066ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.070+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 shutdown_connections 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.071+0000 7fdcc91dd700 1 -- 172.21.15.47:0/710572891 wait complete. 2024-04-20T02:26:50.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.071+0000 7fdcc91dd700 1 librados: shutdown 2024-04-20T02:26:50.113 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:50.113 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:26:50.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:26:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.250+0000 7f7acd05d700 10 monclient: get_monmap_and_config 2024-04-20T02:26:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.250+0000 7f7acd05d700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.250+0000 7f7acd05d700 10 monclient: monmap: 2024-04-20T02:26:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:50.251483+0000 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:50.251483+0000 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:50.252 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.250+0000 7f7acd05d700 1 Processor -- start 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 -- start start 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient: init 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient: _add_conns ranks=[9,6,4,2,7,10,3,1,8,0,5] 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ac8071610 0x7f7ac80719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): picked mon.noname-a con 0x7f7ac8071610 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071f30 0x7f7ac80a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): picked mon.noname-d con 0x7f7ac8071f30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ac80a2e90 0x7f7ac80a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): picked mon.noname-c con 0x7f7ac80a2e90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80727b0 con 0x7f7ac8071610 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80a0880 con 0x7f7ac80a2e90 2024-04-20T02:26:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac809e8f0 con 0x7f7ac8071f30 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7acd05d700 10 monclient(hunting): authenticate will time out at 2289.652650s 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ac8071610 0x7f7ac80719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ac80a2e90 0x7f7ac80a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7ac8071610 0x7f7ac80719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34850/0 (socket says 172.21.15.47:34850) 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1596729518 learned_addr learned my addr 172.21.15.47:0/1596729518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): get_auth_request con 0x7f7ac8071610 auth_method 0 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071f30 0x7f7ac80a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): get_auth_request con 0x7f7ac80a2e90 auth_method 0 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): get_auth_request con 0x7f7ac8071f30 auth_method 0 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac759e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.251+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_done global_id 36099 payload 274 2024-04-20T02:26:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071f30 msgr2=0x7f7ac80a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071f30 0x7f7ac80a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ac80a2e90 msgr2=0x7f7ac80a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ac80a2e90 0x7f7ac80a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 monclient: found mon.noname-a 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1596729518 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80751e0 con 0x7f7ac8071610 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac759e700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ac80a2e90 0x7f7ac80a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac6d9d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071610 0x7f7ac80719f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80098f0 tx=0x7f7ab8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9deaa1c2b69bf8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 1 -- 172.21.15.47:0/1596729518 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab8011aa0 con 0x7f7ac8071610 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: dump: 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 1 -- 172.21.15.47:0/1596729518 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8011c00 con 0x7f7ac8071610 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7acd05d700 5 monclient: authenticate success, global_id 36099 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7acd05d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7acd05d700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 1 -- 172.21.15.47:0/1596729518 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab8011d70 con 0x7f7ac8071610 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7acd05d700 4 set_mon_vals no callback set 2024-04-20T02:26:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.252+0000 7f7ac5d9b700 10 monclient: dump: 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 10 monclient: shutdown 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 -- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071610 msgr2=0x7f7ac80719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071610 0x7f7ac80719f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80098f0 tx=0x7f7ab8009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 -- 172.21.15.47:0/1596729518 shutdown_connections 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ac80a2e90 0x7f7ac80a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071610 0x7f7ac80719f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1596729518 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071f30 0x7f7ac80a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 -- 172.21.15.47:0/1596729518 >> 172.21.15.47:0/1596729518 conn(0x7f7ac80a7570 msgr2=0x7f7ac80a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 -- 172.21.15.47:0/1596729518 shutdown_connections 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 1 -- 172.21.15.47:0/1596729518 wait complete. 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.253+0000 7f7acd05d700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 10 monclient: monmap: 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 1 librados: starting msgr at 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 1 librados: starting objecter 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 1 Processor -- start 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 1 -- start start 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 1 librados: setting wanted keys 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 1 librados: calling monclient init 2024-04-20T02:26:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.254+0000 7f7acd05d700 10 monclient: init 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient: _add_conns ranks=[6,0,8,5,3,7,4,2,1] 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071610 0x7f7ac81c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): picked mon.noname-d con 0x7f7ac8071610 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071f30 0x7f7ac81bb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): picked mon.noname-a con 0x7f7ac8071f30 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7ac80a2e90 0x7f7ac81bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): picked mon.noname-e con 0x7f7ac80a2e90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac81a8ff0 con 0x7f7ac8071f30 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac81a9240 con 0x7f7ac8071610 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac809f260 con 0x7f7ac80a2e90 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7acd05d700 10 monclient(hunting): authenticate will time out at 2289.656119s 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071610 0x7f7ac81c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071610 0x7f7ac81c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:39060/0 (socket says 172.21.15.47:39060) 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1374162205 learned_addr learned my addr 172.21.15.47:0/1374162205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): get_auth_request con 0x7f7ac8071610 auth_method 0 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7ac80a2e90 0x7f7ac81bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071f30 0x7f7ac81bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): get_auth_request con 0x7f7ac80a2e90 auth_method 0 2024-04-20T02:26:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): get_auth_request con 0x7f7ac8071f30 auth_method 0 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac759e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient(hunting): handle_auth_done global_id 54708 payload 824 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7ac80a2e90 msgr2=0x7f7ac81bbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7ac80a2e90 0x7f7ac81bbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071f30 msgr2=0x7f7ac81bb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071f30 0x7f7ac81bb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 monclient: found mon.noname-d 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac0009a70 con 0x7f7ac8071610 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac659c700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ac8071f30 0x7f7ac81bb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7ac6d9d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071610 0x7f7ac81c7e80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009f50 tx=0x7f7ab8004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7aaf7fe700 10 client.?.objecter ms_handle_connect 0x7f7ac8071610 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7aaf7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.255+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab80043d0 con 0x7f7ac8071610 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: dump: 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8004570 con 0x7f7ac8071610 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 5 monclient: authenticate success, global_id 54708 2024-04-20T02:26:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 monclient: _renew_subs 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab8009550 con 0x7f7ac8071610 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ab8024470 con 0x7f7ac8071610 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 client.54708.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7ac559a700 4 set_mon_vals no callback set 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 10 monclient: dump: 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 monclient: _renew_subs 2024-04-20T02:26:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac80723f0 con 0x7f7ac8071610 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 1 librados: init done 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 0x7f7ac81a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac81a93d0 con 0x7f7ac806aac0 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.256+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7ab8021070 con 0x7f7ac8071610 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7ab403a2f0 0x7f7ab403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(428..428 src has 266..428) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f7ab804bb90 con 0x7f7ac8071610 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 10 client.54708.objecter ms_dispatch 0x7f7ac8001550 osd_map(428..428 src has 266..428) v4 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 3 client.54708.objecter handle_osd_map got epochs [428,428] > 0 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 3 client.54708.objecter handle_osd_map decoding full epoch 428 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 0x7f7ac81a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient: get_auth_request con 0x7f7ac806aac0 auth_method 0 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 20 client.54708.objecter dump_active .. 0 homeless 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac759e700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7ab403a2f0 0x7f7ab403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac759e700 10 monclient: get_auth_request con 0x7f7ab403a2f0 auth_method 0 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 10 monclient(hunting): handle_auth_done global_id 64454 payload 824 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ac809ffe0 con 0x7f7ac806aac0 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 10 client.54708.objecter ms_handle_connect 0x7f7ac806aac0 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 10 client.54708.objecter resend_mon_ops 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac659c700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 0x7f7ac81a52e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ac809fd00 tx=0x7f7abc00f520 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7aaf7fe700 10 client.54708.objecter ms_handle_connect 0x7f7ab403a2f0 2024-04-20T02:26:50.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.257+0000 7f7ac759e700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7ab403a2f0 0x7f7ab403c7b0 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 crypto rx=0x7f7ac81a7600 tx=0x7f7ac000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7abc0044a0 con 0x7f7ac806aac0 2024-04-20T02:26:50.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 10 monclient: dump: 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.285 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7abc00fcc0 con 0x7f7ac806aac0 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 msgr2=0x7f7ac81a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 0x7f7ac81a52e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ac809fd00 tx=0x7f7abc00f520 comp rx=0 tx=0).stop 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 msgr2=0x7f7ac81a52e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.258+0000 7f7aaf7fe700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac806aac0 0x7f7ac81a52e0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ac809fd00 tx=0x7f7abc00f520 comp rx=0 tx=0).stop 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7acd05d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 0x7f7ac81a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7acd05d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8068310 con 0x7f7ac81a5820 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 0x7f7ac81a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient: get_auth_request con 0x7f7ac81a5820 auth_method 0 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.266+0000 7f7ac659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7ac659c700 10 monclient(hunting): handle_auth_done global_id 64463 payload 824 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7ac659c700 1 -- 172.21.15.47:0/1374162205 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7ac80a02c0 con 0x7f7ac81a5820 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7ac659c700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 0x7f7ac81a5c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009590 tx=0x7f7abc004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 client.54708.objecter ms_handle_connect 0x7f7ac81a5820 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 client.54708.objecter resend_mon_ops 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7abc003e30 con 0x7f7ac81a5820 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 monclient: dump: 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.290 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f7abc0273a0 con 0x7f7ac81a5820 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 msgr2=0x7f7ac81a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 0x7f7ac81a5c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009590 tx=0x7f7abc004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 msgr2=0x7f7ac81a5c30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.267+0000 7f7aaf7fe700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7ac81a5820 0x7f7ac81a5c30 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009590 tx=0x7f7abc004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:50.292 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:50.293 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:50.294 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:50.295 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:50.296 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:50.299 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:50.300 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.303 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.305 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.308 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:50.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:50.313 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 10 librados: watch_flush enter 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 10 librados: watch_flush exit 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 20 client.54708.objecter shutdown clearing up homeless session... 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 10 client.54708.objecter successfully canceled tick 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7ab403a2f0 msgr2=0x7f7ab403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7ab403a2f0 0x7f7ab403c7b0 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 crypto rx=0x7f7ac81a7600 tx=0x7f7ac000b040 comp rx=0 tx=0).stop 2024-04-20T02:26:50.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 10 monclient: shutdown 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071610 msgr2=0x7f7ac81c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 --2- 172.21.15.47:0/1374162205 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ac8071610 0x7f7ac81c7e80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009f50 tx=0x7f7ab8004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7ac759e700 1 -- 172.21.15.47:0/1374162205 reap_dead start 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7ac759e700 1 -- 172.21.15.47:0/1374162205 reap_dead start 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 shutdown_connections 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 >> 172.21.15.47:0/1374162205 conn(0x7f7ac80a7570 msgr2=0x7f7ac809c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 shutdown_connections 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 -- 172.21.15.47:0/1374162205 wait complete. 2024-04-20T02:26:50.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.268+0000 7f7acd05d700 1 librados: shutdown 2024-04-20T02:26:50.317 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:50.317 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:26:50.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:26:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.456+0000 7f8a2a429700 10 monclient: get_monmap_and_config 2024-04-20T02:26:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.457+0000 7f8a2a429700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.457+0000 7f8a2a429700 10 monclient: monmap: 2024-04-20T02:26:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.457 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:50.458177+0000 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:50.458177+0000 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.458 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:50.459 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:50.459 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.457+0000 7f8a2a429700 1 Processor -- start 2024-04-20T02:26:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.457+0000 7f8a2a429700 1 -- start start 2024-04-20T02:26:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.457+0000 7f8a2a429700 10 monclient: init 2024-04-20T02:26:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient: _add_conns ranks=[4,9,1,8,6,3,0,10,5,2,7] 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a24071610 0x7f8a240719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): picked mon.noname-c con 0x7f8a24071610 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a24071f30 0x7f8a240a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): picked mon.noname-a con 0x7f8a24071f30 addr v2:172.21.15.47:3300/0 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8a240a2e90 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a240751e0 con 0x7f8a240a2e90 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a240727b0 con 0x7f8a24071f30 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240a0880 con 0x7f8a24071610 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a2a429700 10 monclient(hunting): authenticate will time out at 2289.859560s 2024-04-20T02:26:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a24071f30 0x7f8a240a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a24071610 0x7f8a240719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a24071f30 0x7f8a240a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34860/0 (socket says 172.21.15.47:34860) 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): get_auth_request con 0x7f8a24071610 auth_method 0 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 1 -- 172.21.15.47:0/1709785290 learned_addr learned my addr 172.21.15.47:0/1709785290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): get_auth_request con 0x7f8a24071f30 auth_method 0 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a23fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 10 monclient(hunting): handle_auth_done global_id 36117 payload 274 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.458+0000 7f8a237fe700 1 -- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a24071610 msgr2=0x7f8a240719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 1 --2- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a24071610 0x7f8a240719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 1 -- 172.21.15.47:0/1709785290 >> v1:172.21.15.179:6789/0 conn(0x7f8a240a2e90 legacy=0x7f8a2413b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 1 monclient: found mon.noname-a 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 1 -- 172.21.15.47:0/1709785290 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2409e8f0 con 0x7f8a24071f30 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a23fff700 1 --2- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a24071610 0x7f8a240719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a237fe700 1 --2- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a24071f30 0x7f8a240a2950 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0098f0 tx=0x7f8a0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636753791b96ba2f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 1 -- 172.21.15.47:0/1709785290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0c011aa0 con 0x7f8a24071f30 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: dump: 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.473 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a2a429700 5 monclient: authenticate success, global_id 36117 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 1 -- 172.21.15.47:0/1709785290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c011c00 con 0x7f8a24071f30 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a2a429700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 1 -- 172.21.15.47:0/1709785290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0c011d70 con 0x7f8a24071f30 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a2a429700 4 set_mon_vals no callback set 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:50.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.459+0000 7f8a22ffd700 10 monclient: dump: 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.475 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 10 monclient: shutdown 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 -- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a24071f30 msgr2=0x7f8a240a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 --2- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a24071f30 0x7f8a240a2950 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0098f0 tx=0x7f8a0c009c00 comp rx=0 tx=0).stop 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 -- 172.21.15.47:0/1709785290 shutdown_connections 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 --2- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8a24071610 0x7f8a240719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 --2- 172.21.15.47:0/1709785290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a24071f30 0x7f8a240a2950 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 -- 172.21.15.47:0/1709785290 >> 172.21.15.47:0/1709785290 conn(0x7f8a240a7570 msgr2=0x7f8a240a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 -- 172.21.15.47:0/1709785290 shutdown_connections 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 1 -- 172.21.15.47:0/1709785290 wait complete. 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.460+0000 7f8a2a429700 10 monclient: monmap: 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 librados: starting msgr at 2024-04-20T02:26:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 librados: starting objecter 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 Processor -- start 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 -- start start 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 librados: setting wanted keys 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 librados: calling monclient init 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 10 monclient: init 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 10 monclient: _add_conns ranks=[8,6,3,1,0,4,2,7,5] 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 0x7f8a241cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 10 monclient(hunting): picked mon.noname-e con 0x7f8a24071610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a24071f30 0x7f8a241337d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.461+0000 7f8a2a429700 10 monclient(hunting): picked mon.noname-d con 0x7f8a24071f30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a240a2e90 0x7f8a2413c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient(hunting): picked mon.noname-g con 0x7f8a240a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2413fea0 con 0x7f8a240a2e90 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a241cd120 con 0x7f8a24071f30 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240723f0 con 0x7f8a24071610 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient(hunting): authenticate will time out at 2289.863052s 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 0x7f8a241cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a24071f30 0x7f8a241337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 0x7f8a241cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:53062/0 (socket says 172.21.15.47:53062) 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 -- 172.21.15.47:0/909604664 learned_addr learned my addr 172.21.15.47:0/909604664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): get_auth_request con 0x7f8a24071610 auth_method 0 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): get_auth_request con 0x7f8a24071f30 auth_method 0 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a289c6700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a240a2e90 0x7f8a2413c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient(hunting): handle_auth_done global_id 35531 payload 824 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a24071f30 msgr2=0x7f8a241337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a24071f30 0x7f8a241337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a240a2e90 msgr2=0x7f8a2413c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a240a2e90 0x7f8a2413c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 monclient: found mon.noname-e 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140094a0 con 0x7f8a24071610 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a289c6700 10 monclient: get_auth_request con 0x7f8a240a2e90 auth_method 0 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a289c6700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8a240a2e90 0x7f8a2413c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a237fe700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8a24071f30 0x7f8a241337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a23fff700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 0x7f8a241cc1f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400ec10 tx=0x7f8a1400ef20 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 client.?.objecter ms_handle_connect 0x7f8a24071610 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:50.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a1400cd80 con 0x7f8a24071610 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: dump: 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.482 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 5 monclient: authenticate success, global_id 35531 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14013470 con 0x7f8a24071610 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a1400cee0 con 0x7f8a24071610 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient: _renew_subs 2024-04-20T02:26:50.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c009550 con 0x7f8a24071610 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a227fc700 4 set_mon_vals no callback set 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.462+0000 7f8a20ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a20ff9700 10 monclient: dump: 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.484 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 10 client.35531.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 10 monclient: _renew_subs 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2409f2c0 con 0x7f8a24071610 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 1 librados: init done 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 0x7f8a2413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.463+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24068310 con 0x7f8a2406aac0 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8a1400c910 con 0x7f8a24071610 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 0x7f8a2413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient: get_auth_request con 0x7f8a2406aac0 auth_method 0 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8a0403a2f0 0x7f8a0403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(428..428 src has 266..428) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f8a1404a9d0 con 0x7f8a24071610 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 10 client.35531.objecter ms_dispatch 0x7f8a24001550 osd_map(428..428 src has 266..428) v4 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 3 client.35531.objecter handle_osd_map got epochs [428,428] > 0 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 3 client.35531.objecter handle_osd_map decoding full epoch 428 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a20ff9700 20 client.35531.objecter dump_active .. 0 homeless 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a289c6700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8a0403a2f0 0x7f8a0403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a289c6700 10 monclient: get_auth_request con 0x7f8a0403a2f0 auth_method 0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 10 monclient(hunting): handle_auth_done global_id 54655 payload 824 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.464+0000 7f8a237fe700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a240a6810 con 0x7f8a2406aac0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 client.35531.objecter ms_handle_connect 0x7f8a2406aac0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 client.35531.objecter resend_mon_ops 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a237fe700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 0x7f8a2413c5b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a2409fd00 tx=0x7f8a0c004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0c020070 con 0x7f8a2406aac0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a289c6700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8a0403a2f0 0x7f8a0403c7b0 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0012e0 tx=0x7f8a1c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 monclient: dump: 2024-04-20T02:26:50.487 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.488 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 client.35531.objecter ms_handle_connect 0x7f8a0403a2f0 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8a0c0038b0 con 0x7f8a2406aac0 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 msgr2=0x7f8a2413c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 0x7f8a2413c5b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a2409fd00 tx=0x7f8a0c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 msgr2=0x7f8a2413c5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.465+0000 7f8a20ff9700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2406aac0 0x7f8a2413c5b0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a2409fd00 tx=0x7f8a0c004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a2a429700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a2a429700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 0x7f8a2413caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a2a429700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2406a400 con 0x7f8a2413f630 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a237fe700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 0x7f8a2413caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a237fe700 10 monclient: get_auth_request con 0x7f8a2413f630 auth_method 0 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a237fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a237fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a237fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.473+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a237fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a237fe700 10 monclient(hunting): handle_auth_done global_id 54664 payload 824 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a237fe700 1 -- 172.21.15.47:0/909604664 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8a2409ffe0 con 0x7f8a2413f630 2024-04-20T02:26:50.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a237fe700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 0x7f8a2413caf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009130 tx=0x7f8a0c011000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 client.35531.objecter ms_handle_connect 0x7f8a2413f630 2024-04-20T02:26:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 client.35531.objecter resend_mon_ops 2024-04-20T02:26:50.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8a0c01bab0 con 0x7f8a2413f630 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 monclient: dump: 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.495 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9523 (secure 0 0 0) 0x7f8a0c01bc10 con 0x7f8a2413f630 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 msgr2=0x7f8a2413caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 0x7f8a2413caf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009130 tx=0x7f8a0c011000 comp rx=0 tx=0).stop 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 msgr2=0x7f8a2413caf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.474+0000 7f8a20ff9700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8a2413f630 0x7f8a2413caf0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009130 tx=0x7f8a0c011000 comp rx=0 tx=0).stop 2024-04-20T02:26:50.496 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 74, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:26:50.497 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.498 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:26:50.499 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:26:50.500 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:26:50.501 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:26:50.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:26:50.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.505 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:26:50.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:26:50.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:26:50.509 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:26:50.510 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:26:50.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:26:50.513 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:26:50.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:26:50.517 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.518 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:26:50.519 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:26:50.520 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:26:50.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:26:50.520 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:26:50.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:26:50.520 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:26:50.520 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 10 librados: watch_flush enter 2024-04-20T02:26:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 10 librados: watch_flush exit 2024-04-20T02:26:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 20 client.35531.objecter shutdown clearing up homeless session... 2024-04-20T02:26:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 10 client.35531.objecter successfully canceled tick 2024-04-20T02:26:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8a0403a2f0 msgr2=0x7f8a0403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8a0403a2f0 0x7f8a0403c7b0 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0012e0 tx=0x7f8a1c009040 comp rx=0 tx=0).stop 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 10 monclient: shutdown 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 msgr2=0x7f8a241cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 0x7f8a241cc1f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400ec10 tx=0x7f8a1400ef20 comp rx=0 tx=0).stop 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a289c6700 1 -- 172.21.15.47:0/909604664 reap_dead start 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 shutdown_connections 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 --2- 172.21.15.47:0/909604664 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f8a24071610 0x7f8a241cc1f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 >> 172.21.15.47:0/909604664 conn(0x7f8a240a7570 msgr2=0x7f8a240a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 shutdown_connections 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 -- 172.21.15.47:0/909604664 wait complete. 2024-04-20T02:26:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.475+0000 7f8a2a429700 1 librados: shutdown 2024-04-20T02:26:50.524 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:26:50.524 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'i', 'g', 'h'] 2024-04-20T02:26:50.524 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-20T02:26:50.524 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-20T02:26:50.524 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:26:50.524 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T02:26:50.524 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:26:50.535 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T02:26:50.535 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-04-20T02:26:50.535 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:26:50.535 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-04-20T02:26:50.535 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:26:50.552 INFO:tasks.ceph.mon.i:Stopped 2024-04-20T02:26:50.552 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-04-20T02:26:50.552 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-04-20T02:26:50.552 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-04-20T02:26:50.552 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:26:50.608 INFO:tasks.ceph.mon.g:Stopped 2024-04-20T02:26:50.608 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-04-20T02:26:50.608 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h store 2024-04-20T02:26:50.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h sync_force --yes-i-really-mean-it 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.752+0000 7fa1b9d08700 10 monclient: get_monmap_and_config 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.752+0000 7fa1b9d08700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.753+0000 7fa1b9d08700 10 monclient: monmap: 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:50.754116+0000 2024-04-20T02:26:50.753 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:50.754116+0000 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.754 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:50.755 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:50.755 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.753+0000 7fa1b9d08700 1 Processor -- start 2024-04-20T02:26:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.753+0000 7fa1b9d08700 1 -- start start 2024-04-20T02:26:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.753+0000 7fa1b9d08700 10 monclient: init 2024-04-20T02:26:50.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient: _add_conns ranks=[8,4,0,7,1,10,2,9,3,6,5] 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 0x7fa1b40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): picked mon.noname-e con 0x7fa1b40a51d0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1b40a3f10 0x7fa1b40a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): picked mon.noname-c con 0x7fa1b40a3f10 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa1b4137980 addr v1:172.21.15.47:6789/0 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b40751e0 con 0x7fa1b4137980 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4071cc0 con 0x7fa1b40a3f10 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40a55b0 con 0x7fa1b40a51d0 2024-04-20T02:26:50.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b2ffd700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1b40a3f10 msgr2=0x7fa1b40a4370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b9d08700 10 monclient(hunting): authenticate will time out at 2290.155418s 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b2ffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1b40a3f10 0x7fa1b40a4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 0x7fa1b40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b3fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1b4137980 0x7fa1b4137e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47542/0 (socket says 172.21.15.47:47542) 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 0x7fa1b40a39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:53072/0 (socket says 172.21.15.47:53072) 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b3fff700 1 -- 172.21.15.47:0/3750049186 learned_addr learned my addr 172.21.15.47:0/3750049186 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): get_auth_request con 0x7fa1b40a51d0 auth_method 0 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_done global_id 35540 payload 274 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b27fc700 1 -- 172.21.15.47:0/3750049186 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850028140 0 0) 0x7fa1b40751e0 con 0x7fa1b4137980 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 -- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1b40a3f10 msgr2=0x7fa1b40a4370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 --2- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1b40a3f10 0x7fa1b40a4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 -- 172.21.15.47:0/3750049186 >> v1:172.21.15.47:6789/0 conn(0x7fa1b4137980 legacy=0x7fa1b4137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 monclient: found mon.noname-e 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b37fe700 1 -- 172.21.15.47:0/3750049186 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b406c930 con 0x7fa1b40a51d0 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.754+0000 7fa1b27fc700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:26:50.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b37fe700 1 --2- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 0x7fa1b40a39d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009870 tx=0x7fa1a4009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=35616e6d49acae35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 1 -- 172.21.15.47:0/3750049186 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1a4011aa0 con 0x7fa1b40a51d0 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: dump: 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.772 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 1 -- 172.21.15.47:0/3750049186 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a4011c00 con 0x7fa1b40a51d0 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 5 monclient: authenticate success, global_id 35540 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:50.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 4 set_mon_vals no callback set 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 1 -- 172.21.15.47:0/3750049186 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1a4011d70 con 0x7fa1b40a51d0 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b27fc700 10 monclient: dump: 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.774 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 10 monclient: shutdown 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 -- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 msgr2=0x7fa1b40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 0x7fa1b40a39d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009870 tx=0x7fa1a4009b80 comp rx=0 tx=0).stop 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 -- 172.21.15.47:0/3750049186 shutdown_connections 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa1b40a51d0 0x7fa1b40a39d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/3750049186 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa1b40a3f10 0x7fa1b40a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 -- 172.21.15.47:0/3750049186 >> 172.21.15.47:0/3750049186 conn(0x7fa1b40a85f0 msgr2=0x7fa1b40a4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 -- 172.21.15.47:0/3750049186 shutdown_connections 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.755+0000 7fa1b9d08700 1 -- 172.21.15.47:0/3750049186 wait complete. 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.756+0000 7fa1b9d08700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.756+0000 7fa1b9d08700 10 monclient: monmap: 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.776 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.756+0000 7fa1b9d08700 1 librados: starting msgr at 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.756+0000 7fa1b9d08700 1 librados: starting objecter 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.756+0000 7fa1b9d08700 1 Processor -- start 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 -- start start 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 librados: setting wanted keys 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 librados: calling monclient init 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient: init 2024-04-20T02:26:50.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient: _add_conns ranks=[7,6,0,5,2,1,4,3,8] 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1b40a3f10 0x7fa1b41cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): picked mon.noname-i con 0x7fa1b40a3f10 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1b40a51d0 0x7fa1b4136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): picked mon.noname-d con 0x7fa1b40a51d0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b4137980 0x7fa1b41370b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): picked mon.noname-a con 0x7fa1b4137980 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b41375f0 con 0x7fa1b4137980 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4071e80 con 0x7fa1b40a51d0 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4125a80 con 0x7fa1b40a3f10 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b9d08700 10 monclient(hunting): authenticate will time out at 2290.158686s 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b37fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1b40a3f10 msgr2=0x7fa1b41cc370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b37fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1b40a3f10 0x7fa1b41cc370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b4137980 0x7fa1b41370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1b40a51d0 0x7fa1b4136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1b40a51d0 0x7fa1b4136b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:39068/0 (socket says 172.21.15.47:39068) 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 1 -- 172.21.15.47:0/2508123731 learned_addr learned my addr 172.21.15.47:0/2508123731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b4137980 0x7fa1b41370b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34870/0 (socket says 172.21.15.47:34870) 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): get_auth_request con 0x7fa1b40a51d0 auth_method 0 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): get_auth_request con 0x7fa1b4137980 auth_method 0 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.757+0000 7fa1b2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 10 monclient(hunting): handle_auth_done global_id 36135 payload 824 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1b40a3f10 msgr2=0x7fa1b41cc370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:26:50.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa1b40a3f10 0x7fa1b41cc370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1b40a51d0 msgr2=0x7fa1b4136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1b40a51d0 0x7fa1b4136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 monclient: found mon.noname-a 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40094a0 con 0x7fa1b4137980 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b2ffd700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa1b40a51d0 0x7fa1b4136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b3fff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b4137980 0x7fa1b41370b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00caa0 tx=0x7fa1ac00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 client.?.objecter ms_handle_connect 0x7fa1b4137980 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1ac00abf0 con 0x7fa1b4137980 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:50.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: dump: 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.782 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 5 monclient: authenticate success, global_id 36135 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac00ad70 con 0x7fa1b4137980 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 10 monclient: _renew_subs 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b4125d60 con 0x7fa1b4137980 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1ac007540 con 0x7fa1b4137980 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b1ffb700 4 set_mon_vals no callback set 2024-04-20T02:26:50.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 10 client.36135.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa19bfff700 10 monclient: dump: 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.784 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 10 monclient: _renew_subs 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b41240d0 con 0x7fa1b4137980 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.758+0000 7fa1b9d08700 1 librados: init done 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b9d08700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa1ac0070d0 con 0x7fa1b4137980 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 0x7fa1b41228f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4068470 con 0x7fa1b4122510 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa1a003a320 0x7fa1a003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(428..428 src has 266..428) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fa1ac04a010 con 0x7fa1b4137980 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 10 client.36135.objecter ms_dispatch 0x7fa1b4001550 osd_map(428..428 src has 266..428) v4 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 3 client.36135.objecter handle_osd_map got epochs [428,428] > 0 2024-04-20T02:26:50.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 3 client.36135.objecter handle_osd_map decoding full epoch 428 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 0x7fa1b41228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa19bfff700 20 client.36135.objecter dump_active .. 0 homeless 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient: get_auth_request con 0x7fa1b4122510 auth_method 0 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b2ffd700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa1a003a320 0x7fa1a003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b2ffd700 10 monclient: get_auth_request con 0x7fa1a003a320 auth_method 0 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.759+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_done global_id 64472 payload 824 2024-04-20T02:26:50.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa1b37fe700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1b406e300 con 0x7fa1b4122510 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 client.36135.objecter ms_handle_connect 0x7fa1a003a320 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 client.36135.objecter ms_handle_connect 0x7fa1b4122510 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 client.36135.objecter resend_mon_ops 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa1b2ffd700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa1a003a320 0x7fa1a003c7e0 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4123bf0 tx=0x7fa1a8009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa1b37fe700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 0x7fa1b41228f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009870 tx=0x7fa1a4004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1a401bda0 con 0x7fa1b4122510 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 monclient: dump: 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.787 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa1a401b930 con 0x7fa1b4122510 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 msgr2=0x7fa1b41228f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 0x7fa1b41228f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009870 tx=0x7fa1a4004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 msgr2=0x7fa1b41228f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.760+0000 7fa19bfff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122510 0x7fa1b41228f0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009870 tx=0x7fa1a4004750 comp rx=0 tx=0).stop 2024-04-20T02:26:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b9d08700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:26:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 0x7fa1b411e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b9d08700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40a3c30 con 0x7fa1b4122e30 2024-04-20T02:26:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 0x7fa1b411e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient: get_auth_request con 0x7fa1b4122e30 auth_method 0 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 10 monclient(hunting): handle_auth_done global_id 64481 payload 824 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 1 -- 172.21.15.47:0/2508123731 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fa1b406e5e0 con 0x7fa1b4122e30 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa19bfff700 10 client.36135.objecter ms_handle_connect 0x7fa1b4122e30 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa19bfff700 10 client.36135.objecter resend_mon_ops 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.770+0000 7fa1b37fe700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 0x7fa1b411e610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009fd0 tx=0x7fa1a4009210 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1a402caf0 con 0x7fa1b4122e30 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:26:50.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 10 monclient: dump: 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.793 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fa1a402cc50 con 0x7fa1b4122e30 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 msgr2=0x7fa1b411e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 0x7fa1b411e610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009fd0 tx=0x7fa1a4009210 comp rx=0 tx=0).stop 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 msgr2=0x7fa1b411e610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa19bfff700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa1b4122e30 0x7fa1b411e610 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009fd0 tx=0x7fa1a4009210 comp rx=0 tx=0).stop 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:26:50.794 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:26:50.795 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:26:50.795 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:26:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa1b9d08700 10 librados: watch_flush enter 2024-04-20T02:26:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa1b9d08700 10 librados: watch_flush exit 2024-04-20T02:26:50.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa1b9d08700 20 client.36135.objecter shutdown clearing up homeless session... 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.771+0000 7fa1b9d08700 10 client.36135.objecter successfully canceled tick 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa1a003a320 msgr2=0x7fa1a003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa1a003a320 0x7fa1a003c7e0 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4123bf0 tx=0x7fa1a8009210 comp rx=0 tx=0).stop 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 10 monclient: shutdown 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b4137980 msgr2=0x7fa1b41370b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 --2- 172.21.15.47:0/2508123731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1b4137980 0x7fa1b41370b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00caa0 tx=0x7fa1ac00cdb0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b3fff700 1 -- 172.21.15.47:0/2508123731 reap_dead start 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b3fff700 1 -- 172.21.15.47:0/2508123731 reap_dead start 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 shutdown_connections 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 >> 172.21.15.47:0/2508123731 conn(0x7fa1b40a85f0 msgr2=0x7fa1b41bd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 shutdown_connections 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 -- 172.21.15.47:0/2508123731 wait complete. 2024-04-20T02:26:50.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.772+0000 7fa1b9d08700 1 librados: shutdown 2024-04-20T02:26:50.798 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:26:50.798 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-04-20T02:26:50.798 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:26:50.812 INFO:tasks.ceph.mon.h:Stopped 2024-04-20T02:26:50.812 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-04-20T02:26:50.812 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.947+0000 7fdd5fcbc700 10 monclient: get_monmap_and_config 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.947+0000 7fdd5fcbc700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.947+0000 7fdd5fcbc700 10 monclient: monmap: 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:50.948718+0000 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:50.948718+0000 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.948 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 1 Processor -- start 2024-04-20T02:26:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 1 -- start start 2024-04-20T02:26:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 10 monclient: init 2024-04-20T02:26:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 10 monclient: _add_conns ranks=[2,7,8,3,10,0,9,4,6,1,5] 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 0x7fdd5806f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 10 monclient(hunting): picked mon.noname-b con 0x7fdd580716f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdd58071ad0 0x7fdd580a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.948+0000 7fdd5fcbc700 10 monclient(hunting): picked mon.noname-i con 0x7fdd58071ad0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdd58071f30 0x7fdd580a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient(hunting): picked mon.noname-e con 0x7fdd58071f30 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5809e910 con 0x7fdd580716f0 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5809ed20 con 0x7fdd58071ad0 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580a0000 con 0x7fdd58071f30 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient(hunting): authenticate will time out at 2290.350070s 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 0x7fdd5806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5d257700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdd58071ad0 msgr2=0x7fdd580a2950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 0x7fdd5806f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:47520/0 (socket says 172.21.15.47:47520) 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5d257700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdd58071ad0 0x7fdd580a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 1 -- 172.21.15.47:0/2996593912 learned_addr learned my addr 172.21.15.47:0/2996593912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): get_auth_request con 0x7fdd580716f0 auth_method 0 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdd58071f30 0x7fdd580a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): get_auth_request con 0x7fdd58071f30 auth_method 0 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient(hunting): handle_auth_done global_id 35549 payload 274 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 -- 172.21.15.47:0/2996593912 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdd58071ad0 msgr2=0x7fdd580a2950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdd58071ad0 0x7fdd580a2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 -- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 msgr2=0x7fdd5806f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 0x7fdd5806f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 monclient: found mon.noname-e 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 -- 172.21.15.47:0/2996593912 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580751e0 con 0x7fdd58071f30 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5da58700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 0x7fdd5806f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5e259700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdd58071f30 0x7fdd580a2e90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009dc0 tx=0x7fdd5000b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=51fe01aaae48157c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 1 -- 172.21.15.47:0/2996593912 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd5000e070 con 0x7fdd58071f30 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 10 monclient: dump: 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.965 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 5 monclient: authenticate success, global_id 35549 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 1 -- 172.21.15.47:0/2996593912 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50012430 con 0x7fdd58071f30 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5fcbc700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 4 set_mon_vals no callback set 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.949+0000 7fdd5ca56700 1 -- 172.21.15.47:0/2996593912 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd50012a10 con 0x7fdd58071f30 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5ca56700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5ca56700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5ca56700 10 monclient: dump: 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.967 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 10 monclient: shutdown 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdd58071f30 msgr2=0x7fdd580a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdd58071f30 0x7fdd580a2e90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009dc0 tx=0x7fdd5000b330 comp rx=0 tx=0).stop 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/2996593912 shutdown_connections 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdd58071f30 0x7fdd580a2e90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdd580716f0 0x7fdd5806f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/2996593912 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fdd58071ad0 0x7fdd580a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/2996593912 >> 172.21.15.47:0/2996593912 conn(0x7fdd580a7570 msgr2=0x7fdd580a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/2996593912 shutdown_connections 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.950+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/2996593912 wait complete. 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 10 monclient: build_initial_monmap 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 10 monclient: monmap: 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:50.969 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 1 librados: starting msgr at 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 1 librados: starting objecter 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 1 Processor -- start 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 1 -- start start 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 1 librados: setting wanted keys 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 1 librados: calling monclient init 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.951+0000 7fdd5fcbc700 10 monclient: init 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient: _add_conns ranks=[3,0,1,4,8,7,6,2,5] 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd580716f0 0x7fdd5813aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): picked mon.noname-g con 0x7fdd580716f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 0x7fdd58136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): picked mon.noname-a con 0x7fdd58071ad0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd58071f30 0x7fdd58137040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): picked mon.noname-f con 0x7fdd58071f30 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58137580 con 0x7fdd58071ad0 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580724f0 con 0x7fdd58071f30 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd581202a0 con 0x7fdd580716f0 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5fcbc700 10 monclient(hunting): authenticate will time out at 2290.353234s 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 0x7fdd58136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5da58700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd580716f0 msgr2=0x7fdd5813aad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 0x7fdd58136b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34878/0 (socket says 172.21.15.47:34878) 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5da58700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd580716f0 0x7fdd5813aad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 -- 172.21.15.47:0/113336678 learned_addr learned my addr 172.21.15.47:0/113336678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): get_auth_request con 0x7fdd58071ad0 auth_method 0 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5e259700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd58071f30 0x7fdd58137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5e259700 10 monclient(hunting): get_auth_request con 0x7fdd58071f30 auth_method 0 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5e259700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5e259700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:50.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5e259700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient(hunting): handle_auth_done global_id 36144 payload 824 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 -- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd580716f0 msgr2=0x7fdd5813aad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd580716f0 0x7fdd5813aad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 -- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd58071f30 msgr2=0x7fdd58137040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd58071f30 0x7fdd58137040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient: _finish_hunting 0 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 monclient: found mon.noname-a 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 -- 172.21.15.47:0/113336678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd54009550 con 0x7fdd58071ad0 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5e259700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd58071f30 0x7fdd58137040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd5d257700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 0x7fdd58136b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b570 tx=0x7fdd4800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd4e7fc700 10 client.?.objecter ms_handle_connect 0x7fdd58071ad0 2024-04-20T02:26:50.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.952+0000 7fdd4e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd48003c40 con 0x7fdd58071ad0 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: dump: 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.974 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48003de0 con 0x7fdd58071ad0 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 5 monclient: authenticate success, global_id 36144 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 10 monclient: _renew_subs 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd48011400 con 0x7fdd58071ad0 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd50009a70 con 0x7fdd58071ad0 2024-04-20T02:26:50.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4ffff700 4 set_mon_vals no callback set 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 10 monclient: dump: 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:50.976 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 10 client.36144.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 10 monclient: _renew_subs 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5813b730 con 0x7fdd58071ad0 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd5fcbc700 1 librados: init done 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.953+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fdd48005070 con 0x7fdd58071ad0 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd4e7fc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdd3c03a2f0 0x7fdd3c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd5fcbc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd5fcbc700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd5fcbc700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:50.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(428..428 src has 266..428) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fdd4804a0b0 con 0x7fdd58071ad0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd4e7fc700 10 client.36144.objecter ms_dispatch 0x7fdd58001550 osd_map(428..428 src has 266..428) v4 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd4e7fc700 3 client.36144.objecter handle_osd_map got epochs [428,428] > 0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd4e7fc700 3 client.36144.objecter handle_osd_map decoding full epoch 428 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd580008d0 con 0x7fdd58071ad0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd4e7fc700 20 client.36144.objecter dump_active .. 0 homeless 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd5da58700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdd3c03a2f0 0x7fdd3c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.954+0000 7fdd5da58700 10 monclient: get_auth_request con 0x7fdd3c03a2f0 auth_method 0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.955+0000 7fdd4e7fc700 10 client.36144.objecter ms_handle_connect 0x7fdd3c03a2f0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.955+0000 7fdd5da58700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdd3c03a2f0 0x7fdd3c03c7b0 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 crypto rx=0x7fdd54006cb0 tx=0x7fdd540092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.959+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fdd48011560 con 0x7fdd58071ad0 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.959+0000 7fdd4e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:50.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:50.959+0000 7fdd4e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd5fcbc700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd5fcbc700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd5fcbc700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdd580a43c0 con 0x7fdd58071ad0 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd4e7fc700 1 -- 172.21.15.47:0/113336678 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fdd48018af0 con 0x7fdd58071ad0 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd4e7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.266+0000 7fdd4e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:51.267 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 10 librados: watch_flush enter 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 10 librados: watch_flush exit 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 20 client.36144.objecter shutdown clearing up homeless session... 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 10 client.36144.objecter successfully canceled tick 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdd3c03a2f0 msgr2=0x7fdd3c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdd3c03a2f0 0x7fdd3c03c7b0 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 crypto rx=0x7fdd54006cb0 tx=0x7fdd540092c0 comp rx=0 tx=0).stop 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 10 monclient: shutdown 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 msgr2=0x7fdd58136b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 0x7fdd58136b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800b570 tx=0x7fdd4800b930 comp rx=0 tx=0).stop 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 shutdown_connections 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdd3c03a2f0 0x7fdd3c03c7b0 unknown :-1 s=CLOSED pgs=3265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:51.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd58071f30 0x7fdd58137040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd580716f0 0x7fdd5813aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 --2- 172.21.15.47:0/113336678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd58071ad0 0x7fdd58136b00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 >> 172.21.15.47:0/113336678 conn(0x7fdd580a7570 msgr2=0x7fdd580705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 shutdown_connections 2024-04-20T02:26:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 -- 172.21.15.47:0/113336678 wait complete. 2024-04-20T02:26:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:51.269+0000 7fdd5fcbc700 1 librados: shutdown 2024-04-20T02:26:54.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:26:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.427+0000 7f5c73b01700 10 monclient: get_monmap_and_config 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.427+0000 7f5c73b01700 10 monclient: build_initial_monmap 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.427+0000 7f5c73b01700 10 monclient: monmap: 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:54.428333+0000 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:54.428333+0000 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.427+0000 7f5c73b01700 1 Processor -- start 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 1 -- start start 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient: init 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient: _add_conns ranks=[1,0,10,7,9,2,6,3,5,4,8] 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5c6c0a0950 addr v1:172.21.15.179:6789/0 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5c6c0a0160 addr v1:172.21.15.47:6789/0 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5c6c13e460 0x7f5c6c140850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient(hunting): picked mon.noname-f con 0x7f5c6c13e460 addr v2:172.21.15.179:3300/0 2024-04-20T02:26:54.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c0751e0 con 0x7f5c6c0a0160 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c072830 con 0x7f5c6c0a0950 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0a1440 con 0x7f5c6c13e460 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c73b01700 10 monclient(hunting): authenticate will time out at 2293.829685s 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7109c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5c6c0a0160 0x7f5c6c13cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33286/0 (socket says 172.21.15.47:33286) 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7109c700 1 -- 172.21.15.47:0/2793972021 learned_addr learned my addr 172.21.15.47:0/2793972021 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7209e700 1 --2- 172.21.15.47:0/2793972021 >> v2:172.21.15.179:3300/0 conn(0x7f5c6c13e460 0x7f5c6c140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7209e700 10 monclient(hunting): get_auth_request con 0x7f5c6c13e460 auth_method 0 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7209e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7209e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.428+0000 7f5c7209e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:54.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 10 monclient(hunting): handle_auth_done global_id 55072 payload 274 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 1 -- 172.21.15.47:0/2793972021 >> v1:172.21.15.179:6789/0 conn(0x7f5c6c0a0950 legacy=0x7f5c6c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 1 -- 172.21.15.47:0/2793972021 >> v1:172.21.15.47:6789/0 conn(0x7f5c6c0a0160 legacy=0x7f5c6c13cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 10 monclient: _finish_hunting 0 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 1 monclient: found mon.noname-f 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 1 -- 172.21.15.47:0/2793972021 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c09e650 con 0x7f5c6c13e460 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7209e700 1 --2- 172.21.15.47:0/2793972021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c13e460 0x7f5c6c140850 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009dc0 tx=0x7f5c6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8deec6cd36c2bb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 1 -- 172.21.15.47:0/2793972021 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c6400e070 con 0x7f5c6c13e460 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: dump: 2024-04-20T02:26:54.441 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:54.442 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: _finish_auth 0 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: _check_auth_tickets 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 1 -- 172.21.15.47:0/2793972021 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640098f0 con 0x7f5c6c13e460 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c73b01700 5 monclient: authenticate success, global_id 55072 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c73b01700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 1 -- 172.21.15.47:0/2793972021 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c640128a0 con 0x7f5c6c13e460 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c73b01700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c73b01700 4 set_mon_vals no callback set 2024-04-20T02:26:54.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.429+0000 7f5c7089b700 10 monclient: dump: 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:54.444 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 10 monclient: shutdown 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 -- 172.21.15.47:0/2793972021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c13e460 msgr2=0x7f5c6c140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 --2- 172.21.15.47:0/2793972021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c13e460 0x7f5c6c140850 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009dc0 tx=0x7f5c6400b330 comp rx=0 tx=0).stop 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 -- 172.21.15.47:0/2793972021 shutdown_connections 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 --2- 172.21.15.47:0/2793972021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c13e460 0x7f5c6c140850 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 -- 172.21.15.47:0/2793972021 >> 172.21.15.47:0/2793972021 conn(0x7f5c6c0a7570 msgr2=0x7f5c6c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 -- 172.21.15.47:0/2793972021 shutdown_connections 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.430+0000 7f5c73b01700 1 -- 172.21.15.47:0/2793972021 wait complete. 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 10 monclient: build_initial_monmap 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 10 monclient: monmap: 2024-04-20T02:26:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:54.446 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 1 librados: starting msgr at 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 1 librados: starting objecter 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 1 Processor -- start 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 1 -- start start 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 1 librados: setting wanted keys 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 1 librados: calling monclient init 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 10 monclient: init 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.431+0000 7f5c73b01700 10 monclient: _add_conns ranks=[4,1,3,5,7,6,8,0,2] 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c6c0a0160 0x7f5c6c1cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): picked mon.noname-c con 0x7f5c6c0a0160 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 0x7f5c6c137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): picked mon.noname-f con 0x7f5c6c0a0950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:26:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c6c13e460 0x7f5c6c13bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): picked mon.noname-g con 0x7f5c6c13e460 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1ccd00 con 0x7f5c6c0a0950 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1231e0 con 0x7f5c6c13e460 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c09e4b0 con 0x7f5c6c0a0160 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7189d700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c6c0a0160 msgr2=0x7f5c6c1cc7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c73b01700 10 monclient(hunting): authenticate will time out at 2293.833087s 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7189d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c6c0a0160 0x7f5c6c1cc7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7209e700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c6c13e460 msgr2=0x7f5c6c13bd70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7209e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c6c13e460 0x7f5c6c13bd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 0x7f5c6c137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 0x7f5c6c137b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:57512/0 (socket says 172.21.15.47:57512) 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 -- 172.21.15.47:0/1332304233 learned_addr learned my addr 172.21.15.47:0/1332304233 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): get_auth_request con 0x7f5c6c0a0950 auth_method 0 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient(hunting): handle_auth_done global_id 55081 payload 824 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 -- 172.21.15.47:0/1332304233 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c6c0a0160 msgr2=0x7f5c6c1cc7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c6c0a0160 0x7f5c6c1cc7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 -- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c6c13e460 msgr2=0x7f5c6c13bd70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c6c13e460 0x7f5c6c13bd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient: _finish_hunting 0 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 monclient: found mon.noname-f 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 -- 172.21.15.47:0/1332304233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009a70 con 0x7f5c6c0a0950 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c7109c700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 0x7f5c6c137b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c4c0 tx=0x7f5c5c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c627fc700 10 client.?.objecter ms_handle_connect 0x7f5c6c0a0950 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.432+0000 7f5c627fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c5c0070d0 con 0x7f5c6c0a0950 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: dump: 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:54.450 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: _finish_auth 0 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: _check_auth_tickets 2024-04-20T02:26:54.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c007230 con 0x7f5c6c0a0950 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5c5c007810 con 0x7f5c6c0a0950 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c63fff700 4 set_mon_vals no callback set 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c627fc700 10 monclient: dump: 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:54.452 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 5 monclient: authenticate success, global_id 55081 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 10 monclient: _renew_subs 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c1cd6c0 con 0x7f5c6c0a0950 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 10 client.55081.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 10 monclient: _renew_subs 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:54.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c122f00 con 0x7f5c6c0a0950 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.433+0000 7f5c73b01700 1 librados: init done 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c73b01700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c73b01700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c73b01700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c0a6a70 con 0x7f5c6c0a0950 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5c5c0073a0 con 0x7f5c6c0a0950 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c5003a2f0 0x7f5c5003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(428..428 src has 266..428) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5c5c049850 con 0x7f5c6c0a0950 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 10 client.55081.objecter ms_dispatch 0x7f5c6c001550 osd_map(428..428 src has 266..428) v4 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 3 client.55081.objecter handle_osd_map got epochs [428,428] > 0 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 3 client.55081.objecter handle_osd_map decoding full epoch 428 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c627fc700 20 client.55081.objecter dump_active .. 0 homeless 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c7189d700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c5003a2f0 0x7f5c5003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:54.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.434+0000 7f5c7189d700 10 monclient: get_auth_request con 0x7f5c5003a2f0 auth_method 0 2024-04-20T02:26:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.435+0000 7f5c627fc700 10 client.55081.objecter ms_handle_connect 0x7f5c5003a2f0 2024-04-20T02:26:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.435+0000 7f5c7189d700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c5003a2f0 0x7f5c5003c7b0 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f5c68000c00 tx=0x7f5c68007aa0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.439+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f5c5c018680 con 0x7f5c6c0a0950 2024-04-20T02:26:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.439+0000 7f5c627fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:54.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.439+0000 7f5c627fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.746+0000 7f5c73b01700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:26:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.746+0000 7f5c73b01700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.746+0000 7f5c73b01700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:26:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.746+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5c6c06acc0 con 0x7f5c6c0a0950 2024-04-20T02:26:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.747+0000 7f5c627fc700 1 -- 172.21.15.47:0/1332304233 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f5c5c017c80 con 0x7f5c6c0a0950 2024-04-20T02:26:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.747+0000 7f5c627fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.747+0000 7f5c627fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:54.747 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":10,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 10 librados: watch_flush enter 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 10 librados: watch_flush exit 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 20 client.55081.objecter shutdown clearing up homeless session... 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 10 client.55081.objecter successfully canceled tick 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c5003a2f0 msgr2=0x7f5c5003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c5003a2f0 0x7f5c5003c7b0 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 crypto rx=0x7f5c68000c00 tx=0x7f5c68007aa0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 10 monclient: shutdown 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 msgr2=0x7f5c6c137b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 0x7f5c6c137b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c4c0 tx=0x7f5c5c00c880 comp rx=0 tx=0).stop 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 shutdown_connections 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5c6c13e460 0x7f5c6c13bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5c6c0a0160 0x7f5c6c1cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5c5003a2f0 0x7f5c5003c7b0 unknown :-1 s=CLOSED pgs=3266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 --2- 172.21.15.47:0/1332304233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c6c0a0950 0x7f5c6c137b20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 >> 172.21.15.47:0/1332304233 conn(0x7f5c6c0a7570 msgr2=0x7f5c6c09c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 shutdown_connections 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 -- 172.21.15.47:0/1332304233 wait complete. 2024-04-20T02:26:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:54.749+0000 7f5c73b01700 1 librados: shutdown 2024-04-20T02:26:57.764 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.906+0000 7f7300643700 10 monclient: get_monmap_and_config 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.906+0000 7f7300643700 10 monclient: build_initial_monmap 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.906+0000 7f7300643700 10 monclient: monmap: 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:26:57.907747+0000 2024-04-20T02:26:57.907 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:26:57.907747+0000 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:57.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 1 Processor -- start 2024-04-20T02:26:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 1 -- start start 2024-04-20T02:26:57.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 10 monclient: init 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 10 monclient: _add_conns ranks=[0,6,4,2,5,8,10,9,1,3,7] 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f72f8071660 addr v1:172.21.15.47:6789/0 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f72f8137de0 0x7f72f813cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 10 monclient(hunting): picked mon.noname-d con 0x7f72f8137de0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.907+0000 7f7300643700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f813d150 0x7f72f813f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 10 monclient(hunting): picked mon.noname-c con 0x7f72f813d150 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80751e0 con 0x7f72f8071660 2024-04-20T02:26:57.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f806ca10 con 0x7f72f813d150 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f72f806e140 con 0x7f72f8137de0 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 10 monclient(hunting): authenticate will time out at 2297.309061s 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72febe0700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f813d150 msgr2=0x7f72f813f540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72febe0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f813d150 0x7f72f813f540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fe3df700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72f8071660 0x7f72f8071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33288/0 (socket says 172.21.15.47:33288) 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f72f8137de0 0x7f72f813cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fe3df700 1 -- 172.21.15.47:0/2276944967 learned_addr learned my addr 172.21.15.47:0/2276944967 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): get_auth_request con 0x7f72f8137de0 auth_method 0 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:57.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient(hunting): handle_auth_done global_id 54735 payload 274 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 -- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f813d150 msgr2=0x7f72f813f540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 --2- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f813d150 0x7f72f813f540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 -- 172.21.15.47:0/2276944967 >> v1:172.21.15.47:6789/0 conn(0x7f72f8071660 legacy=0x7f72f8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient: _finish_hunting 0 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 monclient: found mon.noname-d 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 -- 172.21.15.47:0/2276944967 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f806e420 con 0x7f72f8137de0 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fdbde700 1 --2- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f72f8137de0 0x7f72f813cc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009870 tx=0x7f72e8009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=dbc40fb35ab8d64d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 1 -- 172.21.15.47:0/2276944967 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72e8011aa0 con 0x7f72f8137de0 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:57.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 10 monclient: dump: 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:57.921 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 10 monclient: _finish_auth 0 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 1 -- 172.21.15.47:0/2276944967 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e8011c00 con 0x7f72f8137de0 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 5 monclient: authenticate success, global_id 54735 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f7300643700 10 monclient: get_monmap_and_config success 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.908+0000 7f72fd3dd700 1 -- 172.21.15.47:0/2276944967 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72e8011d70 con 0x7f72f8137de0 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 4 set_mon_vals no callback set 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f72fd3dd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f72fd3dd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:26:57.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f72fd3dd700 10 monclient: dump: 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:57.923 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 10 monclient: shutdown 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 -- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f72f8137de0 msgr2=0x7f72f813cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 --2- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f72f8137de0 0x7f72f813cc10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009870 tx=0x7f72e8009b80 comp rx=0 tx=0).stop 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 -- 172.21.15.47:0/2276944967 shutdown_connections 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 --2- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f72f813d150 0x7f72f813f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 --2- 172.21.15.47:0/2276944967 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f72f8137de0 0x7f72f813cc10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 -- 172.21.15.47:0/2276944967 >> 172.21.15.47:0/2276944967 conn(0x7f72f80a84a0 msgr2=0x7f72f80a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 -- 172.21.15.47:0/2276944967 shutdown_connections 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.909+0000 7f7300643700 1 -- 172.21.15.47:0/2276944967 wait complete. 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 10 monclient: build_initial_monmap 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 10 monclient: monmap: 2024-04-20T02:26:57.924 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:26:57.925 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 1 librados: starting msgr at 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 1 librados: starting objecter 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 1 Processor -- start 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 1 -- start start 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 1 librados: setting wanted keys 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 1 librados: calling monclient init 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.910+0000 7f7300643700 10 monclient: init 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient: _reopen_session rank -1 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient: _add_conns ranks=[5,2,3,4,6,7,1,0,8] 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f8071660 0x7f72f81cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): picked mon.noname-h con 0x7f72f8071660 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 0x7f72f81cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): picked mon.noname-b con 0x7f72f8137de0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f813d150 0x7f72f81cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:57.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): picked mon.noname-g con 0x7f72f813d150 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f806d3c0 con 0x7f72f8137de0 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f813c050 con 0x7f72f813d150 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): start opening mon connection 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72f806d780 con 0x7f72f8071660 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): _renew_subs 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f7300643700 10 monclient(hunting): authenticate will time out at 2297.312279s 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 0x7f72f81cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72febe0700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f813d150 msgr2=0x7f72f81cce40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fe3df700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f8071660 msgr2=0x7f72f81cc3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72febe0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f813d150 0x7f72f81cce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fe3df700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f8071660 0x7f72f81cc3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 0x7f72f81cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:55440/0 (socket says 172.21.15.47:55440) 2024-04-20T02:26:57.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 -- 172.21.15.47:0/260182783 learned_addr learned my addr 172.21.15.47:0/260182783 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): get_auth_request con 0x7f72f8137de0 auth_method 0 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient(hunting): handle_auth_done global_id 24356 payload 824 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 -- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f8071660 msgr2=0x7f72f81cc3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f8071660 0x7f72f81cc3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 -- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f813d150 msgr2=0x7f72f81cce40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f813d150 0x7f72f81cce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient: _finish_hunting 0 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 monclient: found mon.noname-b 2024-04-20T02:26:57.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 -- 172.21.15.47:0/260182783 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e80094a0 con 0x7f72f8137de0 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72fdbde700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 0x7f72f81cc900 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f72e80090b0 tx=0x7f72e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72eeffd700 10 client.?.objecter ms_handle_connect 0x7f72f8137de0 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.911+0000 7f72eeffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72e8011a00 con 0x7f72f8137de0 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: dump: 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:57.929 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: _finish_auth 0 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: _check_auth_tickets 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e801f070 con 0x7f72f8137de0 2024-04-20T02:26:57.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 5 monclient: authenticate success, global_id 24356 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 10 monclient: _renew_subs 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f8120800 con 0x7f72f8137de0 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72e801a450 con 0x7f72f8137de0 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72eeffd700 10 monclient: dump: 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:26:57.931 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f72fcbdc700 4 set_mon_vals no callback set 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 10 client.24356.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 10 monclient: _renew_subs 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f811f910 con 0x7f72f8137de0 2024-04-20T02:26:57.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.912+0000 7f7300643700 1 librados: init done 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f7300643700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f7300643700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f72e8021030 con 0x7f72f8137de0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f7300643700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f811faa0 con 0x7f72f8137de0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(428..428 src has 266..428) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f72e804bbc0 con 0x7f72f8137de0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 10 client.24356.objecter ms_dispatch 0x7f72f8001550 osd_map(428..428 src has 266..428) v4 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 3 client.24356.objecter handle_osd_map got epochs [428,428] > 0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 3 client.24356.objecter handle_osd_map decoding full epoch 428 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72eeffd700 20 client.24356.objecter dump_active .. 0 homeless 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72fe3df700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.913+0000 7f72fe3df700 10 monclient: get_auth_request con 0x7f72dc03a2f0 auth_method 0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.914+0000 7f72eeffd700 10 client.24356.objecter ms_handle_connect 0x7f72dc03a2f0 2024-04-20T02:26:57.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.914+0000 7f72fe3df700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 crypto rx=0x7f72f40048a0 tx=0x7f72f4007b10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:26:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.918+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f72e80233d0 con 0x7f72f8137de0 2024-04-20T02:26:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.918+0000 7f72eeffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:26:57.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:57.918+0000 7f72eeffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:26:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f7300643700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:26:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f7300643700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:58.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f7300643700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:26:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f72f8120c00 con 0x7f72f8137de0 2024-04-20T02:26:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f72eeffd700 1 -- 172.21.15.47:0/260182783 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f72e8015020 con 0x7f72f8137de0 2024-04-20T02:26:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f72eeffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:26:58.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.231+0000 7f72eeffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:26:58.232 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":74,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:26:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 10 librados: watch_flush enter 2024-04-20T02:26:58.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 10 librados: watch_flush exit 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 20 client.24356.objecter shutdown clearing up homeless session... 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 10 client.24356.objecter successfully canceled tick 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 msgr2=0x7f72dc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 crypto rx=0x7f72f40048a0 tx=0x7f72f4007b10 comp rx=0 tx=0).stop 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 10 monclient: shutdown 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 msgr2=0x7f72f81cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.233+0000 7f7300643700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 0x7f72f81cc900 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f72e80090b0 tx=0x7f72e8004750 comp rx=0 tx=0).stop 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 shutdown_connections 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f72dc03a2f0 0x7f72dc03c7b0 unknown :-1 s=CLOSED pgs=3267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f72f813d150 0x7f72f81cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f72f8071660 0x7f72f81cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 --2- 172.21.15.47:0/260182783 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f72f8137de0 0x7f72f81cc900 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 >> 172.21.15.47:0/260182783 conn(0x7f72f80a84a0 msgr2=0x7f72f8135d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:26:58.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 shutdown_connections 2024-04-20T02:26:58.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 -- 172.21.15.47:0/260182783 wait complete. 2024-04-20T02:26:58.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:26:58.234+0000 7f7300643700 1 librados: shutdown 2024-04-20T02:27:01.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:27:01.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.392+0000 7fc19d9f6700 10 monclient: get_monmap_and_config 2024-04-20T02:27:01.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.392+0000 7fc19d9f6700 10 monclient: build_initial_monmap 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.392+0000 7fc19d9f6700 10 monclient: monmap: 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:27:01.393424+0000 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:27:01.393424+0000 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:01.393 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:01.394 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:01.394 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:01.394 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:27:01.394 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:27:01.394 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.392+0000 7fc19d9f6700 1 Processor -- start 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.392+0000 7fc19d9f6700 1 -- start start 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.392+0000 7fc19d9f6700 10 monclient: init 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient: _add_conns ranks=[7,0,6,2,9,3,1,4,5,10,8] 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 0x7fc1980a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): picked mon.noname-i con 0x7fc1980a4c80 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc1980a0d20 addr v1:172.21.15.47:6789/0 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:01.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): picked mon.noname-d con 0x7fc1980a5770 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980751e0 con 0x7fc1980a0d20 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980a0000 con 0x7fc1980a5770 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980a6a10 con 0x7fc1980a4c80 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc19d9f6700 10 monclient(hunting): authenticate will time out at 2300.794602s 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc196ffd700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 msgr2=0x7fc1980a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1967fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc1980a0d20 0x7fc1980a5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33302/0 (socket says 172.21.15.47:33302) 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1967fc700 1 -- 172.21.15.47:0/1694981681 learned_addr learned my addr 172.21.15.47:0/1694981681 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:27:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 0x7fc1980a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 10 monclient(hunting): get_auth_request con 0x7fc1980a5770 auth_method 0 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:27:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.393+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 msgr2=0x7fc1980a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 0x7fc1980a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 10 monclient(hunting): get_auth_request con 0x7fc1980a5770 auth_method 0 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:01.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:27:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:27:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:27:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:27:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.594+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 msgr2=0x7fc1980a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 0x7fc1980a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 10 monclient(hunting): get_auth_request con 0x7fc1980a5770 auth_method 0 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:27:01.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:27:01.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:27:01.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:27:01.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:01.995+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:02.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 msgr2=0x7fc1980a07e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:02.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 0x7fc1980a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:27:02.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:02.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 10 monclient(hunting): get_auth_request con 0x7fc1980a5770 auth_method 0 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:27:02.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:02.796+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): tick 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 monclient(hunting): continuing hunt 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 msgr2=0x7fc1980a07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1980a4c80 0x7fc1980a07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 msgr2=0x7fc19813ef10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:04.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1980a5770 0x7fc19813ef10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 -- 172.21.15.47:0/1694981681 >> v1:172.21.15.47:6789/0 conn(0x7fc1980a0d20 legacy=0x7fc1980a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient: _add_conns ranks=[5,4,1,9,8,3,2,10] 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 0x7fc178005d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): picked mon.noname-h con 0x7fc1780038d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 0x7fc178008720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): picked mon.noname-c con 0x7fc1780062b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc178008de0 addr v1:172.21.15.179:6789/0 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 -- 172.21.15.47:0/1694981681 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19809e440 con 0x7fc178008de0 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 -- 172.21.15.47:0/1694981681 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980a0000 con 0x7fc1780062b0 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 1 -- 172.21.15.47:0/1694981681 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980a6a10 con 0x7fc1780038d0 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc194ff9700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:04.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc1967fc700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 msgr2=0x7fc178008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 0x7fc178008720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 msgr2=0x7fc178005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:04.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.394+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 0x7fc178005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.595+0000 7fc1967fc700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 msgr2=0x7fc178008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.595+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 0x7fc178008720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.595+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 msgr2=0x7fc178005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:04.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.595+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 0x7fc178005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:04.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.996+0000 7fc1967fc700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 msgr2=0x7fc178008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:04.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.996+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 0x7fc178008720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:04.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.996+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 msgr2=0x7fc178005d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:04.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:04.996+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 0x7fc178005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.355+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579788759 0 0) 0x7fc19809e440 con 0x7fc178008de0 2024-04-20T02:27:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.355+0000 7fc195ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:27:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.355+0000 7fc195ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.355+0000 7fc195ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.355+0000 7fc195ffb700 10 monclient(hunting): my global_id is 55090 2024-04-20T02:27:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.355+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1840035b0 con 0x7fc178008de0 2024-04-20T02:27:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376451693 0 0) 0x7fc1840035b0 con 0x7fc178008de0 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 msgr2=0x7fc178005d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1780038d0 0x7fc178005d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 msgr2=0x7fc178008720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc1780062b0 0x7fc178008720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 10 monclient: _finish_hunting 0 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 reap_dead start 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17800dd20 con 0x7fc178008de0 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 10 monclient: _finish_auth 0 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.360+0000 7fc195ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:27:05.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7fc1900078e0 con 0x7fc178008de0 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc19d9f6700 5 monclient: authenticate success, global_id 55090 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc19d9f6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient: dump: 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 >> v1:172.21.15.179:6789/0 conn(0x7fc178008de0 legacy=0x7fc17800b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient: _add_conns ranks=[8,3,4,5,7,2,0,1,6] 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004250 0x7fc184004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): picked mon.e con 0x7fc184004250 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:27:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc184004f40 0x7fc184009a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): picked mon.g con 0x7fc184004f40 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc184009f40 0x7fc18400c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): picked mon.c con 0x7fc184009f40 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980a0000 con 0x7fc184004f40 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980a6a10 con 0x7fc184009f40 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc18400cba0 con 0x7fc184004250 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc195ffb700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc19d9f6700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc1977fe700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc184009f40 msgr2=0x7fc18400c330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc1967fc700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc184004f40 msgr2=0x7fc184009a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc184009f40 0x7fc18400c330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc184004f40 0x7fc184009a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004250 0x7fc184004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): get_auth_request con 0x7fc184004250 auth_method 0 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient(hunting): handle_auth_done global_id 55090 payload 293 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc184009f40 msgr2=0x7fc18400c330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc184009f40 0x7fc18400c330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc184004f40 msgr2=0x7fc184009a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc184004f40 0x7fc184009a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 monclient: found mon.e 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 1 -- 172.21.15.47:0/1694981681 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1840035b0 con 0x7fc184004250 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient: _finish_auth 0 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.361+0000 7fc196ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004250 0x7fc184004850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d60 tx=0x7fc184005880 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5018a115c18e2b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc188009770 con 0x7fc184004250 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: dump: 2024-04-20T02:27:05.376 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc19d9f6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188015400 con 0x7fc184004250 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc19d9f6700 10 monclient: get_monmap_and_config success 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc19d9f6700 4 set_mon_vals no callback set 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 1 -- 172.21.15.47:0/1694981681 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1880159e0 con 0x7fc184004250 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.362+0000 7fc195ffb700 10 monclient: dump: 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.378 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 10 monclient: shutdown 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004250 msgr2=0x7fc184004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004250 0x7fc184004850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d60 tx=0x7fc184005880 comp rx=0 tx=0).stop 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1694981681 shutdown_connections 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc184009f40 0x7fc18400c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc184004f40 0x7fc184009a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1694981681 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004250 0x7fc184004850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1694981681 >> 172.21.15.47:0/1694981681 conn(0x7fc1980aa280 msgr2=0x7fc1980aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1694981681 shutdown_connections 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.363+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1694981681 wait complete. 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 10 monclient: build_initial_monmap 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 10 monclient: monmap: 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:27:05.380 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 1 librados: starting msgr at 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 1 librados: starting objecter 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 1 Processor -- start 2024-04-20T02:27:05.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 1 -- start start 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 1 librados: setting wanted keys 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 1 librados: calling monclient init 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.364+0000 7fc19d9f6700 10 monclient: init 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient: _add_conns ranks=[0,6,8,5,3,1,7,2,4] 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 0x7fc1981c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): picked mon.noname-a con 0x7fc178008de0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 0x7fc1981bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): picked mon.noname-d con 0x7fc184004250 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 0x7fc1981cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): picked mon.noname-e con 0x7fc184004f40 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981aa740 con 0x7fc178008de0 2024-04-20T02:27:05.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc19809ea00 con 0x7fc184004250 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc19809dd20 con 0x7fc184004f40 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc19d9f6700 10 monclient(hunting): authenticate will time out at 2304.766463s 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 0x7fc1981bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 0x7fc1981c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 0x7fc1981cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 0x7fc1981bacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:47430/0 (socket says 172.21.15.47:47430) 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 0x7fc1981cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:51498/0 (socket says 172.21.15.47:51498) 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 0x7fc1981c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46320/0 (socket says 172.21.15.47:46320) 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 1 -- 172.21.15.47:0/1402418707 learned_addr learned my addr 172.21.15.47:0/1402418707 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): get_auth_request con 0x7fc184004250 auth_method 0 2024-04-20T02:27:05.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): get_auth_request con 0x7fc184004f40 auth_method 0 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): get_auth_request con 0x7fc178008de0 auth_method 0 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1977fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:05.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc1967fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.365+0000 7fc196ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 10 monclient(hunting): handle_auth_done global_id 54744 payload 824 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 -- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 msgr2=0x7fc1981cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 0x7fc1981cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 -- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 msgr2=0x7fc1981c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 0x7fc1981c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 10 monclient: _finish_hunting 0 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 monclient: found mon.noname-d 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:27:05.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 -- 172.21.15.47:0/1402418707 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1981c97d0 con 0x7fc184004250 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1977fe700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 0x7fc1981cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 0x7fc1981c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1967fc700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 0x7fc1981bacd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c4d0 tx=0x7fc18000c7e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 client.?.objecter ms_handle_connect 0x7fc184004250 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc18000cea0 con 0x7fc184004250 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: dump: 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.386 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: _finish_auth 0 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:27:05.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1800011e0 con 0x7fc184004250 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 5 monclient: authenticate success, global_id 54744 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 10 monclient: _renew_subs 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1981a9e10 con 0x7fc184004250 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc180007540 con 0x7fc184004250 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc18ffff700 10 monclient: dump: 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.388 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc1957fa700 4 set_mon_vals no callback set 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 10 client.54744.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 10 monclient: _renew_subs 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:27:05.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981a82e0 con 0x7fc184004250 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.366+0000 7fc19d9f6700 1 librados: init done 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc1800076a0 con 0x7fc184004250 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc19d9f6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc19d9f6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc19d9f6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804ebb0 con 0x7fc184004250 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc18ffff700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc17803b380 0x7fc17803d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(429..429 src has 266..429) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc180049700 con 0x7fc184004250 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc18ffff700 10 client.54744.objecter ms_dispatch 0x7fc198001550 osd_map(429..429 src has 266..429) v4 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc18ffff700 3 client.54744.objecter handle_osd_map got epochs [429,429] > 0 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.367+0000 7fc18ffff700 3 client.54744.objecter handle_osd_map decoding full epoch 429 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.368+0000 7fc18ffff700 20 client.54744.objecter dump_active .. 0 homeless 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.368+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc17803b380 0x7fc17803d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.368+0000 7fc196ffd700 10 monclient: get_auth_request con 0x7fc17803b380 auth_method 0 2024-04-20T02:27:05.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.368+0000 7fc18ffff700 10 client.54744.objecter ms_handle_connect 0x7fc17803b380 2024-04-20T02:27:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.368+0000 7fc196ffd700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc17803b380 0x7fc17803d840 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d60 tx=0x7fc188010000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.373+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc180018860 con 0x7fc184004250 2024-04-20T02:27:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.373+0000 7fc18ffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:05.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.373+0000 7fc18ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:27:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.681+0000 7fc19d9f6700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:27:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.681+0000 7fc19d9f6700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:27:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.681+0000 7fc19d9f6700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:27:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.681+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc1980a1940 con 0x7fc184004250 2024-04-20T02:27:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.682+0000 7fc18ffff700 1 -- 172.21.15.47:0/1402418707 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3151 (secure 0 0 0) 0x7fc180017080 con 0x7fc184004250 2024-04-20T02:27:05.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.682+0000 7fc18ffff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:27:05.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.682+0000 7fc18ffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:27:05.683 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":78,"quorum":[0,1,2,6,8],"quorum_names":["a","f","b","d","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:27:05.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 10 librados: watch_flush enter 2024-04-20T02:27:05.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 10 librados: watch_flush exit 2024-04-20T02:27:05.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 20 client.54744.objecter shutdown clearing up homeless session... 2024-04-20T02:27:05.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 10 client.54744.objecter successfully canceled tick 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc17803b380 msgr2=0x7fc17803d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc17803b380 0x7fc17803d840 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d60 tx=0x7fc188010000 comp rx=0 tx=0).stop 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 10 monclient: shutdown 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 msgr2=0x7fc1981bacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 0x7fc1981bacd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c4d0 tx=0x7fc18000c7e0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 shutdown_connections 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc17803b380 0x7fc17803d840 unknown :-1 s=CLOSED pgs=3268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc184004f40 0x7fc1981cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc178008de0 0x7fc1981c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 --2- 172.21.15.47:0/1402418707 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc184004250 0x7fc1981bacd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 >> 172.21.15.47:0/1402418707 conn(0x7fc1980aa280 msgr2=0x7fc1981bd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.684+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 shutdown_connections 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.685+0000 7fc19d9f6700 1 -- 172.21.15.47:0/1402418707 wait complete. 2024-04-20T02:27:05.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.685+0000 7fc19d9f6700 1 librados: shutdown 2024-04-20T02:27:05.698 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-04-20T02:27:05.698 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-04-20T02:27:05.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:27:05.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.834+0000 7f20a5479700 10 monclient: get_monmap_and_config 2024-04-20T02:27:05.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.834+0000 7f20a5479700 10 monclient: build_initial_monmap 2024-04-20T02:27:05.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.834+0000 7f20a5479700 10 monclient: monmap: 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:27:05.835252+0000 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:27:05.835252+0000 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:05.842 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.834+0000 7f20a5479700 1 Processor -- start 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.834+0000 7f20a5479700 1 -- start start 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.834+0000 7f20a5479700 10 monclient: init 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient: _add_conns ranks=[7,4,2,1,3,10,9,0,6,8,5] 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f20a009d420 0x7f20a0135b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): picked mon.noname-i con 0x7f20a009d420 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a0b90 0x7f20a009d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): picked mon.noname-c con 0x7f20a00a0b90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:05.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 0x7f20a00a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): picked mon.noname-b con 0x7f20a00a2800 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0067dd0 con 0x7f20a00a2800 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00a1010 con 0x7f20a00a0b90 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0067a60 con 0x7f20a009d420 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f20a5479700 10 monclient(hunting): authenticate will time out at 2305.236337s 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209e7fc700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a0b90 msgr2=0x7f20a009d800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a0b90 0x7f20a009d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209effd700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f20a009d420 msgr2=0x7f20a0135b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:05.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209effd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f20a009d420 0x7f20a0135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 0x7f20a00a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 0x7f20a00a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43726/0 (socket says 172.21.15.47:43726) 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 -- 172.21.15.47:0/3589424472 learned_addr learned my addr 172.21.15.47:0/3589424472 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): get_auth_request con 0x7f20a00a2800 auth_method 0 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient(hunting): handle_auth_done global_id 24365 payload 274 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 -- 172.21.15.47:0/3589424472 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f20a009d420 msgr2=0x7f20a0135b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f20a009d420 0x7f20a0135b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 -- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a0b90 msgr2=0x7f20a009d800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:05.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a0b90 0x7f20a009d800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 monclient: found mon.noname-b 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.835+0000 7f209f7fe700 1 -- 172.21.15.47:0/3589424472 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00751e0 con 0x7f20a00a2800 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209f7fe700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 0x7f20a00a2c80 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2098009e80 tx=0x7f209800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c124321d5a4290d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 1 -- 172.21.15.47:0/3589424472 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f209800e070 con 0x7f20a00a2800 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: dump: 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.846 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: _finish_auth 0 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:27:05.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 5 monclient: authenticate success, global_id 24365 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 1 -- 172.21.15.47:0/3589424472 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800be30 con 0x7f20a00a2800 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 10 monclient: get_monmap_and_config success 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 4 set_mon_vals no callback set 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 1 -- 172.21.15.47:0/3589424472 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2098012860 con 0x7f20a00a2800 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f209dffb700 10 monclient: dump: 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.848 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 10 monclient: shutdown 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 -- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 msgr2=0x7f20a00a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 0x7f20a00a2c80 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2098009e80 tx=0x7f209800b330 comp rx=0 tx=0).stop 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 -- 172.21.15.47:0/3589424472 shutdown_connections 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f20a00a2800 0x7f20a00a2c80 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f20a009d420 0x7f20a0135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 --2- 172.21.15.47:0/3589424472 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a0b90 0x7f20a009d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.836+0000 7f20a5479700 1 -- 172.21.15.47:0/3589424472 >> 172.21.15.47:0/3589424472 conn(0x7f20a00a3f80 msgr2=0x7f20a006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.837+0000 7f20a5479700 1 -- 172.21.15.47:0/3589424472 shutdown_connections 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.837+0000 7f20a5479700 1 -- 172.21.15.47:0/3589424472 wait complete. 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.837+0000 7f20a5479700 10 monclient: build_initial_monmap 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.837+0000 7f20a5479700 10 monclient: monmap: 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:27:05.850 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.837+0000 7f20a5479700 1 librados: starting msgr at 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.837+0000 7f20a5479700 1 librados: starting objecter 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 Processor -- start 2024-04-20T02:27:05.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 -- start start 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 librados: setting wanted keys 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 librados: calling monclient init 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient: init 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient: _add_conns ranks=[3,4,5,7,2,8,6,0,1] 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 0x7f20a01cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): picked mon.noname-g con 0x7f20a009d420 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 0x7f20a0133a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): picked mon.noname-c con 0x7f20a00a2800 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 0x7f20a0123560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): picked mon.noname-h con 0x7f20a0133f90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0126eb0 con 0x7f20a009d420 2024-04-20T02:27:05.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01255b0 con 0x7f20a00a2800 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00722f0 con 0x7f20a0133f90 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f20a5479700 10 monclient(hunting): authenticate will time out at 2305.239896s 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f209e7fc700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 msgr2=0x7f20a0133a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 0x7f20a0133a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f209effd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 msgr2=0x7f20a01cc7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.838+0000 7f209f7fe700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 msgr2=0x7f20a0123560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.839+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 0x7f20a0123560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:05.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:05.839+0000 7f209effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 0x7f20a01cc7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.039+0000 7f209e7fc700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 msgr2=0x7f20a0133a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:06.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.039+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 0x7f20a0133a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:06.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.040+0000 7f209effd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 msgr2=0x7f20a01cc7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:06.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.040+0000 7f209f7fe700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 msgr2=0x7f20a0123560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:06.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.040+0000 7f209effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 0x7f20a01cc7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:06.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.040+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 0x7f20a0123560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:27:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.440+0000 7f209e7fc700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 msgr2=0x7f20a0133a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:06.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.440+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 0x7f20a0133a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.441+0000 7f209f7fe700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 msgr2=0x7f20a0123560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.441+0000 7f209effd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 msgr2=0x7f20a01cc7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.441+0000 7f209effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 0x7f20a01cc7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:06.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:06.441+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 0x7f20a0123560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:27:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:07.241+0000 7f209e7fc700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 msgr2=0x7f20a0133a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:07.241+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 0x7f20a0133a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:27:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:07.242+0000 7f209f7fe700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 msgr2=0x7f20a0123560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:07.242+0000 7f209effd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 msgr2=0x7f20a01cc7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:07.242+0000 7f209effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 0x7f20a01cc7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:27:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:07.242+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 0x7f20a0123560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:27:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): tick 2024-04-20T02:27:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:27:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:27:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:27:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 msgr2=0x7f20a0123560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:08.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f20a0133f90 0x7f20a0123560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 msgr2=0x7f20a0133a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f20a00a2800 0x7f20a0133a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 msgr2=0x7f20a01cc7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20a009d420 0x7f20a01cc7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient: _add_conns ranks=[1,7,0,8,2,6] 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2080003870 0x7f2080005c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): picked mon.noname-f con 0x7f2080003870 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2080006260 0x7f20800086f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): picked mon.noname-i con 0x7f2080006260 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 0x7f208000b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): picked mon.noname-a con 0x7f2080008d90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:08.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0126eb0 con 0x7f2080008d90 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01255b0 con 0x7f2080003870 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00722f0 con 0x7f2080006260 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20867fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209e7fc700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2080006260 msgr2=0x7f20800086f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2080006260 0x7f20800086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 0x7f208000b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 0x7f208000b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46332/0 (socket says 172.21.15.47:46332) 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 -- 172.21.15.47:0/1369827717 learned_addr learned my addr 172.21.15.47:0/1369827717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): get_auth_request con 0x7f2080008d90 auth_method 0 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:08.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2080003870 0x7f2080005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): get_auth_request con 0x7f2080003870 auth_method 0 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:08.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient(hunting): handle_auth_done global_id 36162 payload 824 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2080006260 msgr2=0x7f20800086f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2080006260 0x7f20800086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2080003870 msgr2=0x7f2080005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2080003870 0x7f2080005c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 monclient: found mon.noname-a 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098009b30 con 0x7f2080008d90 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 -- 172.21.15.47:0/1369827717 reap_dead start 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f209f7fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 0x7f208000b250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2098009e20 tx=0x7f2098004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:08.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.839+0000 7f20877fe700 10 client.?.objecter ms_handle_connect 0x7f2080008d90 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2098004840 con 0x7f2080008d90 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: dump: 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:08.850 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: _finish_auth 0 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: _check_auth_tickets 2024-04-20T02:27:08.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 5 monclient: authenticate success, global_id 36162 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209801c410 con 0x7f2080008d90 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 10 monclient: _renew_subs 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a01250a0 con 0x7f2080008d90 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f209801c9f0 con 0x7f2080008d90 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 10 client.36162.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20877fe700 10 monclient: dump: 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:08.852 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:08.853 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f209d7fa700 4 set_mon_vals no callback set 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 10 monclient: _renew_subs 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a0071ef0 con 0x7f2080008d90 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.840+0000 7f20a5479700 1 librados: init done 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f209801c580 con 0x7f2080008d90 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20a5479700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 0x7f20a01cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:08.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0127850 con 0x7f20a00a0b90 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f208c03a260 0x7f208c03c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 0x7f20a01cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient: get_auth_request con 0x7f20a00a0b90 auth_method 0 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(431..431 src has 266..431) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f209804b490 con 0x7f2080008d90 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 10 client.36162.objecter ms_dispatch 0x7f20a0001550 osd_map(431..431 src has 266..431) v4 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 3 client.36162.objecter handle_osd_map got epochs [431,431] > 0 2024-04-20T02:27:08.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 3 client.36162.objecter handle_osd_map decoding full epoch 431 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f20877fe700 20 client.36162.objecter dump_active .. 0 homeless 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209e7fc700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f208c03a260 0x7f208c03c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.841+0000 7f209e7fc700 10 monclient: get_auth_request con 0x7f208c03a260 auth_method 0 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f209effd700 10 monclient(hunting): handle_auth_done global_id 36171 payload 824 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f209effd700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20a006d180 con 0x7f20a00a0b90 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 client.36162.objecter ms_handle_connect 0x7f20a00a0b90 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 client.36162.objecter resend_mon_ops 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f209effd700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 0x7f20a01cc7e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2094003a60 tx=0x7f2094007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2094010040 con 0x7f20a00a0b90 2024-04-20T02:27:08.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 monclient: dump: 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:08.857 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 client.36162.objecter ms_handle_connect 0x7f208c03a260 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f209e7fc700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f208c03a260 0x7f208c03c720 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7f20a0124cc0 tx=0x7f2090009250 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2094003d70 con 0x7f20a00a0b90 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 msgr2=0x7f20a01cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:08.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 0x7f20a01cc7e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2094003a60 tx=0x7f2094007570 comp rx=0 tx=0).stop 2024-04-20T02:27:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 msgr2=0x7f20a01cc7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:08.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.842+0000 7f20877fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 0x7f20a01cc7e0 secure :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2094003a60 tx=0x7f2094007570 comp rx=0 tx=0).stop 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f20a5479700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 0x7f20a0133a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f20a5479700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0068210 con 0x7f20a0134370 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 0x7f20a0133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient: get_auth_request con 0x7f20a0134370 auth_method 0 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:08.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 10 monclient(hunting): handle_auth_done global_id 36180 payload 824 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 1 -- 172.21.15.47:0/1369827717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f20a00a31c0 con 0x7f20a0134370 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f20877fe700 10 client.36162.objecter ms_handle_connect 0x7f20a0134370 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f20877fe700 10 client.36162.objecter resend_mon_ops 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.851+0000 7f209effd700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 0x7f20a0133a50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2094003a60 tx=0x7f2094005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2094005540 con 0x7f20a0134370 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:27:08.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 10 monclient: dump: 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:08.863 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f20940050d0 con 0x7f20a0134370 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 msgr2=0x7f20a0133a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 0x7f20a0133a50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2094003a60 tx=0x7f2094005bf0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 msgr2=0x7f20a0133a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20877fe700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 0x7f20a0133a50 secure :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2094003a60 tx=0x7f2094005bf0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.864 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 78, 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:27:08.865 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:08.866 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:08.867 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:08.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:08.869 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.870 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.871 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.872 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:27:08.873 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.875 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.876 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:27:08.877 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:27:08.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.879 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.880 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:27:08.883 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:27:08.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.885 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:08.886 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.887 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:08.888 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:08.889 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:27:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:08.890 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:27:08.890 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:27:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20a5479700 10 librados: watch_flush enter 2024-04-20T02:27:08.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20a5479700 10 librados: watch_flush exit 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20a5479700 20 client.36162.objecter shutdown clearing up homeless session... 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.852+0000 7f20a5479700 10 client.36162.objecter successfully canceled tick 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f208c03a260 msgr2=0x7f208c03c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f208c03a260 0x7f208c03c720 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 crypto rx=0x7f20a0124cc0 tx=0x7f2090009250 comp rx=0 tx=0).stop 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 10 monclient: shutdown 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 msgr2=0x7f208000b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 0x7f208000b250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2098009e20 tx=0x7f2098004ad0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 shutdown_connections 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f208c03a260 0x7f208c03c720 unknown :-1 s=CLOSED pgs=3269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2080008d90 0x7f208000b250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00a0b90 0x7f20a01cc7e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 --2- 172.21.15.47:0/1369827717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0134370 0x7f20a0133a50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:08.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 >> 172.21.15.47:0/1369827717 conn(0x7f20a00a3f80 msgr2=0x7f20a00a6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:08.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 shutdown_connections 2024-04-20T02:27:08.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 -- 172.21.15.47:0/1369827717 wait complete. 2024-04-20T02:27:08.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:08.853+0000 7f20a5479700 1 librados: shutdown 2024-04-20T02:27:08.894 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:27:08.894 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon b 2024-04-20T02:27:08.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:27:09.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.039+0000 7f76dbc72700 10 monclient: get_monmap_and_config 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.039+0000 7f76dbc72700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.039+0000 7f76dbc72700 10 monclient: monmap: 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:27:09.040628+0000 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:27:09.040628+0000 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.040 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:27:09.041 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 Processor -- start 2024-04-20T02:27:09.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 -- start start 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient: init 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient: _add_conns ranks=[1,7,5,3,0,4,2,8,9,10,6] 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f76d40a2950 addr v1:172.21.15.179:6789/0 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76d413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): picked mon.noname-i con 0x7f76d40a3440 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76d413f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): picked mon.noname-h con 0x7f76d413d490 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d40751e0 con 0x7f76d40a2950 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4072830 con 0x7f76d413d490 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76d409f150 con 0x7f76d40a3440 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.040+0000 7f76dbc72700 10 monclient(hunting): authenticate will time out at 2308.441917s 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d920d700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 msgr2=0x7f76d413cf50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76da20f700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 msgr2=0x7f76d413f880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d920d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76d413cf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76da20f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76d413f880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d9a0e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f76d40a2950 0x7f76d40a2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:48306/0 (socket says 172.21.15.47:48306) 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d9a0e700 1 -- 172.21.15.47:0/3828701235 learned_addr learned my addr 172.21.15.47:0/3828701235 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135772223 0 0) 0x7f76d40751e0 con 0x7f76d40a2950 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): _negotiate 2024-04-20T02:27:09.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): my global_id is 55108 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76bc0035b0 con 0x7f76d40a2950 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513530671 0 0) 0x7f76bc0035b0 con 0x7f76d40a2950 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 msgr2=0x7f76d413cf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76d413cf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 msgr2=0x7f76d413f880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76d413f880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d409f710 con 0x7f76d40a2950 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f76c4001c60 con 0x7f76d40a2950 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: dump: 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.057 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 >> v1:172.21.15.179:6789/0 conn(0x7f76d40a2950 legacy=0x7f76d40a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient: _add_conns ranks=[5,7,1,6,0,4,2,8,3] 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76bc004250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): picked mon.h con 0x7f76d413d490 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76bc004a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): picked mon.i con 0x7f76d40a3440 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76bc0051e0 0x7f76bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): picked mon.f con 0x7f76bc0051e0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4072830 con 0x7f76bc0051e0 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76d409f150 con 0x7f76d413d490 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc00c0e0 con 0x7f76d40a3440 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d8a0c700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d9a0e700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 msgr2=0x7f76bc004250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d920d700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 msgr2=0x7f76bc004a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d9a0e700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76bc004250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.041+0000 7f76d920d700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76bc004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76bc0051e0 0x7f76bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): get_auth_request con 0x7f76bc0051e0 auth_method 0 2024-04-20T02:27:09.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient(hunting): handle_auth_done global_id 55108 payload 293 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 msgr2=0x7f76bc004a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76bc004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 msgr2=0x7f76bc004250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76bc004250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 monclient: found mon.f 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 -- 172.21.15.47:0/3828701235 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0035b0 con 0x7f76bc0051e0 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76dbc72700 5 monclient: authenticate success, global_id 55108 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76dbc72700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76da20f700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76bc0051e0 0x7f76bc00ba10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0039c0 tx=0x7f76bc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab4cb341c3fac42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76cc009810 con 0x7f76bc0051e0 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: dump: 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.061 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76dbc72700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00b8a0 con 0x7f76bc0051e0 2024-04-20T02:27:09.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76dbc72700 10 monclient: get_monmap_and_config success 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 1 -- 172.21.15.47:0/3828701235 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76cc00be80 con 0x7f76bc0051e0 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76dbc72700 4 set_mon_vals no callback set 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.042+0000 7f76d8a0c700 10 monclient: dump: 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.063 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 10 monclient: shutdown 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 -- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76bc0051e0 msgr2=0x7f76bc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76bc0051e0 0x7f76bc00ba10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0039c0 tx=0x7f76bc005880 comp rx=0 tx=0).stop 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 -- 172.21.15.47:0/3828701235 shutdown_connections 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76bc0051e0 0x7f76bc00ba10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f76d40a3440 0x7f76bc004a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 --2- 172.21.15.47:0/3828701235 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f76d413d490 0x7f76bc004250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 -- 172.21.15.47:0/3828701235 >> 172.21.15.47:0/3828701235 conn(0x7f76d40a7570 msgr2=0x7f76d40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 -- 172.21.15.47:0/3828701235 shutdown_connections 2024-04-20T02:27:09.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.043+0000 7f76dbc72700 1 -- 172.21.15.47:0/3828701235 wait complete. 2024-04-20T02:27:09.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 10 monclient: monmap: 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.066 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 1 librados: starting msgr at 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 1 librados: starting objecter 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 1 Processor -- start 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 1 -- start start 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 1 librados: setting wanted keys 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 1 librados: calling monclient init 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.044+0000 7f76dbc72700 10 monclient: init 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient: _add_conns ranks=[4,6,0,3,2,7,1,8,5] 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f76bc0051e0 0x7f76d41c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): picked mon.noname-c con 0x7f76bc0051e0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 0x7f76d413caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): picked mon.noname-d con 0x7f76d40a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76d40a3440 0x7f76d41cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): picked mon.noname-a con 0x7f76d40a3440 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4127ea0 con 0x7f76d40a3440 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76d409dd70 con 0x7f76bc0051e0 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76d41c9220 con 0x7f76d40a2950 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76dbc72700 10 monclient(hunting): authenticate will time out at 2308.446113s 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d9a0e700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f76bc0051e0 msgr2=0x7f76d41c87c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d9a0e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f76bc0051e0 0x7f76d41c87c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 0x7f76d413caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76d40a3440 0x7f76d41cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76d40a3440 0x7f76d41cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46354/0 (socket says 172.21.15.47:46354) 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 0x7f76d413caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:47458/0 (socket says 172.21.15.47:47458) 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 1 -- 172.21.15.47:0/2659176052 learned_addr learned my addr 172.21.15.47:0/2659176052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): get_auth_request con 0x7f76d40a3440 auth_method 0 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): get_auth_request con 0x7f76d40a2950 auth_method 0 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient(hunting): handle_auth_done global_id 54753 payload 824 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f76bc0051e0 msgr2=0x7f76d41c87c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f76bc0051e0 0x7f76d41c87c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76d40a3440 msgr2=0x7f76d41cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76d40a3440 0x7f76d41cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 monclient: found mon.noname-d 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d409d960 con 0x7f76d40a2950 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76da20f700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f76d40a3440 0x7f76d41cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d920d700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 0x7f76d413caa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f76d40a7e40 tx=0x7f76c800b7e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d27fc700 10 client.?.objecter ms_handle_connect 0x7f76d40a2950 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.045+0000 7f76d27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76c8003c40 con 0x7f76d40a2950 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: dump: 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.071 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c8003de0 con 0x7f76d40a2950 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 5 monclient: authenticate success, global_id 54753 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 monclient: _renew_subs 2024-04-20T02:27:09.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d4127bc0 con 0x7f76d40a2950 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76c8011400 con 0x7f76d40a2950 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d27fc700 10 monclient: dump: 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.073 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76d3fff700 4 set_mon_vals no callback set 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 client.54753.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 monclient: _renew_subs 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d4126e80 con 0x7f76d40a2950 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 1 librados: init done 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 0x7f76d413cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.046+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4127160 con 0x7f76d406aac0 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 0x7f76d413cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient: get_auth_request con 0x7f76d406aac0 auth_method 0 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f76c8005070 con 0x7f76d40a2950 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76c003a320 0x7f76c003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_done global_id 24374 payload 824 2024-04-20T02:27:09.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76d409ee70 con 0x7f76d406aac0 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(431..431 src has 266..431) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f76c8049f90 con 0x7f76d40a2950 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 10 client.54753.objecter ms_dispatch 0x7f76d4001550 osd_map(431..431 src has 266..431) v4 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 3 client.54753.objecter handle_osd_map got epochs [431,431] > 0 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 3 client.54753.objecter handle_osd_map decoding full epoch 431 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d9a0e700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 0x7f76d413cfe0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f76c40014a0 tx=0x7f76c400a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 20 client.54753.objecter dump_active .. 0 homeless 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 10 client.54753.objecter ms_handle_connect 0x7f76d406aac0 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 10 client.54753.objecter resend_mon_ops 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76da20f700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76c003a320 0x7f76c003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76da20f700 10 monclient: get_auth_request con 0x7f76c003a320 auth_method 0 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76c400ada0 con 0x7f76d406aac0 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.047+0000 7f76d27fc700 10 monclient: dump: 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.077 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f76c400a930 con 0x7f76d406aac0 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 msgr2=0x7f76d413cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 0x7f76d413cfe0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f76c40014a0 tx=0x7f76c400a060 comp rx=0 tx=0).stop 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 msgr2=0x7f76d413cfe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d406aac0 0x7f76d413cfe0 secure :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f76c40014a0 tx=0x7f76c400a060 comp rx=0 tx=0).stop 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76da20f700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76c003a320 0x7f76c003c7e0 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009fd0 tx=0x7f76cc005000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.048+0000 7f76d27fc700 10 client.54753.objecter ms_handle_connect 0x7f76c003a320 2024-04-20T02:27:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76dbc72700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:27:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76dbc72700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 0x7f76d41cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76dbc72700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40724b0 con 0x7f76d41280c0 2024-04-20T02:27:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 0x7f76d41cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient: get_auth_request con 0x7f76d41280c0 auth_method 0 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 10 monclient(hunting): handle_auth_done global_id 24383 payload 824 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 1 -- 172.21.15.47:0/2659176052 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76d4067730 con 0x7f76d41280c0 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d9a0e700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 0x7f76d41cc260 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f76c40012e0 tx=0x7f76d40670b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d27fc700 10 client.54753.objecter ms_handle_connect 0x7f76d41280c0 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d27fc700 10 client.54753.objecter resend_mon_ops 2024-04-20T02:27:09.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76c4020070 con 0x7f76d41280c0 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.056+0000 7f76d27fc700 10 monclient: dump: 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.085 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.086 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9141 (secure 0 0 0) 0x7f76c4024410 con 0x7f76d41280c0 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 msgr2=0x7f76d41cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 0x7f76d41cc260 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f76c40012e0 tx=0x7f76d40670b0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 msgr2=0x7f76d41cc260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76d27fc700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f76d41280c0 0x7f76d41cc260 secure :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f76c40012e0 tx=0x7f76d40670b0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 78, 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:27:09.088 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:27:09.089 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:27:09.089 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:27:09.089 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.089 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:27:09.089 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.089 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.090 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.091 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.091 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.091 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:27:09.091 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:27:09.091 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.092 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.092 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.092 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.092 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:27:09.093 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:27:09.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.095 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.096 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:27:09.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.099 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.103 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.104 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:27:09.106 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:27:09.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.113 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.116 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.117 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:27:09.117 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:27:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 10 librados: watch_flush enter 2024-04-20T02:27:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 10 librados: watch_flush exit 2024-04-20T02:27:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 20 client.54753.objecter shutdown clearing up homeless session... 2024-04-20T02:27:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 10 client.54753.objecter successfully canceled tick 2024-04-20T02:27:09.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76c003a320 msgr2=0x7f76c003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f76c003a320 0x7f76c003c7e0 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009fd0 tx=0x7f76cc005000 comp rx=0 tx=0).stop 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 10 monclient: shutdown 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 msgr2=0x7f76d413caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 0x7f76d413caa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f76d40a7e40 tx=0x7f76c800b7e0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76da20f700 1 -- 172.21.15.47:0/2659176052 reap_dead start 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 shutdown_connections 2024-04-20T02:27:09.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 --2- 172.21.15.47:0/2659176052 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f76d40a2950 0x7f76d413caa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 >> 172.21.15.47:0/2659176052 conn(0x7f76d40a7570 msgr2=0x7f76d409faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 shutdown_connections 2024-04-20T02:27:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 -- 172.21.15.47:0/2659176052 wait complete. 2024-04-20T02:27:09.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.057+0000 7f76dbc72700 1 librados: shutdown 2024-04-20T02:27:09.122 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:27:09.122 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-04-20T02:27:09.122 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:27:09.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.261+0000 7fd2df34e700 10 monclient: get_monmap_and_config 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.261+0000 7fd2df34e700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.261+0000 7fd2df34e700 10 monclient: monmap: 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:27:09.262546+0000 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:27:09.262546+0000 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.262 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:27:09.263 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 Processor -- start 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 -- start start 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient: init 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient: _add_conns ranks=[10,6,0,5,7,1,2,8,4,3,9] 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd2d8066610 0x7fd2d80a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): picked mon.noname-f con 0x7fd2d8066610 addr v2:172.21.15.179:3300/0 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d80a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): picked mon.noname-d con 0x7fd2d8099e00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd2d80a2990 addr v1:172.21.15.47:6789/0 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80751e0 con 0x7fd2d80a2990 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d809ba30 con 0x7fd2d8066610 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a3f70 con 0x7fd2d8099e00 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.262+0000 7fd2df34e700 10 monclient(hunting): authenticate will time out at 2308.663968s 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d80a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd8eb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd2d80a2990 0x7fd2d813e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50864/0 (socket says 172.21.15.47:50864) 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd8eb700 1 -- 172.21.15.47:0/1324552741 learned_addr learned my addr 172.21.15.47:0/1324552741 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:27:09.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): get_auth_request con 0x7fd2d8099e00 auth_method 0 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd0ea700 1 --2- 172.21.15.47:0/1324552741 >> v2:172.21.15.179:3300/0 conn(0x7fd2d8066610 0x7fd2d80a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd0ea700 10 monclient(hunting): get_auth_request con 0x7fd2d8066610 auth_method 0 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd0ea700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd0ea700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd0ea700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 1 -- 172.21.15.47:0/1324552741 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174975418 0 0) 0x7fd2d80751e0 con 0x7fd2d80a2990 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient(hunting): _negotiate 2024-04-20T02:27:09.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient(hunting): my global_id is 36198 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 1 -- 172.21.15.47:0/1324552741 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c40035b0 con 0x7fd2d80a2990 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_done global_id 54762 payload 274 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/1324552741 >> v2:172.21.15.179:3300/0 conn(0x7fd2d8066610 msgr2=0x7fd2d80a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 --2- 172.21.15.47:0/1324552741 >> v2:172.21.15.179:3300/0 conn(0x7fd2d8066610 0x7fd2d80a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/1324552741 >> v1:172.21.15.47:6789/0 conn(0x7fd2d80a2990 legacy=0x7fd2d813e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 monclient: found mon.noname-d 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/1324552741 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d809ced0 con 0x7fd2d8099e00 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dd0ea700 1 --2- 172.21.15.47:0/1324552741 >> v2:172.21.15.179:3300/0 conn(0x7fd2d8066610 0x7fd2d80a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2dc8e9700 1 --2- 172.21.15.47:0/1324552741 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d80a2450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009840 tx=0x7fd2c8009b50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9be0d33e1b187701 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 1 -- 172.21.15.47:0/1324552741 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2c8011aa0 con 0x7fd2d8099e00 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: dump: 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.283 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 1 -- 172.21.15.47:0/1324552741 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8011c00 con 0x7fd2d8099e00 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2df34e700 5 monclient: authenticate success, global_id 54762 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2df34e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2df34e700 10 monclient: get_monmap_and_config success 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 1 -- 172.21.15.47:0/1324552741 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2c8011d70 con 0x7fd2d8099e00 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2df34e700 4 set_mon_vals no callback set 2024-04-20T02:27:09.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.263+0000 7fd2cffff700 10 monclient: dump: 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.285 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 10 monclient: shutdown 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 -- 172.21.15.47:0/1324552741 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 msgr2=0x7fd2d80a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 --2- 172.21.15.47:0/1324552741 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d80a2450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009840 tx=0x7fd2c8009b50 comp rx=0 tx=0).stop 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 -- 172.21.15.47:0/1324552741 shutdown_connections 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 --2- 172.21.15.47:0/1324552741 >> v2:172.21.15.179:3300/0 conn(0x7fd2d8066610 0x7fd2d80a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 --2- 172.21.15.47:0/1324552741 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d80a2450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 -- 172.21.15.47:0/1324552741 >> 172.21.15.47:0/1324552741 conn(0x7fd2d80a3000 msgr2=0x7fd2d80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 -- 172.21.15.47:0/1324552741 shutdown_connections 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.264+0000 7fd2df34e700 1 -- 172.21.15.47:0/1324552741 wait complete. 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 10 monclient: monmap: 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.287 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 1 librados: starting msgr at 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 1 librados: starting objecter 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 1 Processor -- start 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 1 -- start start 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 1 librados: setting wanted keys 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 1 librados: calling monclient init 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.265+0000 7fd2df34e700 10 monclient: init 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient: _add_conns ranks=[4,2,3,6,0,5,1,7,8] 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2d8066610 0x7fd2d81cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): picked mon.noname-c con 0x7fd2d8066610 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2d80a2990 0x7fd2d8098630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): picked mon.noname-b con 0x7fd2d80a2990 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:27:09.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2d8098b70 0x7fd2d806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): picked mon.noname-g con 0x7fd2d8098b70 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8072a60 con 0x7fd2d80a2990 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d809a9d0 con 0x7fd2d8098b70 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d809d0b0 con 0x7fd2d8066610 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dd0ea700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2d8066610 msgr2=0x7fd2d81cc550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient(hunting): authenticate will time out at 2308.667113s 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dd0ea700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2d8066610 0x7fd2d81cc550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2d80a2990 0x7fd2d8098630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dd8eb700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2d8098b70 msgr2=0x7fd2d806ef30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2d80a2990 0x7fd2d8098630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43742/0 (socket says 172.21.15.47:43742) 2024-04-20T02:27:09.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/2301170842 learned_addr learned my addr 172.21.15.47:0/2301170842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): get_auth_request con 0x7fd2d80a2990 auth_method 0 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dd8eb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2d8098b70 0x7fd2d806ef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient(hunting): handle_auth_done global_id 24392 payload 824 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2d8066610 msgr2=0x7fd2d81cc550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2d8066610 0x7fd2d81cc550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2d8098b70 msgr2=0x7fd2d806ef30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2d8098b70 0x7fd2d806ef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 monclient: found mon.noname-b 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80094a0 con 0x7fd2d80a2990 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2dc8e9700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2d80a2990 0x7fd2d8098630 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009b90 tx=0x7fd2c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 client.?.objecter ms_handle_connect 0x7fd2d80a2990 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2c80045d0 con 0x7fd2d80a2990 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: dump: 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.291 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c801f070 con 0x7fd2d80a2990 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 5 monclient: authenticate success, global_id 24392 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient: _renew_subs 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d8072cb0 con 0x7fd2d80a2990 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2c8011a00 con 0x7fd2d80a2990 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cdffb700 10 monclient: dump: 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.293 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 10 client.24392.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.266+0000 7fd2cf7fe700 4 set_mon_vals no callback set 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 10 monclient: _renew_subs 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:27:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d8070900 con 0x7fd2d80a2990 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 1 librados: init done 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.267+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8068180 con 0x7fd2d8099e00 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd2c8021030 con 0x7fd2d80a2990 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient: get_auth_request con 0x7fd2d8099e00 auth_method 0 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd2bc03a320 0x7fd2bc03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(431..431 src has 266..431) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd2c804bbb0 con 0x7fd2d80a2990 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 10 client.24392.objecter ms_dispatch 0x7fd2d8001550 osd_map(431..431 src has 266..431) v4 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 3 client.24392.objecter handle_osd_map got epochs [431,431] > 0 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 3 client.24392.objecter handle_osd_map decoding full epoch 431 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_done global_id 54771 payload 824 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2d809db30 con 0x7fd2d8099e00 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 20 client.24392.objecter dump_active .. 0 homeless 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 10 client.24392.objecter ms_handle_connect 0x7fd2d8099e00 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 10 client.24392.objecter resend_mon_ops 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd0ea700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d806f470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40091d0 tx=0x7fd2d400da90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd8eb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd2bc03a320 0x7fd2bc03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2dd8eb700 10 monclient: get_auth_request con 0x7fd2bc03a320 auth_method 0 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2d4012a60 con 0x7fd2d8099e00 2024-04-20T02:27:09.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.268+0000 7fd2cdffb700 10 monclient: dump: 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd2d4012bc0 con 0x7fd2d8099e00 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 msgr2=0x7fd2d806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d806f470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40091d0 tx=0x7fd2d400da90 comp rx=0 tx=0).stop 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 msgr2=0x7fd2d806f470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d8099e00 0x7fd2d806f470 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40091d0 tx=0x7fd2d400da90 comp rx=0 tx=0).stop 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2cdffb700 10 client.24392.objecter ms_handle_connect 0x7fd2bc03a320 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.269+0000 7fd2dd8eb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd2bc03a320 0x7fd2bc03c7e0 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0004c20 tx=0x7fd2d0001610 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.276+0000 7fd2df34e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2df34e700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 0x7fd2d806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2df34e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d806a400 con 0x7fd2d806aac0 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 0x7fd2d806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient: get_auth_request con 0x7fd2d806aac0 auth_method 0 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 10 monclient(hunting): handle_auth_done global_id 54780 payload 824 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 1 -- 172.21.15.47:0/2301170842 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd2d80669f0 con 0x7fd2d806aac0 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2dd0ea700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 0x7fd2d806f9b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40091d0 tx=0x7fd2d40047c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 client.24392.objecter ms_handle_connect 0x7fd2d806aac0 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 client.24392.objecter resend_mon_ops 2024-04-20T02:27:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2d401ee50 con 0x7fd2d806aac0 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 monclient: dump: 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.300 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7fd2d4022070 con 0x7fd2d806aac0 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 msgr2=0x7fd2d806f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 0x7fd2d806f9b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40091d0 tx=0x7fd2d40047c0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 msgr2=0x7fd2d806f9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.277+0000 7fd2cdffb700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fd2d806aac0 0x7fd2d806f9b0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40091d0 tx=0x7fd2d40047c0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 78, 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:27:09.302 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.303 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.304 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:27:09.305 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:27:09.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.311 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:27:09.312 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.314 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.316 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:27:09.318 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.319 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.322 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:27:09.323 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:27:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 10 librados: watch_flush enter 2024-04-20T02:27:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 10 librados: watch_flush exit 2024-04-20T02:27:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 20 client.24392.objecter shutdown clearing up homeless session... 2024-04-20T02:27:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 10 client.24392.objecter successfully canceled tick 2024-04-20T02:27:09.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd2bc03a320 msgr2=0x7fd2bc03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd2bc03a320 0x7fd2bc03c7e0 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0004c20 tx=0x7fd2d0001610 comp rx=0 tx=0).stop 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 10 monclient: shutdown 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2d80a2990 msgr2=0x7fd2d8098630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 --2- 172.21.15.47:0/2301170842 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2d80a2990 0x7fd2d8098630 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009b90 tx=0x7fd2c8004750 comp rx=0 tx=0).stop 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2dd8eb700 1 -- 172.21.15.47:0/2301170842 reap_dead start 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2dd8eb700 1 -- 172.21.15.47:0/2301170842 reap_dead start 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 shutdown_connections 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 >> 172.21.15.47:0/2301170842 conn(0x7fd2d80a3000 msgr2=0x7fd2d80a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 shutdown_connections 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 -- 172.21.15.47:0/2301170842 wait complete. 2024-04-20T02:27:09.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.278+0000 7fd2df34e700 1 librados: shutdown 2024-04-20T02:27:09.327 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:27:09.327 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-04-20T02:27:09.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:27:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.467+0000 7f9f5b656700 10 monclient: get_monmap_and_config 2024-04-20T02:27:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.467+0000 7f9f5b656700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.467+0000 7f9f5b656700 10 monclient: monmap: 2024-04-20T02:27:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.468 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:27:09.468278+0000 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:27:09.468278+0000 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.469 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:27:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:27:09.470 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.467+0000 7f9f5b656700 1 Processor -- start 2024-04-20T02:27:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 -- start start 2024-04-20T02:27:09.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient: init 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient: _add_conns ranks=[5,9,3,6,7,10,8,1,2,4,0] 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f54137d90 0x7f9f540a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): picked mon.noname-h con 0x7f9f54137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f54138170 0x7f9f540a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): picked mon.noname-a con 0x7f9f54138170 addr v2:172.21.15.47:3300/0 2024-04-20T02:27:09.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f541385b0 0x7f9f540a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): picked mon.noname-g con 0x7f9f541385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5409fa40 con 0x7f9f54138170 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5409dfc0 con 0x7f9f541385b0 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f540a7bc0 con 0x7f9f54137d90 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f5b656700 10 monclient(hunting): authenticate will time out at 2308.869620s 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f593f2700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f54137d90 msgr2=0x7f9f540a09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f59bf3700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f541385b0 msgr2=0x7f9f540a61d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:09.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f54138170 0x7f9f540a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f593f2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f54137d90 0x7f9f540a09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f59bf3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f541385b0 0x7f9f540a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f54138170 0x7f9f540a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46366/0 (socket says 172.21.15.47:46366) 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 1 -- 172.21.15.47:0/946562612 learned_addr learned my addr 172.21.15.47:0/946562612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): get_auth_request con 0x7f9f54138170 auth_method 0 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.468+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_done global_id 36207 payload 274 2024-04-20T02:27:09.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 -- 172.21.15.47:0/946562612 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f54137d90 msgr2=0x7f9f540a09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f54137d90 0x7f9f540a09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 -- 172.21.15.47:0/946562612 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f541385b0 msgr2=0x7f9f540a61d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f541385b0 0x7f9f540a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 monclient: found mon.noname-a 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 -- 172.21.15.47:0/946562612 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f540751e0 con 0x7f9f54138170 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f58bf1700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f54138170 0x7f9f540a5c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9f440098f0 tx=0x7f9f44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3758a02e59be114 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 1 -- 172.21.15.47:0/946562612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f4400bb10 con 0x7f9f54138170 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: dump: 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.485 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f5b656700 5 monclient: authenticate success, global_id 36207 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f5b656700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 1 -- 172.21.15.47:0/946562612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400bc70 con 0x7f9f54138170 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f5b656700 10 monclient: get_monmap_and_config success 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f5b656700 4 set_mon_vals no callback set 2024-04-20T02:27:09.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 1 -- 172.21.15.47:0/946562612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f4400bde0 con 0x7f9f54138170 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.469+0000 7f9f4bfff700 10 monclient: dump: 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.488 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 10 monclient: shutdown 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 -- 172.21.15.47:0/946562612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f54138170 msgr2=0x7f9f540a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f54138170 0x7f9f540a5c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9f440098f0 tx=0x7f9f44009c00 comp rx=0 tx=0).stop 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 -- 172.21.15.47:0/946562612 shutdown_connections 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f541385b0 0x7f9f540a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f54137d90 0x7f9f540a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 --2- 172.21.15.47:0/946562612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f54138170 0x7f9f540a5c90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 -- 172.21.15.47:0/946562612 >> 172.21.15.47:0/946562612 conn(0x7f9f540aa8b0 msgr2=0x7f9f540a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 -- 172.21.15.47:0/946562612 shutdown_connections 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 1 -- 172.21.15.47:0/946562612 wait complete. 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.470+0000 7f9f5b656700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 10 monclient: monmap: 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.490 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 librados: starting msgr at 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 librados: starting objecter 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 Processor -- start 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 -- start start 2024-04-20T02:27:09.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 librados: setting wanted keys 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 librados: calling monclient init 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 10 monclient: init 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 10 monclient: _add_conns ranks=[4,8,5,0,1,7,6,3,2] 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f54137d90 0x7f9f541c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 10 monclient(hunting): picked mon.noname-c con 0x7f9f54137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 0x7f9f54136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.471+0000 7f9f5b656700 10 monclient(hunting): picked mon.noname-e con 0x7f9f54138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f541385b0 0x7f9f541373b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f593f2700 1 -- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f54137d90 msgr2=0x7f9f541c8ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3302/0 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient(hunting): picked mon.noname-h con 0x7f9f541385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5409ea50 con 0x7f9f54137d90 2024-04-20T02:27:09.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f593f2700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f54137d90 0x7f9f541c8ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54124270 con 0x7f9f541385b0 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5409dc20 con 0x7f9f54138170 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient(hunting): authenticate will time out at 2308.873067s 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 0x7f9f54136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f59bf3700 1 -- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f541385b0 msgr2=0x7f9f541373b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3302/0 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f59bf3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f541385b0 0x7f9f541373b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 0x7f9f54136e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:51514/0 (socket says 172.21.15.47:51514) 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 -- 172.21.15.47:0/1374401605 learned_addr learned my addr 172.21.15.47:0/1374401605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): get_auth_request con 0x7f9f54138170 auth_method 0 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient(hunting): handle_auth_done global_id 35567 payload 824 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f541385b0 msgr2=0x7f9f541373b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f541385b0 0x7f9f541373b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f54137d90 msgr2=0x7f9f541c8ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f54137d90 0x7f9f541c8ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 monclient: found mon.noname-e 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44009550 con 0x7f9f54138170 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f58bf1700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 0x7f9f54136e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009840 tx=0x7f9f44004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 client.?.objecter ms_handle_connect 0x7f9f54138170 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f44004490 con 0x7f9f54138170 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 monclient: dump: 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.495 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.496 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 5 monclient: authenticate success, global_id 35567 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f44003b10 con 0x7f9f54138170 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient: _renew_subs 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f54122460 con 0x7f9f54138170 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f44003c80 con 0x7f9f54138170 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f49ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f49ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f5b656700 10 client.35567.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:27:09.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.472+0000 7f9f4b7fe700 4 set_mon_vals no callback set 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f49ffb700 10 monclient: dump: 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.499 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f5b656700 10 monclient: _renew_subs 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f5b656700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f54123250 con 0x7f9f54138170 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.473+0000 7f9f5b656700 1 librados: init done 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f5b656700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f5b656700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 0x7f9f541cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54068180 con 0x7f9f5406aac0 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9f44021070 con 0x7f9f54138170 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 0x7f9f541cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient: get_auth_request con 0x7f9f5406aac0 auth_method 0 2024-04-20T02:27:09.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9f3803a2f0 0x7f9f3803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 10 monclient(hunting): handle_auth_done global_id 35576 payload 824 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(431..431 src has 266..431) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9f4404bce0 con 0x7f9f54138170 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 10 client.35567.objecter ms_dispatch 0x7f9f54001550 osd_map(431..431 src has 266..431) v4 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 3 client.35567.objecter handle_osd_map got epochs [431,431] > 0 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 3 client.35567.objecter handle_osd_map decoding full epoch 431 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f540a11e0 con 0x7f9f5406aac0 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 20 client.35567.objecter dump_active .. 0 homeless 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f593f2700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 0x7f9f541cc1e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b450 tx=0x7f9f5000b810 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 10 client.35567.objecter ms_handle_connect 0x7f9f5406aac0 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 10 client.35567.objecter resend_mon_ops 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f50003c50 con 0x7f9f5406aac0 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f59bf3700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9f3803a2f0 0x7f9f3803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f49ffb700 10 monclient: dump: 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.502 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.474+0000 7f9f59bf3700 10 monclient: get_auth_request con 0x7f9f3803a2f0 auth_method 0 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9f50003db0 con 0x7f9f5406aac0 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 msgr2=0x7f9f541cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 0x7f9f541cc1e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b450 tx=0x7f9f5000b810 comp rx=0 tx=0).stop 2024-04-20T02:27:09.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 msgr2=0x7f9f541cc1e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f5406aac0 0x7f9f541cc1e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f49ffb700 10 client.35567.objecter ms_handle_connect 0x7f9f3803a2f0 2024-04-20T02:27:09.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.475+0000 7f9f59bf3700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9f3803a2f0 0x7f9f3803c7b0 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 crypto rx=0x7f9f54138a20 tx=0x7f9f4c009230 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.482+0000 7f9f5b656700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f5b656700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 0x7f9f541cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f5b656700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5406a400 con 0x7f9f541244b0 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 0x7f9f541cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient: get_auth_request con 0x7f9f541244b0 auth_method 0 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 10 monclient(hunting): handle_auth_done global_id 35585 payload 824 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 1 -- 172.21.15.47:0/1374401605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f5409f760 con 0x7f9f541244b0 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f49ffb700 10 client.35567.objecter ms_handle_connect 0x7f9f541244b0 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f49ffb700 10 client.35567.objecter resend_mon_ops 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f593f2700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 0x7f9f541cc720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b450 tx=0x7f9f50005a60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f5001e070 con 0x7f9f541244b0 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f49ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f49ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.483+0000 7f9f49ffb700 10 monclient: dump: 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.507 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9309 (secure 0 0 0) 0x7f9f50009350 con 0x7f9f541244b0 2024-04-20T02:27:09.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 msgr2=0x7f9f541cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 0x7f9f541cc720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b450 tx=0x7f9f50005a60 comp rx=0 tx=0).stop 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 msgr2=0x7f9f541cc720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f49ffb700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f541244b0 0x7f9f541cc720 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b450 tx=0x7f9f50005a60 comp rx=0 tx=0).stop 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 78, 2024-04-20T02:27:09.509 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.510 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:27:09.511 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:27:09.512 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:27:09.513 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:27:09.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.517 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:27:09.520 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.521 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.523 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:27:09.524 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.528 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.531 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:27:09.532 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:27:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 10 librados: watch_flush enter 2024-04-20T02:27:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 10 librados: watch_flush exit 2024-04-20T02:27:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 20 client.35567.objecter shutdown clearing up homeless session... 2024-04-20T02:27:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 10 client.35567.objecter successfully canceled tick 2024-04-20T02:27:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9f3803a2f0 msgr2=0x7f9f3803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9f3803a2f0 0x7f9f3803c7b0 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 crypto rx=0x7f9f54138a20 tx=0x7f9f4c009230 comp rx=0 tx=0).stop 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 10 monclient: shutdown 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 msgr2=0x7f9f54136e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f59bf3700 1 -- 172.21.15.47:0/1374401605 reap_dead start 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 0x7f9f54136e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009840 tx=0x7f9f44004750 comp rx=0 tx=0).stop 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 shutdown_connections 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 --2- 172.21.15.47:0/1374401605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9f54138170 0x7f9f54136e70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 >> 172.21.15.47:0/1374401605 conn(0x7f9f540aa8b0 msgr2=0x7f9f541bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 shutdown_connections 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 -- 172.21.15.47:0/1374401605 wait complete. 2024-04-20T02:27:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.484+0000 7f9f5b656700 1 librados: shutdown 2024-04-20T02:27:09.535 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:27:09.536 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon f 2024-04-20T02:27:09.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.673+0000 7f1ab1bb3700 10 monclient: get_monmap_and_config 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.673+0000 7f1ab1bb3700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.673+0000 7f1ab1bb3700 10 monclient: monmap: 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:27:09.674941+0000 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:27:09.674941+0000 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:27:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:27:09.675 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.674+0000 7f1ab1bb3700 1 Processor -- start 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.674+0000 7f1ab1bb3700 1 -- start start 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.674+0000 7f1ab1bb3700 10 monclient: init 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient: _add_conns ranks=[7,2,8,9,0,6,3,4,5,10,1] 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1aac0a50f0 0x7f1aac137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): picked mon.noname-i con 0x7f1aac0a50f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aac1378a0 0x7f1aac138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): picked mon.noname-b con 0x7f1aac1378a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 0x7f1aac0a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): picked mon.noname-e con 0x7f1aac0a38f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0a54d0 con 0x7f1aac1378a0 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac1387b0 con 0x7f1aac0a50f0 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac06da20 con 0x7f1aac0a38f0 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1ab1bb3700 10 monclient(hunting): authenticate will time out at 2309.076223s 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aab7fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1aac0a50f0 msgr2=0x7f1aac137d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aab7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1aac0a50f0 0x7f1aac137d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 0x7f1aac0a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aac1378a0 0x7f1aac138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 0x7f1aac0a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:51532/0 (socket says 172.21.15.47:51532) 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 1 -- 172.21.15.47:0/560430731 learned_addr learned my addr 172.21.15.47:0/560430731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): get_auth_request con 0x7f1aac0a38f0 auth_method 0 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): get_auth_request con 0x7f1aac1378a0 auth_method 0 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_done global_id 24401 payload 274 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 -- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 msgr2=0x7f1aac0a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 0x7f1aac0a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 -- 172.21.15.47:0/560430731 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1aac0a50f0 msgr2=0x7f1aac137d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1aac0a50f0 0x7f1aac137d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 monclient: found mon.noname-b 2024-04-20T02:27:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:27:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 -- 172.21.15.47:0/560430731 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0751e0 con 0x7f1aac1378a0 2024-04-20T02:27:09.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aabfff700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 0x7f1aac0a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.675+0000 7f1aaaffd700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aac1378a0 0x7f1aac138270 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009870 tx=0x7f1aa0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e4cdc091706b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 1 -- 172.21.15.47:0/560430731 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aa0011aa0 con 0x7f1aac1378a0 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: dump: 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 1 -- 172.21.15.47:0/560430731 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0011c00 con 0x7f1aac1378a0 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 5 monclient: authenticate success, global_id 24401 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 10 monclient: get_monmap_and_config success 2024-04-20T02:27:09.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 1 -- 172.21.15.47:0/560430731 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aa0011d70 con 0x7f1aac1378a0 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 4 set_mon_vals no callback set 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1aaa7fc700 10 monclient: dump: 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 10 monclient: shutdown 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aac1378a0 msgr2=0x7f1aac138270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aac1378a0 0x7f1aac138270 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009870 tx=0x7f1aa0009b80 comp rx=0 tx=0).stop 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/560430731 shutdown_connections 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a38f0 0x7f1aac0a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f1aac0a50f0 0x7f1aac137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/560430731 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1aac1378a0 0x7f1aac138270 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/560430731 >> 172.21.15.47:0/560430731 conn(0x7f1aac0a8510 msgr2=0x7f1aac0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/560430731 shutdown_connections 2024-04-20T02:27:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.676+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/560430731 wait complete. 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.677+0000 7f1ab1bb3700 10 monclient: build_initial_monmap 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.677+0000 7f1ab1bb3700 10 monclient: monmap: 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:27:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.677+0000 7f1ab1bb3700 1 librados: starting msgr at 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.677+0000 7f1ab1bb3700 1 librados: starting objecter 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.677+0000 7f1ab1bb3700 1 Processor -- start 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 -- start start 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 librados: setting wanted keys 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 librados: calling monclient init 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient: init 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient: _reopen_session rank -1 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient: _add_conns ranks=[1,8,3,0,5,4,7,2,6] 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac0a38f0 0x7f1aac1cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): picked mon.noname-f con 0x7f1aac0a38f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:27:09.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 0x7f1aac1369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): picked mon.noname-e con 0x7f1aac0a50f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1aac136f10 0x7f1aac120480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): picked mon.noname-g con 0x7f1aac136f10 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac1229c0 con 0x7f1aac0a38f0 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac072260 con 0x7f1aac136f10 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac071e80 con 0x7f1aac0a50f0 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): _renew_subs 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1ab1bb3700 10 monclient(hunting): authenticate will time out at 2309.079611s 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 0x7f1aac1369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aabfff700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1aac136f10 msgr2=0x7f1aac120480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:27:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 0x7f1aac1369d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:51540/0 (socket says 172.21.15.47:51540) 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 1 -- 172.21.15.47:0/664963318 learned_addr learned my addr 172.21.15.47:0/664963318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): get_auth_request con 0x7f1aac0a50f0 auth_method 0 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aabfff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1aac136f10 0x7f1aac120480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aab7fe700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac0a38f0 0x7f1aac1cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aab7fe700 10 monclient(hunting): get_auth_request con 0x7f1aac0a38f0 auth_method 0 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.678+0000 7f1aab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 10 monclient(hunting): handle_auth_done global_id 35603 payload 824 2024-04-20T02:27:09.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1aac136f10 msgr2=0x7f1aac120480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1aac136f10 0x7f1aac120480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac0a38f0 msgr2=0x7f1aac1cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac0a38f0 0x7f1aac1cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 monclient: found mon.noname-e 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009b30 con 0x7f1aac0a50f0 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aaaffd700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 0x7f1aac1369d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009790 tx=0x7f1aa0004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 client.?.objecter ms_handle_connect 0x7f1aac0a50f0 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aa00043d0 con 0x7f1aac0a50f0 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:27:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: dump: 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.701 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: _finish_auth 0 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: _check_auth_tickets 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 5 monclient: authenticate success, global_id 35603 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 10 monclient: _renew_subs 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0004570 con 0x7f1aac0a50f0 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa00094a0 con 0x7f1aac0a50f0 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1aa001a420 con 0x7f1aac0a50f0 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1a93fff700 10 monclient: dump: 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.702 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 10 client.35603.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:27:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 10 monclient: _renew_subs 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac123af0 con 0x7f1aac0a50f0 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1ab1bb3700 1 librados: init done 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.679+0000 7f1aa9ffb700 4 set_mon_vals no callback set 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1ab1bb3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 0x7f1aac1209c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac06d850 con 0x7f1aac06aac0 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1aa00036a0 con 0x7f1aac0a50f0 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9803a320 0x7f1a9803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 0x7f1aac1209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient: get_auth_request con 0x7f1aac06aac0 auth_method 0 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(431..431 src has 266..431) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1aa004b1d0 con 0x7f1aac0a50f0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 10 client.35603.objecter ms_dispatch 0x7f1aac001550 osd_map(431..431 src has 266..431) v4 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 3 client.35603.objecter handle_osd_map got epochs [431,431] > 0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 3 client.35603.objecter handle_osd_map decoding full epoch 431 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1a93fff700 20 client.35603.objecter dump_active .. 0 homeless 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aabfff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9803a320 0x7f1a9803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.680+0000 7f1aabfff700 10 monclient: get_auth_request con 0x7f1a9803a320 auth_method 0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_done global_id 55135 payload 824 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1aab7fe700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aac06e370 con 0x7f1aac06aac0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1aab7fe700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 0x7f1aac1209c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d700 tx=0x7f1a9c00dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 client.35603.objecter ms_handle_connect 0x7f1aac06aac0 2024-04-20T02:27:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 client.35603.objecter resend_mon_ops 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 client.35603.objecter ms_handle_connect 0x7f1a9803a320 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1aabfff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9803a320 0x7f1a9803c7e0 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7f1aac121be0 tx=0x7f1aa400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1a9c003c50 con 0x7f1aac06aac0 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 monclient: dump: 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1a9c003db0 con 0x7f1aac06aac0 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 msgr2=0x7f1aac1209c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 0x7f1aac1209c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d700 tx=0x7f1a9c00dac0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 msgr2=0x7f1aac1209c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.681+0000 7f1a93fff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac06aac0 0x7f1aac1209c0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d700 tx=0x7f1a9c00dac0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1ab1bb3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:27:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 0x7f1aac120f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1ab1bb3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0682f0 con 0x7f1aac137830 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 0x7f1aac120f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient: get_auth_request con 0x7f1aac137830 auth_method 0 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.689+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1aab7fe700 10 monclient(hunting): handle_auth_done global_id 55144 payload 824 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1aab7fe700 1 -- 172.21.15.47:0/664963318 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1aac06e650 con 0x7f1aac137830 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 client.35603.objecter ms_handle_connect 0x7f1aac137830 2024-04-20T02:27:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1aab7fe700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 0x7f1aac120f00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d700 tx=0x7f1a9c005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 client.35603.objecter resend_mon_ops 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1a9c020070 con 0x7f1aac137830 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 monclient: dump: 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:27:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9479 (secure 0 0 0) 0x7f1a9c0050d0 con 0x7f1aac137830 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 msgr2=0x7f1aac120f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 0x7f1aac120f00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d700 tx=0x7f1a9c005a60 comp rx=0 tx=0).stop 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 msgr2=0x7f1aac120f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:27:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.690+0000 7f1a93fff700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1aac137830 0x7f1aac120f00 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d700 tx=0x7f1a9c005a60 comp rx=0 tx=0).stop 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 78, 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.713 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.714 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.715 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:27:09.716 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:27:09.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:27:09.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.720 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:27:09.721 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.723 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.725 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:27:09.726 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.729 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:27:09.731 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.732 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:27:09.733 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:27:09.734 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:27:09.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:27:09.734 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:27:09.734 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 10 librados: watch_flush enter 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 10 librados: watch_flush exit 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 20 client.35603.objecter shutdown clearing up homeless session... 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 10 client.35603.objecter successfully canceled tick 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9803a320 msgr2=0x7f1a9803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1a9803a320 0x7f1a9803c7e0 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 crypto rx=0x7f1aac121be0 tx=0x7f1aa400b040 comp rx=0 tx=0).stop 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 10 monclient: shutdown 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 msgr2=0x7f1aac1369d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 0x7f1aac1369d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009790 tx=0x7f1aa0004750 comp rx=0 tx=0).stop 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1aabfff700 1 -- 172.21.15.47:0/664963318 reap_dead start 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 shutdown_connections 2024-04-20T02:27:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 --2- 172.21.15.47:0/664963318 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1aac0a50f0 0x7f1aac1369d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:27:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 >> 172.21.15.47:0/664963318 conn(0x7f1aac0a8510 msgr2=0x7f1aac070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:27:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 shutdown_connections 2024-04-20T02:27:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 -- 172.21.15.47:0/664963318 wait complete. 2024-04-20T02:27:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:27:09.691+0000 7f1ab1bb3700 1 librados: shutdown 2024-04-20T02:27:09.737 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-04-20T02:27:09.737 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:27:42.124 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T02:27:42.123+0000 7f54e6d8e700 -1 snap_mapper.add_oid found existing snaps mapped on 61:5d021726:test-rados-api-smithi179-46984-97::foo:32, removing 2024-04-20T02:27:42.124 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T02:27:42.123+0000 7ffb1873d700 -1 snap_mapper.add_oid found existing snaps mapped on 61:5d021726:test-rados-api-smithi179-46984-97::foo:32, removing 2024-04-20T02:28:39.738 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-20T02:28:39.738 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-20T02:28:39.738 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T02:28:39.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T02:28:39.742 INFO:tasks.ceph.mon.c:Started 2024-04-20T02:28:39.742 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:28:39.742 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-04-20T02:28:39.742 INFO:tasks.ceph.mon.i:Restarting daemon 2024-04-20T02:28:39.742 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-04-20T02:28:39.745 INFO:tasks.ceph.mon.i:Started 2024-04-20T02:28:39.745 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-04-20T02:28:39.745 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-04-20T02:28:39.745 INFO:tasks.ceph.mon.g:Restarting daemon 2024-04-20T02:28:39.746 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-04-20T02:28:39.748 INFO:tasks.ceph.mon.g:Started 2024-04-20T02:28:39.748 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-04-20T02:28:39.748 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-04-20T02:28:39.748 INFO:tasks.ceph.mon.h:Restarting daemon 2024-04-20T02:28:39.748 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-04-20T02:28:39.750 INFO:tasks.ceph.mon.h:Started 2024-04-20T02:28:39.750 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:28:39.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:28:39.876 INFO:tasks.ceph.mon.h.smithi179.stderr:2024-04-20T02:28:39.877+0000 7f9da4796cc0 -1 mon.h@-1(???).log v0 log_external_backlog rewinding external_log_to from 447 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:28:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.894+0000 7f73390c4700 10 monclient: get_monmap_and_config 2024-04-20T02:28:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.894+0000 7f73390c4700 10 monclient: build_initial_monmap 2024-04-20T02:28:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.894+0000 7f73390c4700 10 monclient: monmap: 2024-04-20T02:28:39.894 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:39.895215+0000 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:39.895215+0000 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:39.895 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:39.896 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:39.896 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.894+0000 7f73390c4700 1 Processor -- start 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.894+0000 7f73390c4700 1 -- start start 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.894+0000 7f73390c4700 10 monclient: init 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient: _add_conns ranks=[7,0,4,1,6,2,3,10,5,8,9] 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 0x7f73340a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): picked mon.noname-i con 0x7f73340a0ad0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:39.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f73340a2800 addr v1:172.21.15.47:6789/0 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 0x7f733413fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): picked mon.noname-c con 0x7f73340a3390 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73340751e0 con 0x7f73340a2800 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f733409ee40 con 0x7f73340a3390 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f733409ddb0 con 0x7f73340a0ad0 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f73390c4700 10 monclient(hunting): authenticate will time out at 2399.296414s 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733259c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f73340a2800 0x7f73340a2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:42164/0 (socket says 172.21.15.47:42164) 2024-04-20T02:28:39.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733259c700 1 -- 172.21.15.47:0/4247205222 learned_addr learned my addr 172.21.15.47:0/4247205222 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 0x7f733413fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 0x7f73340a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 10 monclient(hunting): get_auth_request con 0x7f73340a3390 auth_method 0 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 10 monclient(hunting): get_auth_request con 0x7f73340a0ad0 auth_method 0 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 msgr2=0x7f733413fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 msgr2=0x7f733413fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:39.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 0x7f733413fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 msgr2=0x7f73340a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 msgr2=0x7f73340a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 0x7f73340a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7332d9d700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 0x7f73340a0eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 0x7f733413fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250043956 0 0) 0x7f73340751e0 con 0x7f73340a2800 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7331d9b700 10 monclient(hunting): _negotiate 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7331d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7331d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7331d9b700 10 monclient(hunting): my global_id is 36252 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.895+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73200035b0 con 0x7f73340a2800 2024-04-20T02:28:39.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2600906734 0 0) 0x7f73200035b0 con 0x7f73340a2800 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 msgr2=0x7f73340a0eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f73340a0ad0 0x7f73340a0eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 msgr2=0x7f733413fa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f73340a3390 0x7f733413fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340673a0 con 0x7f73340a2800 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: _finish_auth 0 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:39.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f73240018e0 con 0x7f73340a2800 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: dump: 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:39.915 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 >> v1:172.21.15.47:6789/0 conn(0x7f73340a2800 legacy=0x7f73340a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient: _add_conns ranks=[5,3,8,6,2,7,1,0,4] 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 0x7f7320004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): picked mon.h con 0x7f7320004250 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320004e60 0x7f73200052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): picked mon.g con 0x7f7320004e60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7320009c40 0x7f732000c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): picked mon.e con 0x7f7320009c40 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733409ee40 con 0x7f7320004e60 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f733409ddb0 con 0x7f7320004250 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f732000c9e0 con 0x7f7320009c40 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7331d9b700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7320009c40 0x7f732000c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): get_auth_request con 0x7f7320009c40 auth_method 0 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:39.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320004e60 0x7f73200052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 0x7f7320004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): get_auth_request con 0x7f7320004e60 auth_method 0 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 10 monclient(hunting): get_auth_request con 0x7f7320004250 auth_method 0 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:39.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733259c700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 msgr2=0x7f7320004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 msgr2=0x7f7320004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient(hunting): handle_auth_done global_id 36252 payload 293 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 0x7f7320004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 msgr2=0x7f7320004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f7332d9d700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 0x7f7320004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7320004250 0x7f7320004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320004e60 msgr2=0x7f73200052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320004e60 0x7f73200052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient: _finish_hunting 0 2024-04-20T02:28:39.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 monclient: found mon.e 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 -- 172.21.15.47:0/4247205222 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200035b0 con 0x7f7320009c40 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient: _finish_auth 0 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 10 monclient: _check_auth_tickets 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f73390c4700 5 monclient: authenticate success, global_id 36252 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f73390c4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 -- 172.21.15.47:0/4247205222 reap_dead start 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.896+0000 7f733359e700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7320009c40 0x7f732000c0d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f732c009610 tx=0x7f7320005880 comp rx=0 tx=0).ready entity=mon.8 client_cookie=574beb9b1583cbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f732c007bf0 con 0x7f7320009c40 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: dump: 2024-04-20T02:28:39.920 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:39.921 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c007d50 con 0x7f7320009c40 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 1 -- 172.21.15.47:0/4247205222 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f732c013c80 con 0x7f7320009c40 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 4 set_mon_vals no callback set 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f7331d9b700 10 monclient: dump: 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.922 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 10 monclient: shutdown 2024-04-20T02:28:39.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 -- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7320009c40 msgr2=0x7f732000c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7320009c40 0x7f732000c0d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f732c009610 tx=0x7f7320005880 comp rx=0 tx=0).stop 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 -- 172.21.15.47:0/4247205222 shutdown_connections 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 --2- 172.21.15.47:0/4247205222 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7320009c40 0x7f732000c0d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 -- 172.21.15.47:0/4247205222 >> 172.21.15.47:0/4247205222 conn(0x7f73340a7420 msgr2=0x7f73340a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 -- 172.21.15.47:0/4247205222 shutdown_connections 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.897+0000 7f73390c4700 1 -- 172.21.15.47:0/4247205222 wait complete. 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.898+0000 7f73390c4700 10 monclient: build_initial_monmap 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.898+0000 7f73390c4700 10 monclient: monmap: 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:39.924 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.898+0000 7f73390c4700 1 librados: starting msgr at 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.898+0000 7f73390c4700 1 librados: starting objecter 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 Processor -- start 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 -- start start 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 librados: setting wanted keys 2024-04-20T02:28:39.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 librados: calling monclient init 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient: init 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient: _add_conns ranks=[3,8,5,4,0,2,1,6,7] 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 0x7f73341cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): picked mon.noname-g con 0x7f7320009c40 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 0x7f73341bcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): picked mon.noname-e con 0x7f73341a9090 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 0x7f73341c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): picked mon.noname-h con 0x7f73341bd1d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73341ccfa0 con 0x7f7320009c40 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73341aa0b0 con 0x7f73341bd1d0 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7334072130 con 0x7f73341a9090 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.899+0000 7f73390c4700 10 monclient(hunting): authenticate will time out at 2399.300939s 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 0x7f73341bcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 0x7f73341bcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55418/0 (socket says 172.21.15.47:55418) 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 -- 172.21.15.47:0/2682587555 learned_addr learned my addr 172.21.15.47:0/2682587555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): get_auth_request con 0x7f73341a9090 auth_method 0 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 0x7f73341c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 0x7f73341cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 10 monclient(hunting): get_auth_request con 0x7f7320009c40 auth_method 0 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 10 monclient(hunting): get_auth_request con 0x7f73341bd1d0 auth_method 0 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 msgr2=0x7f73341cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 msgr2=0x7f73341cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 0x7f73341cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:39.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f7332d9d700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 0x7f73341cc540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 msgr2=0x7f73341c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 msgr2=0x7f73341c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 0x7f73341c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733359e700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 0x7f73341c9040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient(hunting): handle_auth_done global_id 35630 payload 824 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 msgr2=0x7f73341c9040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 0x7f73341c9040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 msgr2=0x7f73341cc540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 0x7f73341cc540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient: _finish_hunting 0 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 monclient: found mon.noname-e 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:39.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 -- 172.21.15.47:0/2682587555 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733409e510 con 0x7f73341a9090 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733259c700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 0x7f73341bcc90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7324007340 tx=0x7f7324007650 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 client.?.objecter ms_handle_connect 0x7f73341a9090 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f732400fb30 con 0x7f73341a9090 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: dump: 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:39.930 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:28:39.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732400fcb0 con 0x7f73341a9090 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 5 monclient: authenticate success, global_id 35630 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 10 monclient: _renew_subs 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f732400fe20 con 0x7f73341a9090 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73341a9dd0 con 0x7f73341a9090 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f733159a700 4 set_mon_vals no callback set 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f732b7fe700 10 monclient: dump: 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:39.932 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 10 client.35630.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 10 monclient: _renew_subs 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:39.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73341c9580 con 0x7f73341a9090 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.900+0000 7f73390c4700 1 librados: init done 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.901+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f732400f6c0 con 0x7f73341a9090 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.901+0000 7f73390c4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.901+0000 7f73390c4700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.901+0000 7f73390c4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.901+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73340008d0 con 0x7f73341a9090 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f731403a2f0 0x7f731403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(496..496 src has 266..496) v4 ==== 7358+0+0 (secure 0 0 0) 0x7f732404acd0 con 0x7f73341a9090 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 10 client.35630.objecter ms_dispatch 0x7f7334001550 osd_map(496..496 src has 266..496) v4 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 3 client.35630.objecter handle_osd_map got epochs [496,496] > 0 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 3 client.35630.objecter handle_osd_map decoding full epoch 496 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 20 client.35630.objecter dump_active .. 0 homeless 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f7332d9d700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f731403a2f0 0x7f731403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:39.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f7332d9d700 10 monclient: get_auth_request con 0x7f731403a2f0 auth_method 0 2024-04-20T02:28:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f732b7fe700 10 client.35630.objecter ms_handle_connect 0x7f731403a2f0 2024-04-20T02:28:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.902+0000 7f7332d9d700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f731403a2f0 0x7f731403c7b0 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f731c009100 tx=0x7f731c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.907+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f732400f970 con 0x7f73341a9090 2024-04-20T02:28:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.907+0000 7f732b7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:39.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:39.907+0000 7f732b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:40.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f73390c4700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:28:40.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f73390c4700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:40.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f73390c4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:40.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f73340666c0 con 0x7f73341a9090 2024-04-20T02:28:40.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f732b7fe700 1 -- 172.21.15.47:0/2682587555 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3152 (secure 0 0 0) 0x7f732401b360 con 0x7f73341a9090 2024-04-20T02:28:40.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f732b7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:40.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.220+0000 7f732b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:40.221 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":78,"quorum":[0,1,2,6,8],"quorum_names":["a","f","b","d","e"],"quorum_leader_name":"b","quorum_age":94,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 10 librados: watch_flush enter 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 10 librados: watch_flush exit 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 20 client.35630.objecter shutdown clearing up homeless session... 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 10 client.35630.objecter successfully canceled tick 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f731403a2f0 msgr2=0x7f731403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f731403a2f0 0x7f731403c7b0 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 crypto rx=0x7f731c009100 tx=0x7f731c008040 comp rx=0 tx=0).stop 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 10 monclient: shutdown 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 msgr2=0x7f73341bcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 0x7f73341bcc90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7324007340 tx=0x7f7324007650 comp rx=0 tx=0).stop 2024-04-20T02:28:40.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 shutdown_connections 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f731403a2f0 0x7f731403c7b0 unknown :-1 s=CLOSED pgs=3281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73341bd1d0 0x7f73341c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7320009c40 0x7f73341cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 --2- 172.21.15.47:0/2682587555 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f73341a9090 0x7f73341bcc90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 >> 172.21.15.47:0/2682587555 conn(0x7f73340a7420 msgr2=0x7f73341bfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 shutdown_connections 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 -- 172.21.15.47:0/2682587555 wait complete. 2024-04-20T02:28:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:40.223+0000 7f73390c4700 1 librados: shutdown 2024-04-20T02:28:43.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:28:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.381+0000 7f3a55f80700 10 monclient: get_monmap_and_config 2024-04-20T02:28:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.381+0000 7f3a55f80700 10 monclient: build_initial_monmap 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.381+0000 7f3a55f80700 10 monclient: monmap: 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:43.382348+0000 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:43.382348+0000 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:43.383 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.381+0000 7f3a55f80700 1 Processor -- start 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 1 -- start start 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient: init 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient: _add_conns ranks=[4,2,6,5,10,1,8,9,7,0,3] 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient(hunting): picked mon.noname-c con 0x7f3a500a0950 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a500a0160 0x7f3a500a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient(hunting): picked mon.noname-b con 0x7f3a500a0160 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3a500a2e90 0x7f3a500a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient(hunting): picked mon.noname-d con 0x7f3a500a2e90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.382+0000 7f3a55f80700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50072830 con 0x7f3a500a0160 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a55f80700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a55f80700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5009e650 con 0x7f3a500a0950 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a55f80700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a55f80700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5009eee0 con 0x7f3a500a2e90 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a55f80700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a55f80700 10 monclient(hunting): authenticate will time out at 2402.784029s 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48314/0 (socket says 172.21.15.47:48314) 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 -- 172.21.15.47:0/1688934399 learned_addr learned my addr 172.21.15.47:0/1688934399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 10 monclient(hunting): get_auth_request con 0x7f3a500a0950 auth_method 0 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3a500a2e90 0x7f3a500a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): get_auth_request con 0x7f3a500a2e90 auth_method 0 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a500a0160 0x7f3a500a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 -- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 msgr2=0x7f3a500a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 -- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 msgr2=0x7f3a500a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): get_auth_request con 0x7f3a500a0160 auth_method 0 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4f7fe700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient(hunting): handle_auth_done global_id 54798 payload 274 2024-04-20T02:28:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 -- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 msgr2=0x7f3a500a0d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 -- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a500a0160 msgr2=0x7f3a500a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a500a0160 0x7f3a500a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 monclient: found mon.noname-d 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 -- 172.21.15.47:0/1688934399 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500751e0 con 0x7f3a500a2e90 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3a500a2e90 0x7f3a500a32f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009df0 tx=0x7f3a4800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=da24101802ac5f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4e7fc700 1 -- 172.21.15.47:0/1688934399 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a4800e070 con 0x7f3a500a2e90 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.383+0000 7f3a4e7fc700 10 monclient: dump: 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 1 -- 172.21.15.47:0/1688934399 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a480098f0 con 0x7f3a500a2e90 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 5 monclient: authenticate success, global_id 54798 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 1 -- 172.21.15.47:0/1688934399 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a480128a0 con 0x7f3a500a2e90 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 4 set_mon_vals no callback set 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a4e7fc700 10 monclient: dump: 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 10 monclient: shutdown 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 -- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3a500a2e90 msgr2=0x7f3a500a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3a500a2e90 0x7f3a500a32f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009df0 tx=0x7f3a4800b330 comp rx=0 tx=0).stop 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 -- 172.21.15.47:0/1688934399 shutdown_connections 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3a500a0950 0x7f3a500a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3a500a2e90 0x7f3a500a32f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 --2- 172.21.15.47:0/1688934399 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3a500a0160 0x7f3a500a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 -- 172.21.15.47:0/1688934399 >> 172.21.15.47:0/1688934399 conn(0x7f3a500a7570 msgr2=0x7f3a500a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 -- 172.21.15.47:0/1688934399 shutdown_connections 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.384+0000 7f3a55f80700 1 -- 172.21.15.47:0/1688934399 wait complete. 2024-04-20T02:28:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 10 monclient: build_initial_monmap 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 10 monclient: monmap: 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:43.403 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 1 librados: starting msgr at 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 1 librados: starting objecter 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 1 Processor -- start 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 1 -- start start 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 1 librados: setting wanted keys 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 1 librados: calling monclient init 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.385+0000 7f3a55f80700 10 monclient: init 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient: _add_conns ranks=[1,8,3,4,0,2,5,6,7] 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500a0160 0x7f3a501cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): picked mon.noname-f con 0x7f3a500a0160 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:28:43.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 0x7f3a501babe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): picked mon.noname-e con 0x7f3a500a0950 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 0x7f3a501bb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): picked mon.noname-g con 0x7f3a500a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501bb660 con 0x7f3a500a0160 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5009e890 con 0x7f3a500a2e90 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5009e440 con 0x7f3a500a0950 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a55f80700 10 monclient(hunting): authenticate will time out at 2402.787374s 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 0x7f3a501babe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 0x7f3a501babe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:47960/0 (socket says 172.21.15.47:47960) 2024-04-20T02:28:43.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 -- 172.21.15.47:0/357806387 learned_addr learned my addr 172.21.15.47:0/357806387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): get_auth_request con 0x7f3a500a0950 auth_method 0 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4f7fe700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500a0160 0x7f3a501cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 0x7f3a501bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4f7fe700 10 monclient(hunting): get_auth_request con 0x7f3a500a0160 auth_method 0 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:43.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 10 monclient(hunting): get_auth_request con 0x7f3a500a2e90 auth_method 0 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient(hunting): handle_auth_done global_id 35639 payload 824 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 -- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 msgr2=0x7f3a501bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 1 -- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 msgr2=0x7f3a501bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 1 -- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 msgr2=0x7f3a501bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 0x7f3a501bb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4ffff700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 0x7f3a501bb120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 0x7f3a501bb120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 -- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500a0160 msgr2=0x7f3a501cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500a0160 0x7f3a501cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient: _finish_hunting 0 2024-04-20T02:28:43.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 monclient: found mon.noname-e 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4effd700 1 -- 172.21.15.47:0/357806387 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a48009a70 con 0x7f3a500a0950 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.386+0000 7f3a4f7fe700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500a0160 0x7f3a501cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a4effd700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 0x7f3a501babe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400e9b0 tx=0x7f3a4400ed70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 client.?.objecter ms_handle_connect 0x7f3a500a0950 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a4400cc60 con 0x7f3a500a0950 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: dump: 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.408 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: _finish_auth 0 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: _check_auth_tickets 2024-04-20T02:28:43.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4400ce20 con 0x7f3a500a0950 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 5 monclient: authenticate success, global_id 35639 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 10 monclient: _renew_subs 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a501cc940 con 0x7f3a500a0950 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a440138e0 con 0x7f3a500a0950 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 10 client.35639.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a37fff700 10 monclient: dump: 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:43.410 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:43.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a4dffb700 4 set_mon_vals no callback set 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 10 monclient: _renew_subs 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a501a9af0 con 0x7f3a500a0950 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.387+0000 7f3a55f80700 1 librados: init done 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3a44013470 con 0x7f3a500a0950 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a55f80700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a55f80700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a55f80700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a501a7fe0 con 0x7f3a500a0950 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3a3c03a2f0 0x7f3a3c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(500..500 src has 266..500) v4 ==== 7466+0+0 (secure 0 0 0) 0x7f3a4404ae80 con 0x7f3a500a0950 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 10 client.35639.objecter ms_dispatch 0x7f3a50001550 osd_map(500..500 src has 266..500) v4 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 3 client.35639.objecter handle_osd_map got epochs [500,500] > 0 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 3 client.35639.objecter handle_osd_map decoding full epoch 500 2024-04-20T02:28:43.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a37fff700 20 client.35639.objecter dump_active .. 0 homeless 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a4f7fe700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3a3c03a2f0 0x7f3a3c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.388+0000 7f3a4f7fe700 10 monclient: get_auth_request con 0x7f3a3c03a2f0 auth_method 0 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.389+0000 7f3a37fff700 10 client.35639.objecter ms_handle_connect 0x7f3a3c03a2f0 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.389+0000 7f3a4f7fe700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3a3c03a2f0 0x7f3a3c03c7b0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009180 tx=0x7f3a40008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.393+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f3a44013720 con 0x7f3a500a0950 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.393+0000 7f3a37fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:43.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.393+0000 7f3a37fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.695+0000 7f3a55f80700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.695+0000 7f3a55f80700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.695+0000 7f3a55f80700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.695+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3a50066730 con 0x7f3a500a0950 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.696+0000 7f3a37fff700 1 -- 172.21.15.47:0/357806387 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3152 (secure 0 0 0) 0x7f3a44017070 con 0x7f3a500a0950 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.696+0000 7f3a37fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.696+0000 7f3a37fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:43.697 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":78,"quorum":[0,1,2,6,8],"quorum_names":["a","f","b","d","e"],"quorum_leader_name":"b","quorum_age":98,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:28:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 10 librados: watch_flush enter 2024-04-20T02:28:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 10 librados: watch_flush exit 2024-04-20T02:28:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 20 client.35639.objecter shutdown clearing up homeless session... 2024-04-20T02:28:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 10 client.35639.objecter successfully canceled tick 2024-04-20T02:28:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3a3c03a2f0 msgr2=0x7f3a3c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:43.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3a3c03a2f0 0x7f3a3c03c7b0 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009180 tx=0x7f3a40008040 comp rx=0 tx=0).stop 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 10 monclient: shutdown 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 msgr2=0x7f3a501babe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 0x7f3a501babe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400e9b0 tx=0x7f3a4400ed70 comp rx=0 tx=0).stop 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 shutdown_connections 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a500a2e90 0x7f3a501bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3a3c03a2f0 0x7f3a3c03c7b0 unknown :-1 s=CLOSED pgs=3283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a500a0160 0x7f3a501cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 --2- 172.21.15.47:0/357806387 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3a500a0950 0x7f3a501babe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 >> 172.21.15.47:0/357806387 conn(0x7f3a500a7570 msgr2=0x7f3a500a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 shutdown_connections 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 -- 172.21.15.47:0/357806387 wait complete. 2024-04-20T02:28:43.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:43.698+0000 7f3a55f80700 1 librados: shutdown 2024-04-20T02:28:46.713 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:28:46.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient: get_monmap_and_config 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient: build_initial_monmap 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient: monmap: 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:46.857615+0000 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:46.857615+0000 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:46.857 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:46.858 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 Processor -- start 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 -- start start 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient: init 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient: _add_conns ranks=[8,7,0,10,5,2,3,4,1,6,9] 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): picked mon.noname-e con 0x7f95500a5080 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:28:46.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): picked mon.noname-i con 0x7f95500a1a80 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f95500a3dc0 addr v1:172.21.15.47:6789/0 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500751e0 con 0x7f95500a3dc0 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95500a5460 con 0x7f95500a1a80 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9550070b10 con 0x7f95500a5080 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f9555c75700 10 monclient(hunting): authenticate will time out at 2406.258851s 2024-04-20T02:28:46.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.857+0000 7f954ffff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f95500a3dc0 0x7f955013ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45648/0 (socket says 172.21.15.47:45648) 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 learned_addr learned my addr 172.21.15.47:0/2493474266 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f95500a5080 auth_method 0 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f95500a1a80 auth_method 0 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:46.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:46.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:46.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:46.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:46.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:46.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:46.858+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f95500a5080 auth_method 0 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:28:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f95500a1a80 auth_method 0 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:47.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.059+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:28:47.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:47.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f95500a5080 auth_method 0 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f95500a1a80 auth_method 0 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:47.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:47.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:47.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:47.460+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:28:48.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f95500a5080 auth_method 0 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:48.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.261+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f95500a1a80 auth_method 0 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:48.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:48.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:48.262+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:28:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.858+0000 7f954d7fa700 10 monclient(hunting): tick 2024-04-20T02:28:49.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.858+0000 7f954d7fa700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.858+0000 7f954d7fa700 1 monclient(hunting): continuing hunt 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.858+0000 7f954d7fa700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.858+0000 7f954d7fa700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 msgr2=0x7f95500a1540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.858+0000 7f954d7fa700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f95500a5080 0x7f95500a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 msgr2=0x7f95500a3880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f95500a1a80 0x7f95500a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 -- 172.21.15.47:0/2493474266 >> v1:172.21.15.47:6789/0 conn(0x7f95500a3dc0 legacy=0x7f955013ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient: _add_conns ranks=[3,4,6,10,1,2,5,9] 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): picked mon.noname-g con 0x7f953c0038d0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): picked mon.noname-c con 0x7f953c006330 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:49.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): picked mon.noname-d con 0x7f953c008eb0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 -- 172.21.15.47:0/2493474266 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500a5460 con 0x7f953c0038d0 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 -- 172.21.15.47:0/2493474266 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9550070b10 con 0x7f953c006330 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 1 -- 172.21.15.47:0/2493474266 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f953c00bae0 con 0x7f953c008eb0 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954d7fa700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f953c006330 auth_method 0 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 10 monclient(hunting): get_auth_request con 0x7f953c008eb0 auth_method 0 2024-04-20T02:28:49.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f953c0038d0 auth_method 0 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:49.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:49.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:49.859+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 10 monclient(hunting): get_auth_request con 0x7f953c008eb0 auth_method 0 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f953c006330 auth_method 0 2024-04-20T02:28:50.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:28:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f953c0038d0 auth_method 0 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:50.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.060+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:28:50.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:50.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:50.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 10 monclient(hunting): get_auth_request con 0x7f953c008eb0 auth_method 0 2024-04-20T02:28:50.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f953c006330 auth_method 0 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:50.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f953c0038d0 auth_method 0 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:50.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:50.461+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:28:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): get_auth_request con 0x7f953c008eb0 auth_method 0 2024-04-20T02:28:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:51.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f953c006330 auth_method 0 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f953c0038d0 auth_method 0 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:51.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954effd700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient(hunting): handle_auth_done global_id 54807 payload 274 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 msgr2=0x7f953c0087c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f953c006330 0x7f953c0087c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 msgr2=0x7f953c005ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:28:51.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f953c0038d0 0x7f953c005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 monclient: found mon.noname-d 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550136370 con 0x7f953c008eb0 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.262+0000 7f954ffff700 1 -- 172.21.15.47:0/2493474266 reap_dead start 2024-04-20T02:28:51.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.263+0000 7f954ffff700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f95480020a0 tx=0x7f9548007330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=cdf38ff0ab1310c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 1 -- 172.21.15.47:0/2493474266 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954801a070 con 0x7f953c008eb0 2024-04-20T02:28:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: dump: 2024-04-20T02:28:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:51.276 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:51.277 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:51.278 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:28:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:28:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 1 -- 172.21.15.47:0/2493474266 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9548007940 con 0x7f953c008eb0 2024-04-20T02:28:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f9555c75700 5 monclient: authenticate success, global_id 54807 2024-04-20T02:28:51.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f9555c75700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f9555c75700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f9555c75700 4 set_mon_vals no callback set 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 1 -- 172.21.15.47:0/2493474266 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9548007af0 con 0x7f953c008eb0 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:51.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.264+0000 7f954e7fc700 10 monclient: dump: 2024-04-20T02:28:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:51.281 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:51.282 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 10 monclient: shutdown 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 -- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f953c00b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:51.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f95480020a0 tx=0x7f9548007330 comp rx=0 tx=0).stop 2024-04-20T02:28:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 -- 172.21.15.47:0/2493474266 shutdown_connections 2024-04-20T02:28:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 --2- 172.21.15.47:0/2493474266 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f953c00b320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 -- 172.21.15.47:0/2493474266 >> 172.21.15.47:0/2493474266 conn(0x7f95500a84a0 msgr2=0x7f95500a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 -- 172.21.15.47:0/2493474266 shutdown_connections 2024-04-20T02:28:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.265+0000 7f9555c75700 1 -- 172.21.15.47:0/2493474266 wait complete. 2024-04-20T02:28:51.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 10 monclient: build_initial_monmap 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 10 monclient: monmap: 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:51.285 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:51.286 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 1 librados: starting msgr at 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 1 librados: starting objecter 2024-04-20T02:28:51.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 1 Processor -- start 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 1 -- start start 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 1 librados: setting wanted keys 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 1 librados: calling monclient init 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.266+0000 7f9555c75700 10 monclient: init 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient: _add_conns ranks=[6,1,0,2,4,3,8,7,5] 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f95501c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:51.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): picked mon.noname-d con 0x7f953c008eb0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501c8da0 0x7f95501c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): picked mon.noname-f con 0x7f95501c8da0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955011e910 0x7f955011ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): picked mon.noname-a con 0x7f955011e910 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95501223b0 con 0x7f955011e910 2024-04-20T02:28:51.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9550122c90 con 0x7f95501c8da0 2024-04-20T02:28:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95501352c0 con 0x7f953c008eb0 2024-04-20T02:28:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:51.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f9555c75700 10 monclient(hunting): authenticate will time out at 2410.667434s 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f95501c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f95501c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:34268/0 (socket says 172.21.15.47:34268) 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 1 -- 172.21.15.47:0/2413959396 learned_addr learned my addr 172.21.15.47:0/2413959396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955011e910 0x7f955011ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): get_auth_request con 0x7f953c008eb0 auth_method 0 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:51.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): get_auth_request con 0x7f955011e910 auth_method 0 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501c8da0 0x7f95501c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): get_auth_request con 0x7f95501c8da0 auth_method 0 2024-04-20T02:28:51.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:51.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 10 monclient(hunting): handle_auth_done global_id 54816 payload 824 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 1 -- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501c8da0 msgr2=0x7f95501c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.267+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501c8da0 0x7f95501c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 1 -- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955011e910 msgr2=0x7f955011ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955011e910 0x7f955011ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 1 monclient: found mon.noname-d 2024-04-20T02:28:51.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 1 -- 172.21.15.47:0/2413959396 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954800f040 con 0x7f953c008eb0 2024-04-20T02:28:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954ffff700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955011e910 0x7f955011ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954f7fe700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f95501c8860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9540016790 tx=0x7f9540016aa0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 client.?.objecter ms_handle_connect 0x7f953c008eb0 2024-04-20T02:28:51.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f954001e070 con 0x7f953c008eb0 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: dump: 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.297 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:51.298 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: _finish_auth 0 2024-04-20T02:28:51.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: _check_auth_tickets 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000c040 con 0x7f953c008eb0 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 5 monclient: authenticate success, global_id 54816 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 10 monclient: _renew_subs 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95501354f0 con 0x7f953c008eb0 2024-04-20T02:28:51.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95400173e0 con 0x7f953c008eb0 2024-04-20T02:28:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9537fff700 10 monclient: dump: 2024-04-20T02:28:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:51.301 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:51.302 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:51.303 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f954dffb700 4 set_mon_vals no callback set 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 10 client.54816.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 10 monclient: _renew_subs 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95501c96c0 con 0x7f953c008eb0 2024-04-20T02:28:51.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.268+0000 7f9555c75700 1 librados: init done 2024-04-20T02:28:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9540015690 con 0x7f953c008eb0 2024-04-20T02:28:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9555c75700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9555c75700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9555c75700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500a2240 con 0x7f953c008eb0 2024-04-20T02:28:51.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f953c03b680 0x7f953c03db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(503..503 src has 266..503) v4 ==== 6294+0+0 (secure 0 0 0) 0x7f954004afd0 con 0x7f953c008eb0 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 10 client.54816.objecter ms_dispatch 0x7f9550001550 osd_map(503..503 src has 266..503) v4 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 3 client.54816.objecter handle_osd_map got epochs [503,503] > 0 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 3 client.54816.objecter handle_osd_map decoding full epoch 503 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.269+0000 7f9537fff700 20 client.54816.objecter dump_active .. 0 homeless 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.270+0000 7f954effd700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f953c03b680 0x7f953c03db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:51.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.270+0000 7f954effd700 10 monclient: get_auth_request con 0x7f953c03b680 auth_method 0 2024-04-20T02:28:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.270+0000 7f954effd700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f953c03b680 0x7f953c03db40 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7f95500a8ef0 tx=0x7f9544008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.270+0000 7f9537fff700 10 client.54816.objecter ms_handle_connect 0x7f953c03b680 2024-04-20T02:28:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.275+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f954007e7f0 con 0x7f953c008eb0 2024-04-20T02:28:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.275+0000 7f9537fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:51.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.275+0000 7f9537fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.588+0000 7f9555c75700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:28:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.589+0000 7f9555c75700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.589+0000 7f9555c75700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.589+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f955006d8b0 con 0x7f953c008eb0 2024-04-20T02:28:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.589+0000 7f9537fff700 1 -- 172.21.15.47:0/2413959396 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3157 (secure 0 0 0) 0x7f9540027080 con 0x7f953c008eb0 2024-04-20T02:28:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.589+0000 7f9537fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.589+0000 7f9537fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:51.589 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":80,"quorum":[0,1,2,6,7,8],"quorum_names":["a","f","b","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:28:51.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 10 librados: watch_flush enter 2024-04-20T02:28:51.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 10 librados: watch_flush exit 2024-04-20T02:28:51.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 20 client.54816.objecter shutdown clearing up homeless session... 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 10 client.54816.objecter successfully canceled tick 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f953c03b680 msgr2=0x7f953c03db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f953c03b680 0x7f953c03db40 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 crypto rx=0x7f95500a8ef0 tx=0x7f9544008040 comp rx=0 tx=0).stop 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 10 monclient: shutdown 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.591+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 msgr2=0x7f95501c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f95501c8860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9540016790 tx=0x7f9540016aa0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 shutdown_connections 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f955011e910 0x7f955011ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f953c008eb0 0x7f95501c8860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f953c03b680 0x7f953c03db40 unknown :-1 s=CLOSED pgs=3284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 --2- 172.21.15.47:0/2413959396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95501c8da0 0x7f95501c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:51.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 >> 172.21.15.47:0/2413959396 conn(0x7f95500a84a0 msgr2=0x7f95500a63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:51.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 shutdown_connections 2024-04-20T02:28:51.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 -- 172.21.15.47:0/2413959396 wait complete. 2024-04-20T02:28:51.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:51.592+0000 7f9555c75700 1 librados: shutdown 2024-04-20T02:28:54.605 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.748+0000 7fc837598700 10 monclient: get_monmap_and_config 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.748+0000 7fc837598700 10 monclient: build_initial_monmap 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.748+0000 7fc837598700 10 monclient: monmap: 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:54.749718+0000 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:54.749718+0000 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:54.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:54.750 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 1 Processor -- start 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 1 -- start start 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 10 monclient: init 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 10 monclient: _add_conns ranks=[6,2,5,4,3,9,1,7,8,0,10] 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 0x7fc8300a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 10 monclient(hunting): picked mon.noname-d con 0x7fc8300a5080 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.749+0000 7fc837598700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc8300a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): picked mon.noname-b con 0x7fc8300a1a80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc8300a3dc0 0x7fc8300a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): picked mon.noname-h con 0x7fc8300a3dc0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc830072380 con 0x7fc8300a1a80 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300a5460 con 0x7fc8300a3dc0 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc830070b10 con 0x7fc8300a5080 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient(hunting): authenticate will time out at 2414.151062s 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc8300a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 0x7fc8300a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc8300a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53982/0 (socket says 172.21.15.47:53982) 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 0x7fc8300a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38714/0 (socket says 172.21.15.47:38714) 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 -- 172.21.15.47:0/3473272758 learned_addr learned my addr 172.21.15.47:0/3473272758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): get_auth_request con 0x7fc8300a1a80 auth_method 0 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): get_auth_request con 0x7fc8300a5080 auth_method 0 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:54.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835b35700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc8300a3dc0 0x7fc8300a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient(hunting): handle_auth_done global_id 24455 payload 274 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 -- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 msgr2=0x7fc8300a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 0x7fc8300a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 -- 172.21.15.47:0/3473272758 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc8300a3dc0 msgr2=0x7fc8300a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc8300a3dc0 0x7fc8300a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient: _finish_hunting 0 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 monclient: found mon.noname-b 2024-04-20T02:28:54.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 -- 172.21.15.47:0/3473272758 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300751e0 con 0x7fc8300a1a80 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835b35700 10 monclient: get_auth_request con 0x7fc8300a3dc0 auth_method 0 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835334700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 0x7fc8300a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc835b35700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc8300a3dc0 0x7fc8300a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc834b33700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc8300a3880 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fc820009920 tx=0x7fc820009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3e73ec9f612eba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 1 -- 172.21.15.47:0/3473272758 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc820011aa0 con 0x7fc8300a1a80 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: dump: 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.764 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: _finish_auth 0 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: _check_auth_tickets 2024-04-20T02:28:54.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 5 monclient: authenticate success, global_id 24455 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 1 -- 172.21.15.47:0/3473272758 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820011c00 con 0x7fc8300a1a80 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 1 -- 172.21.15.47:0/3473272758 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc820011d70 con 0x7fc8300a1a80 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc837598700 4 set_mon_vals no callback set 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:54.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.750+0000 7fc827fff700 10 monclient: dump: 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:54.767 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 10 monclient: shutdown 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 -- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 msgr2=0x7fc8300a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc8300a3880 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fc820009920 tx=0x7fc820009c30 comp rx=0 tx=0).stop 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 -- 172.21.15.47:0/3473272758 shutdown_connections 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc8300a3dc0 0x7fc8300a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8300a5080 0x7fc8300a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 --2- 172.21.15.47:0/3473272758 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc8300a3880 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 -- 172.21.15.47:0/3473272758 >> 172.21.15.47:0/3473272758 conn(0x7fc8300a84a0 msgr2=0x7fc8300a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 -- 172.21.15.47:0/3473272758 shutdown_connections 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.751+0000 7fc837598700 1 -- 172.21.15.47:0/3473272758 wait complete. 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 10 monclient: build_initial_monmap 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 10 monclient: monmap: 2024-04-20T02:28:54.768 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:54.769 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 1 librados: starting msgr at 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 1 librados: starting objecter 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 1 Processor -- start 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 1 -- start start 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 1 librados: setting wanted keys 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 1 librados: calling monclient init 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.752+0000 7fc837598700 10 monclient: init 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient: _add_conns ranks=[2,4,3,8,6,7,1,0,5] 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc83009dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): picked mon.noname-b con 0x7fc8300a1a80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 0x7fc8301cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): picked mon.noname-c con 0x7fc8300a3dc0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 0x7fc8301cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): picked mon.noname-g con 0x7fc8300a5080 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc830071260 con 0x7fc8300a1a80 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300a21e0 con 0x7fc8300a5080 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc83009d330 con 0x7fc8300a3dc0 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc837598700 10 monclient(hunting): authenticate will time out at 2414.154368s 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc83009dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc83009dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53988/0 (socket says 172.21.15.47:53988) 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 -- 172.21.15.47:0/1865327978 learned_addr learned my addr 172.21.15.47:0/1865327978 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): get_auth_request con 0x7fc8300a1a80 auth_method 0 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:54.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 0x7fc8301cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 10 monclient(hunting): get_auth_request con 0x7fc8300a3dc0 auth_method 0 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 0x7fc8301cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 10 monclient(hunting): get_auth_request con 0x7fc8300a5080 auth_method 0 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:54.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 msgr2=0x7fc8301cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 msgr2=0x7fc8301cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 0x7fc8301cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc834b33700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 0x7fc8301cc3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient(hunting): handle_auth_done global_id 24464 payload 824 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 msgr2=0x7fc8301cc3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 0x7fc8301cc3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 msgr2=0x7fc8301cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 msgr2=0x7fc8301cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 msgr2=0x7fc8301cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 0x7fc8301cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835b35700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 0x7fc8301cc910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 0x7fc8301cc910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient: _finish_hunting 0 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 monclient: found mon.noname-b 2024-04-20T02:28:54.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 -- 172.21.15.47:0/1865327978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c0094a0 con 0x7fc8300a1a80 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.753+0000 7fc835334700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc83009dff0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00ebe0 tx=0x7fc82c009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 client.?.objecter ms_handle_connect 0x7fc8300a1a80 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc82c00cd80 con 0x7fc8300a1a80 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: dump: 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:54.774 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: _finish_auth 0 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c013470 con 0x7fc8300a1a80 2024-04-20T02:28:54.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc82c00cee0 con 0x7fc8300a1a80 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc825ffb700 10 monclient: dump: 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:54.776 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc8277fe700 4 set_mon_vals no callback set 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 5 monclient: authenticate success, global_id 24464 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 10 monclient: _renew_subs 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc820009550 con 0x7fc8300a1a80 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 10 client.24464.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 10 monclient: _renew_subs 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:54.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83009e530 con 0x7fc8300a1a80 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.754+0000 7fc837598700 1 librados: init done 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc82c00c910 con 0x7fc8300a1a80 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc81403a320 0x7fc81403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(507..507 src has 266..507) v4 ==== 5389+0+0 (secure 0 0 0) 0x7fc82c04a6a0 con 0x7fc8300a1a80 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 10 client.24464.objecter ms_dispatch 0x7fc830001550 osd_map(507..507 src has 266..507) v4 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 3 client.24464.objecter handle_osd_map got epochs [507,507] > 0 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 3 client.24464.objecter handle_osd_map decoding full epoch 507 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc825ffb700 20 client.24464.objecter dump_active .. 0 homeless 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc834b33700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc81403a320 0x7fc81403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc834b33700 10 monclient: get_auth_request con 0x7fc81403a320 auth_method 0 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc837598700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc837598700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc837598700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.755+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83009e6c0 con 0x7fc8300a1a80 2024-04-20T02:28:54.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.756+0000 7fc825ffb700 10 client.24464.objecter ms_handle_connect 0x7fc81403a320 2024-04-20T02:28:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.756+0000 7fc834b33700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc81403a320 0x7fc81403c7e0 secure :-1 s=READY pgs=3286 cs=0 l=1 rev1=1 crypto rx=0x7fc820014040 tx=0x7fc820008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.760+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc82c017080 con 0x7fc8300a1a80 2024-04-20T02:28:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.760+0000 7fc825ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:54.760+0000 7fc825ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:55.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc837598700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc837598700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc837598700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc8300a7580 con 0x7fc8300a1a80 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc825ffb700 1 -- 172.21.15.47:0/1865327978 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3157 (secure 0 0 0) 0x7fc82c00cbc0 con 0x7fc8300a1a80 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc825ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.072+0000 7fc825ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:55.073 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":80,"quorum":[0,1,2,6,7,8],"quorum_names":["a","f","b","d","i","e"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:28:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.074+0000 7fc837598700 10 librados: watch_flush enter 2024-04-20T02:28:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 10 librados: watch_flush exit 2024-04-20T02:28:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 20 client.24464.objecter shutdown clearing up homeless session... 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 10 client.24464.objecter successfully canceled tick 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc81403a320 msgr2=0x7fc81403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc81403a320 0x7fc81403c7e0 secure :-1 s=READY pgs=3286 cs=0 l=1 rev1=1 crypto rx=0x7fc820014040 tx=0x7fc820008030 comp rx=0 tx=0).stop 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 10 monclient: shutdown 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 msgr2=0x7fc83009dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc83009dff0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00ebe0 tx=0x7fc82c009ec0 comp rx=0 tx=0).stop 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 shutdown_connections 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc81403a320 0x7fc81403c7e0 unknown :-1 s=CLOSED pgs=3286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8300a5080 0x7fc8301cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8300a1a80 0x7fc83009dff0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 --2- 172.21.15.47:0/1865327978 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8300a3dc0 0x7fc8301cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 >> 172.21.15.47:0/1865327978 conn(0x7fc8300a84a0 msgr2=0x7fc8300a5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 shutdown_connections 2024-04-20T02:28:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 -- 172.21.15.47:0/1865327978 wait complete. 2024-04-20T02:28:55.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:55.075+0000 7fc837598700 1 librados: shutdown 2024-04-20T02:28:58.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.244+0000 7f0780c81700 10 monclient: get_monmap_and_config 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.244+0000 7f0780c81700 10 monclient: build_initial_monmap 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.244+0000 7f0780c81700 10 monclient: monmap: 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:58.245311+0000 2024-04-20T02:28:58.244 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:58.245311+0000 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:58.245 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.244+0000 7f0780c81700 1 Processor -- start 2024-04-20T02:28:58.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.244+0000 7f0780c81700 1 -- start start 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.244+0000 7f0780c81700 10 monclient: init 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient: _add_conns ranks=[0,1,8,7,3,2,6,9,10,4,5] 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f077c0a0950 addr v1:172.21.15.47:6789/0 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f077c0a0160 addr v1:172.21.15.179:6789/0 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f077c13e460 0x7f077c140850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient(hunting): picked mon.noname-e con 0x7f077c13e460 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0751e0 con 0x7f077c0a0950 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c072830 con 0x7f077c0a0160 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0a1440 con 0x7f077c13e460 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f0780c81700 10 monclient(hunting): authenticate will time out at 2417.646617s 2024-04-20T02:28:58.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077a59c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f077c0a0950 0x7f077c0a0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47230/0 (socket says 172.21.15.47:47230) 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077a59c700 1 -- 172.21.15.47:0/3285237669 learned_addr learned my addr 172.21.15.47:0/3285237669 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 1 --2- 172.21.15.47:0/3285237669 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f077c13e460 0x7f077c140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): get_auth_request con 0x7f077c13e460 auth_method 0 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.245+0000 7f077ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 1 -- 172.21.15.47:0/3285237669 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377784544 0 0) 0x7f077c0751e0 con 0x7f077c0a0950 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 10 monclient(hunting): handle_auth_done global_id 35648 payload 274 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 1 -- 172.21.15.47:0/3285237669 >> v1:172.21.15.179:6789/0 conn(0x7f077c0a0160 legacy=0x7f077c13cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 1 -- 172.21.15.47:0/3285237669 >> v1:172.21.15.47:6789/0 conn(0x7f077c0a0950 legacy=0x7f077c0a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:28:58.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 1 monclient: found mon.noname-e 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 1 -- 172.21.15.47:0/3285237669 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c09e650 con 0x7f077c13e460 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077ad9d700 1 --2- 172.21.15.47:0/3285237669 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f077c13e460 0x7f077c140850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0774009dc0 tx=0x7f077400b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7fc575487a19cbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 1 -- 172.21.15.47:0/3285237669 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f077400e070 con 0x7f077c13e460 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: dump: 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.257 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: _finish_auth 0 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: _check_auth_tickets 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 1 -- 172.21.15.47:0/3285237669 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774003a20 con 0x7f077c13e460 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f0780c81700 5 monclient: authenticate success, global_id 35648 2024-04-20T02:28:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f0780c81700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f0780c81700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 1 -- 172.21.15.47:0/3285237669 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0774004030 con 0x7f077c13e460 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f0780c81700 4 set_mon_vals no callback set 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.246+0000 7f077959a700 10 monclient: dump: 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.259 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 10 monclient: shutdown 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 -- 172.21.15.47:0/3285237669 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f077c13e460 msgr2=0x7f077c140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 --2- 172.21.15.47:0/3285237669 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f077c13e460 0x7f077c140850 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0774009dc0 tx=0x7f077400b330 comp rx=0 tx=0).stop 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 -- 172.21.15.47:0/3285237669 shutdown_connections 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 --2- 172.21.15.47:0/3285237669 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f077c13e460 0x7f077c140850 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 -- 172.21.15.47:0/3285237669 >> 172.21.15.47:0/3285237669 conn(0x7f077c0a7570 msgr2=0x7f077c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 -- 172.21.15.47:0/3285237669 shutdown_connections 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 1 -- 172.21.15.47:0/3285237669 wait complete. 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.247+0000 7f0780c81700 10 monclient: build_initial_monmap 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient: monmap: 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 librados: starting msgr at 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 librados: starting objecter 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 Processor -- start 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 -- start start 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 librados: setting wanted keys 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 librados: calling monclient init 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient: init 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient: _add_conns ranks=[1,6,3,0,2,5,8,7,4] 2024-04-20T02:28:58.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f077c0a0160 0x7f077c13c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): picked mon.noname-f con 0x7f077c0a0160 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 0x7f077c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): picked mon.noname-d con 0x7f077c0a0950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f077c13e460 0x7f077c070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): picked mon.noname-g con 0x7f077c13e460 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c13d900 con 0x7f077c0a0160 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c09e890 con 0x7f077c13e460 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f077c09e440 con 0x7f077c0a0950 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.248+0000 7f0780c81700 10 monclient(hunting): authenticate will time out at 2417.649898s 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 0x7f077c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 0x7f077c070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38730/0 (socket says 172.21.15.47:38730) 2024-04-20T02:28:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 -- 172.21.15.47:0/2753760734 learned_addr learned my addr 172.21.15.47:0/2753760734 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): get_auth_request con 0x7f077c0a0950 auth_method 0 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f077c13e460 0x7f077c070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f077c0a0160 0x7f077c13c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): get_auth_request con 0x7f077c13e460 auth_method 0 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): get_auth_request con 0x7f077c0a0160 auth_method 0 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f077a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient(hunting): handle_auth_done global_id 54843 payload 824 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 -- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f077c13e460 msgr2=0x7f077c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f077c13e460 0x7f077c070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 -- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f077c0a0160 msgr2=0x7f077c13c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f077c0a0160 0x7f077c13c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 monclient: found mon.noname-d 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 -- 172.21.15.47:0/2753760734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0774009a70 con 0x7f077c0a0950 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0779d9b700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 0x7f077c070580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f077000c650 tx=0x7f077000c960 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 client.?.objecter ms_handle_connect 0x7f077c0a0950 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f07700052e0 con 0x7f077c0a0950 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: dump: 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.266 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: _finish_auth 0 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: _check_auth_tickets 2024-04-20T02:28:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770005480 con 0x7f077c0a0950 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 5 monclient: authenticate success, global_id 54843 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0770010300 con 0x7f077c0a0950 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0778d99700 4 set_mon_vals no callback set 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f076affd700 10 monclient: dump: 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 10 monclient: _renew_subs 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c13bce0 con 0x7f077c0a0950 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 10 client.54843.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 10 monclient: _renew_subs 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c13b720 con 0x7f077c0a0950 2024-04-20T02:28:58.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.249+0000 7f0780c81700 1 librados: init done 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0770003680 con 0x7f077c0a0950 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f0780c81700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f0780c81700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f0780c81700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c06acc0 con 0x7f077c0a0950 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f076affd700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f075c03a2f0 0x7f075c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(511..511 src has 266..511) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f07700497f0 con 0x7f077c0a0950 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f076affd700 10 client.54843.objecter ms_dispatch 0x7f077c001550 osd_map(511..511 src has 266..511) v4 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f076affd700 3 client.54843.objecter handle_osd_map got epochs [511,511] > 0 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.250+0000 7f076affd700 3 client.54843.objecter handle_osd_map decoding full epoch 511 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.251+0000 7f076affd700 20 client.54843.objecter dump_active .. 0 homeless 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.251+0000 7f077a59c700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f075c03a2f0 0x7f075c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.251+0000 7f077a59c700 10 monclient: get_auth_request con 0x7f075c03a2f0 auth_method 0 2024-04-20T02:28:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.251+0000 7f076affd700 10 client.54843.objecter ms_handle_connect 0x7f075c03a2f0 2024-04-20T02:28:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.251+0000 7f077a59c700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f075c03a2f0 0x7f075c03c7b0 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f076c007b10 tx=0x7f076c007a60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.256+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0770010460 con 0x7f077c0a0950 2024-04-20T02:28:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.256+0000 7f076affd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.256+0000 7f076affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f0780c81700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:28:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f0780c81700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f0780c81700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f077c066730 con 0x7f077c0a0950 2024-04-20T02:28:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f076affd700 1 -- 172.21.15.47:0/2753760734 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f0770017080 con 0x7f077c0a0950 2024-04-20T02:28:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f076affd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:28:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.562+0000 7f076affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:58.563 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":88,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:28:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 10 librados: watch_flush enter 2024-04-20T02:28:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 10 librados: watch_flush exit 2024-04-20T02:28:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 20 client.54843.objecter shutdown clearing up homeless session... 2024-04-20T02:28:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 10 client.54843.objecter successfully canceled tick 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f075c03a2f0 msgr2=0x7f075c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f075c03a2f0 0x7f075c03c7b0 secure :-1 s=READY pgs=3287 cs=0 l=1 rev1=1 crypto rx=0x7f076c007b10 tx=0x7f076c007a60 comp rx=0 tx=0).stop 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 10 monclient: shutdown 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 msgr2=0x7f077c070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 0x7f077c070580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f077000c650 tx=0x7f077000c960 comp rx=0 tx=0).stop 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 shutdown_connections 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f075c03a2f0 0x7f075c03c7b0 unknown :-1 s=CLOSED pgs=3287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f077c0a0160 0x7f077c13c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f077c13e460 0x7f077c070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 --2- 172.21.15.47:0/2753760734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f077c0a0950 0x7f077c070580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 >> 172.21.15.47:0/2753760734 conn(0x7f077c0a7570 msgr2=0x7f077c09cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 shutdown_connections 2024-04-20T02:28:58.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 -- 172.21.15.47:0/2753760734 wait complete. 2024-04-20T02:28:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.564+0000 7f0780c81700 1 librados: shutdown 2024-04-20T02:28:58.578 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:28:58.578 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:28:58.578 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:28:58.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.713+0000 7fbcdf6f1700 10 monclient: get_monmap_and_config 2024-04-20T02:28:58.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.713+0000 7fbcdf6f1700 10 monclient: build_initial_monmap 2024-04-20T02:28:58.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.714+0000 7fbcdf6f1700 10 monclient: monmap: 2024-04-20T02:28:58.714 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:58.714 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:58.715109+0000 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:58.715109+0000 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:58.715 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:58.716 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.714+0000 7fbcdf6f1700 1 Processor -- start 2024-04-20T02:28:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.714+0000 7fbcdf6f1700 1 -- start start 2024-04-20T02:28:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.714+0000 7fbcdf6f1700 10 monclient: init 2024-04-20T02:28:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient: _add_conns ranks=[5,3,1,7,2,9,6,8,10,4,0] 2024-04-20T02:28:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbcd80a2950 0x7fbcd80a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): picked mon.noname-h con 0x7fbcd80a2950 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcd80a3270 0x7fbcd8137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): picked mon.noname-g con 0x7fbcd80a3270 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbcd8138040 addr v1:172.21.15.179:6789/0 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80751e0 con 0x7fbcd8138040 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd809f150 con 0x7fbcd80a3270 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd809ee70 con 0x7fbcd80a2950 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdf6f1700 10 monclient(hunting): authenticate will time out at 2418.116469s 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdcc8c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcd80a3270 0x7fbcd8137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdd48d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbcd80a2950 0x7fbcd80a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcddc8e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbcd8138040 0x7fbcd813efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:41548/0 (socket says 172.21.15.47:41548) 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcddc8e700 1 -- 172.21.15.47:0/3963911410 learned_addr learned my addr 172.21.15.47:0/3963911410 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdcc8c700 10 monclient(hunting): get_auth_request con 0x7fbcd80a3270 auth_method 0 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdcc8c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdcc8c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdcc8c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdd48d700 10 monclient(hunting): get_auth_request con 0x7fbcd80a2950 auth_method 0 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdd48d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdd48d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.715+0000 7fbcdd48d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbccffff700 1 -- 172.21.15.47:0/3963911410 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631273741 0 0) 0x7fbcd80751e0 con 0x7fbcd8138040 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbccffff700 10 monclient(hunting): _negotiate 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbccffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbccffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbccffff700 10 monclient(hunting): my global_id is 55198 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbccffff700 1 -- 172.21.15.47:0/3963911410 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc40035b0 con 0x7fbcd8138040 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_done global_id 84117 payload 274 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbcd80a2950 msgr2=0x7fbcd80a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 1 --2- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbcd80a2950 0x7fbcd80a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/3963911410 >> v1:172.21.15.179:6789/0 conn(0x7fbcd8138040 legacy=0x7fbcd813efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 10 monclient: _finish_hunting 0 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 1 monclient: found mon.noname-g 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/3963911410 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd809f710 con 0x7fbcd80a3270 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbcd80a2950 0x7fbcd80a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.716+0000 7fbcdcc8c700 1 --2- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcd80a3270 0x7fbcd8137b00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009870 tx=0x7fbcc8009b80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=732b0b4fd13cf381 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 1 -- 172.21.15.47:0/3963911410 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcc8011aa0 con 0x7fbcd80a3270 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: dump: 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.732 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: _finish_auth 0 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: _check_auth_tickets 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 5 monclient: authenticate success, global_id 84117 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 1 -- 172.21.15.47:0/3963911410 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8011c00 con 0x7fbcd80a3270 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 4 set_mon_vals no callback set 2024-04-20T02:28:58.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 1 -- 172.21.15.47:0/3963911410 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcc8011d70 con 0x7fbcd80a3270 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbccffff700 10 monclient: dump: 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.734 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 10 monclient: shutdown 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcd80a3270 msgr2=0x7fbcd8137b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcd80a3270 0x7fbcd8137b00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009870 tx=0x7fbcc8009b80 comp rx=0 tx=0).stop 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/3963911410 shutdown_connections 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbcd80a2950 0x7fbcd80a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/3963911410 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbcd80a3270 0x7fbcd8137b00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/3963911410 >> 172.21.15.47:0/3963911410 conn(0x7fbcd80a7570 msgr2=0x7fbcd80a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/3963911410 shutdown_connections 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.717+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/3963911410 wait complete. 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 10 monclient: build_initial_monmap 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 10 monclient: monmap: 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:58.736 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 1 librados: starting msgr at 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 1 librados: starting objecter 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 1 Processor -- start 2024-04-20T02:28:58.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 1 -- start start 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 1 librados: setting wanted keys 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 1 librados: calling monclient init 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.718+0000 7fbcdf6f1700 10 monclient: init 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient: _add_conns ranks=[7,2,0,6,1,8,3,4,5] 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a2950 0x7fbcd81cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): picked mon.noname-i con 0x7fbcd80a2950 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 0x7fbcd81ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): picked mon.noname-b con 0x7fbcd80a3270 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd8138040 0x7fbcd8123350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): picked mon.noname-a con 0x7fbcd8138040 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:28:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd809dd70 con 0x7fbcd8138040 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8124e90 con 0x7fbcd80a3270 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8125a60 con 0x7fbcd80a2950 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdf6f1700 10 monclient(hunting): authenticate will time out at 2418.120415s 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd8138040 0x7fbcd8123350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 0x7fbcd81ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 0x7fbcd81ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53994/0 (socket says 172.21.15.47:53994) 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd8138040 0x7fbcd8123350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40482/0 (socket says 172.21.15.47:40482) 2024-04-20T02:28:58.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/2277998864 learned_addr learned my addr 172.21.15.47:0/2277998864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): get_auth_request con 0x7fbcd80a3270 auth_method 0 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): get_auth_request con 0x7fbcd8138040 auth_method 0 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a2950 0x7fbcd81cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdd48d700 10 monclient(hunting): get_auth_request con 0x7fbcd80a2950 auth_method 0 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdd48d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdd48d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdd48d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient(hunting): handle_auth_done global_id 24473 payload 824 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a2950 msgr2=0x7fbcd81cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a2950 0x7fbcd81cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd8138040 msgr2=0x7fbcd8123350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd8138040 0x7fbcd8123350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient: _finish_hunting 0 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 monclient: found mon.noname-b 2024-04-20T02:28:58.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdcc8c700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40094a0 con 0x7fbcd80a3270 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcddc8e700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd8138040 0x7fbcd8123350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.719+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbcd80a2950 0x7fbcd81cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdcc8c700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 0x7fbcd81ccaa0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80090b0 tx=0x7fbcc8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 client.?.objecter ms_handle_connect 0x7fbcd80a3270 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcc8004330 con 0x7fbcd80a3270 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: dump: 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.742 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.743 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: _finish_auth 0 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: _check_auth_tickets 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc801f070 con 0x7fbcd80a3270 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 5 monclient: authenticate success, global_id 24473 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 10 monclient: _renew_subs 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc80094a0 con 0x7fbcd80a3270 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcc80044d0 con 0x7fbcd80a3270 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbccdffb700 10 monclient: dump: 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.745 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcceffd700 4 set_mon_vals no callback set 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 10 client.24473.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 10 monclient: _renew_subs 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd8126f80 con 0x7fbcd80a3270 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.720+0000 7fbcdf6f1700 1 librados: init done 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbcc8021030 con 0x7fbcd80a3270 2024-04-20T02:28:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdf6f1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 0x7fbcd8123890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8068310 con 0x7fbcd806aac0 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 0x7fbcd8123890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcbc03a320 0x7fbcbc03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient: get_auth_request con 0x7fbcd806aac0 auth_method 0 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(511..511 src has 266..511) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fbcc804b9e0 con 0x7fbcd80a3270 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 10 client.24473.objecter ms_dispatch 0x7fbcd8001550 osd_map(511..511 src has 266..511) v4 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 3 client.24473.objecter handle_osd_map got epochs [511,511] > 0 2024-04-20T02:28:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 3 client.24473.objecter handle_osd_map decoding full epoch 511 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.721+0000 7fbccdffb700 20 client.24473.objecter dump_active .. 0 homeless 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_done global_id 36288 payload 824 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbcdd48d700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbcd80672b0 con 0x7fbcd806aac0 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbcddc8e700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcbc03a320 0x7fbcbc03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbcddc8e700 10 monclient: get_auth_request con 0x7fbcbc03a320 auth_method 0 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 client.24473.objecter ms_handle_connect 0x7fbcd806aac0 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 client.24473.objecter resend_mon_ops 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 0x7fbcd8123890 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4011880 tx=0x7fbcd4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcd4011e80 con 0x7fbcd806aac0 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 monclient: dump: 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.749 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 client.24473.objecter ms_handle_connect 0x7fbcbc03a320 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbcddc8e700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcbc03a320 0x7fbcbc03c7e0 secure :-1 s=READY pgs=3288 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0001560 tx=0x7fbcd0001970 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbcd400f860 con 0x7fbcd806aac0 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 msgr2=0x7fbcd8123890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 0x7fbcd8123890 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4011880 tx=0x7fbcd4011c40 comp rx=0 tx=0).stop 2024-04-20T02:28:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 msgr2=0x7fbcd8123890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.722+0000 7fbccdffb700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd806aac0 0x7fbcd8123890 secure :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4011880 tx=0x7fbcd4011c40 comp rx=0 tx=0).stop 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdf6f1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 0x7fbcd8123dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdf6f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd806a400 con 0x7fbcd81269c0 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 0x7fbcd8123dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient: get_auth_request con 0x7fbcd81269c0 auth_method 0 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.730+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbcdd48d700 10 monclient(hunting): handle_auth_done global_id 36297 payload 824 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbcdd48d700 1 -- 172.21.15.47:0/2277998864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbcd80a36b0 con 0x7fbcd81269c0 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbcdd48d700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 0x7fbcd8123dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4011880 tx=0x7fbcd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 client.24473.objecter ms_handle_connect 0x7fbcd81269c0 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 client.24473.objecter resend_mon_ops 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcd40066c0 con 0x7fbcd81269c0 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 monclient: dump: 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.754 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.755 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fbcd4006820 con 0x7fbcd81269c0 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 msgr2=0x7fbcd8123dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 0x7fbcd8123dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4011880 tx=0x7fbcd4004750 comp rx=0 tx=0).stop 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 msgr2=0x7fbcd8123dd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.731+0000 7fbccdffb700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcd81269c0 0x7fbcd8123dd0 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4011880 tx=0x7fbcd4004750 comp rx=0 tx=0).stop 2024-04-20T02:28:58.756 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:28:58.757 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:58.758 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:58.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:28:58.760 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:28:58.761 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:28:58.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.764 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:28:58.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:58.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:28:58.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:28:58.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.771 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.773 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:28:58.778 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.779 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:28:58.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.781 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:28:58.781 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 10 librados: watch_flush enter 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 10 librados: watch_flush exit 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 20 client.24473.objecter shutdown clearing up homeless session... 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 10 client.24473.objecter successfully canceled tick 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcbc03a320 msgr2=0x7fbcbc03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbcbc03a320 0x7fbcbc03c7e0 secure :-1 s=READY pgs=3288 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0001560 tx=0x7fbcd0001970 comp rx=0 tx=0).stop 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 10 monclient: shutdown 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 msgr2=0x7fbcd81ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 0x7fbcd81ccaa0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80090b0 tx=0x7fbcc8004750 comp rx=0 tx=0).stop 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcddc8e700 1 -- 172.21.15.47:0/2277998864 reap_dead start 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 shutdown_connections 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 --2- 172.21.15.47:0/2277998864 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbcd80a3270 0x7fbcd81ccaa0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 >> 172.21.15.47:0/2277998864 conn(0x7fbcd80a7570 msgr2=0x7fbcd8137370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 shutdown_connections 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 -- 172.21.15.47:0/2277998864 wait complete. 2024-04-20T02:28:58.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.732+0000 7fbcdf6f1700 1 librados: shutdown 2024-04-20T02:28:58.784 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:28:58.784 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:28:58.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:28:58.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.924+0000 7fe0c0e91700 10 monclient: get_monmap_and_config 2024-04-20T02:28:58.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.924+0000 7fe0c0e91700 10 monclient: build_initial_monmap 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.924+0000 7fe0c0e91700 10 monclient: monmap: 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:58.925649+0000 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:58.925649+0000 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 1 Processor -- start 2024-04-20T02:28:58.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 1 -- start start 2024-04-20T02:28:58.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient: init 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient: _add_conns ranks=[2,0,6,7,1,8,4,10,3,5,9] 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 0x7fe0bc0a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient(hunting): picked mon.noname-b con 0x7fe0bc0a0830 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0bc0a01b0 addr v1:172.21.15.47:6789/0 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 0x7fe0bc0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient(hunting): picked mon.noname-d con 0x7fe0bc0a1150 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0751e0 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.925+0000 7fe0c0e91700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc067820 con 0x7fe0bc0a0830 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc09deb0 con 0x7fe0bc0a1150 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 10 monclient(hunting): authenticate will time out at 2418.327028s 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 0x7fe0bc0a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 0x7fe0bc0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 0x7fe0bc0a3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38744/0 (socket says 172.21.15.47:38744) 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b9d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe0bc0a01b0 0x7fe0bc0a2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47240/0 (socket says 172.21.15.47:47240) 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 0x7fe0bc0a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:53996/0 (socket says 172.21.15.47:53996) 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 -- 172.21.15.47:0/2413678087 learned_addr learned my addr 172.21.15.47:0/2413678087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): get_auth_request con 0x7fe0bc0a1150 auth_method 0 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): get_auth_request con 0x7fe0bc0a0830 auth_method 0 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient(hunting): handle_auth_done global_id 54852 payload 274 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 -- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 msgr2=0x7fe0bc0a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 --2- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 0x7fe0bc0a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 1 -- 172.21.15.47:0/2413678087 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245713694 0 0) 0x7fe0bc0751e0 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 -- 172.21.15.47:0/2413678087 >> v1:172.21.15.47:6789/0 conn(0x7fe0bc0a01b0 legacy=0x7fe0bc0a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 monclient: found mon.noname-d 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 -- 172.21.15.47:0/2413678087 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc067330 con 0x7fe0bc0a1150 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0ba59c700 1 --2- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 0x7fe0bc0a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0bad9d700 1 --2- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 0x7fe0bc0a3060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e80 tx=0x7fe0b400b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b7cd61ee3a7e242b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 1 -- 172.21.15.47:0/2413678087 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0b400e070 con 0x7fe0bc0a1150 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: dump: 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: _finish_auth 0 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: _check_auth_tickets 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 1 -- 172.21.15.47:0/2413678087 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400bd80 con 0x7fe0bc0a1150 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 5 monclient: authenticate success, global_id 54852 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0b959a700 1 -- 172.21.15.47:0/2413678087 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0b4012860 con 0x7fe0bc0a1150 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.926+0000 7fe0c0e91700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 4 set_mon_vals no callback set 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0b959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0b959a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0b959a700 10 monclient: dump: 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.941 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 10 monclient: shutdown 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 -- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 msgr2=0x7fe0bc0a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 0x7fe0bc0a3060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e80 tx=0x7fe0b400b330 comp rx=0 tx=0).stop 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 -- 172.21.15.47:0/2413678087 shutdown_connections 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a1150 0x7fe0bc0a3060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/2413678087 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc0a0830 0x7fe0bc0a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 -- 172.21.15.47:0/2413678087 >> 172.21.15.47:0/2413678087 conn(0x7fe0bc0a7570 msgr2=0x7fe0bc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 -- 172.21.15.47:0/2413678087 shutdown_connections 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.927+0000 7fe0c0e91700 1 -- 172.21.15.47:0/2413678087 wait complete. 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.928+0000 7fe0c0e91700 10 monclient: build_initial_monmap 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.928+0000 7fe0c0e91700 10 monclient: monmap: 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.928+0000 7fe0c0e91700 1 librados: starting msgr at 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.928+0000 7fe0c0e91700 1 librados: starting objecter 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.928+0000 7fe0c0e91700 1 Processor -- start 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 -- start start 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 librados: setting wanted keys 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 librados: calling monclient init 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient: init 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient: _add_conns ranks=[6,7,3,4,8,1,0,2,5] 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 0x7fe0bc1cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): picked mon.noname-d con 0x7fe0bc0a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:58.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe0bc0a0830 0x7fe0bc1bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): picked mon.noname-i con 0x7fe0bc0a0830 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc0a1150 0x7fe0bc1bc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): picked mon.noname-g con 0x7fe0bc0a1150 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1ab5d0 con 0x7fe0bc0a1150 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1ab820 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc09e900 con 0x7fe0bc0a0830 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0c0e91700 10 monclient(hunting): authenticate will time out at 2418.330574s 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 0x7fe0bc1cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 0x7fe0bc1cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38758/0 (socket says 172.21.15.47:38758) 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 -- 172.21.15.47:0/1468682909 learned_addr learned my addr 172.21.15.47:0/1468682909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:58.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): get_auth_request con 0x7fe0bc0a01b0 auth_method 0 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0bad9d700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc0a1150 0x7fe0bc1bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0bad9d700 10 monclient(hunting): get_auth_request con 0x7fe0bc0a1150 auth_method 0 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0bad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0bad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0bad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0b9d9b700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe0bc0a0830 0x7fe0bc1bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient(hunting): handle_auth_done global_id 54861 payload 824 2024-04-20T02:28:58.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe0bc0a0830 msgr2=0x7fe0bc1bbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe0bc0a0830 0x7fe0bc1bbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc0a1150 msgr2=0x7fe0bc1bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc0a1150 0x7fe0bc1bc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 10 monclient: _finish_hunting 0 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.929+0000 7fe0ba59c700 1 monclient: found mon.noname-d 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0ba59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0ba59c700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0ba59c700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0094a0 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0ba59c700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 0x7fe0bc1cc460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00ec10 tx=0x7fe0ac009ec0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 client.?.objecter ms_handle_connect 0x7fe0bc0a01b0 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0ac00cd50 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: dump: 2024-04-20T02:28:58.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: _finish_auth 0 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00ceb0 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 5 monclient: authenticate success, global_id 54861 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 10 monclient: _renew_subs 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b4009b30 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0ac0138e0 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0b8d99700 4 set_mon_vals no callback set 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 10 client.54861.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0aaffd700 10 monclient: dump: 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 10 monclient: _renew_subs 2024-04-20T02:28:58.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc1a9450 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.930+0000 7fe0c0e91700 1 librados: init done 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0c0e91700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 0x7fe0bc1a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1ab9b0 con 0x7fe0bc06aac0 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 0x7fe0bc1a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient: get_auth_request con 0x7fe0bc06aac0 auth_method 0 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe0ac013470 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe09c03a2f0 0x7fe09c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(511..511 src has 266..511) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fe0ac04abd0 con 0x7fe0bc0a01b0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 10 client.54861.objecter ms_dispatch 0x7fe0bc001550 osd_map(511..511 src has 266..511) v4 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 3 client.54861.objecter handle_osd_map got epochs [511,511] > 0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 3 client.54861.objecter handle_osd_map decoding full epoch 511 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_done global_id 24491 payload 824 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0bc06d700 con 0x7fe0bc06aac0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 20 client.54861.objecter dump_active .. 0 homeless 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0b9d9b700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 0x7fe0bc1a7910 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc09f860 tx=0x7fe0b000d850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 10 client.54861.objecter ms_handle_connect 0x7fe0bc06aac0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0aaffd700 10 client.54861.objecter resend_mon_ops 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0bad9d700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe09c03a2f0 0x7fe09c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.931+0000 7fe0bad9d700 10 monclient: get_auth_request con 0x7fe09c03a2f0 auth_method 0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0b00070d0 con 0x7fe0bc06aac0 2024-04-20T02:28:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 10 monclient: dump: 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe0b0004a80 con 0x7fe0bc06aac0 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 msgr2=0x7fe0bc1a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0bad9d700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe09c03a2f0 0x7fe09c03c7b0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e80 tx=0x7fe0b4009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 0x7fe0bc1a7910 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc09f860 tx=0x7fe0b000d850 comp rx=0 tx=0).stop 2024-04-20T02:28:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 msgr2=0x7fe0bc1a7910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc06aac0 0x7fe0bc1a7910 secure :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc09f860 tx=0x7fe0b000d850 comp rx=0 tx=0).stop 2024-04-20T02:28:58.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.932+0000 7fe0aaffd700 10 client.54861.objecter ms_handle_connect 0x7fe09c03a2f0 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0c0e91700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 0x7fe0bc1a8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0c0e91700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc068310 con 0x7fe0bc1a7e50 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 0x7fe0bc1a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient: get_auth_request con 0x7fe0bc1a7e50 auth_method 0 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 10 monclient(hunting): handle_auth_done global_id 24500 payload 824 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 1 -- 172.21.15.47:0/1468682909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe0bc09f1c0 con 0x7fe0bc1a7e50 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0aaffd700 10 client.54861.objecter ms_handle_connect 0x7fe0bc1a7e50 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0aaffd700 10 client.54861.objecter resend_mon_ops 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0b9d9b700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 0x7fe0bc1a8260 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0014000 tx=0x7fe0b0000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0b0020070 con 0x7fe0bc1a7e50 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0aaffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.940+0000 7fe0aaffd700 10 monclient: dump: 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fe0b001a890 con 0x7fe0bc1a7e50 2024-04-20T02:28:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 msgr2=0x7fe0bc1a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 0x7fe0bc1a8260 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0014000 tx=0x7fe0b0000ed0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 msgr2=0x7fe0bc1a8260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0aaffd700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe0bc1a7e50 0x7fe0bc1a8260 secure :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0014000 tx=0x7fe0b0000ed0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:28:58.960 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 3, 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:28:58.961 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:58.962 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:28:58.963 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:28:58.964 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:58.965 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:28:58.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:28:58.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.973 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:28:58.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:58.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:28:58.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:28:58.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:28:58.981 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 10 librados: watch_flush enter 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 10 librados: watch_flush exit 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 20 client.54861.objecter shutdown clearing up homeless session... 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 10 client.54861.objecter successfully canceled tick 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe09c03a2f0 msgr2=0x7fe09c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe09c03a2f0 0x7fe09c03c7b0 secure :-1 s=READY pgs=3289 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e80 tx=0x7fe0b4009000 comp rx=0 tx=0).stop 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 10 monclient: shutdown 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 msgr2=0x7fe0bc1cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 0x7fe0bc1cc460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00ec10 tx=0x7fe0ac009ec0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0bad9d700 1 -- 172.21.15.47:0/1468682909 reap_dead start 2024-04-20T02:28:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 shutdown_connections 2024-04-20T02:28:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 --2- 172.21.15.47:0/1468682909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe0bc0a01b0 0x7fe0bc1cc460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 >> 172.21.15.47:0/1468682909 conn(0x7fe0bc0a7570 msgr2=0x7fe0bc0a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 shutdown_connections 2024-04-20T02:28:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 -- 172.21.15.47:0/1468682909 wait complete. 2024-04-20T02:28:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:58.941+0000 7fe0c0e91700 1 librados: shutdown 2024-04-20T02:28:58.984 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:28:58.984 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:28:58.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:28:59.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.125+0000 7fcc5955c700 10 monclient: get_monmap_and_config 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.125+0000 7fcc5955c700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.125+0000 7fcc5955c700 10 monclient: monmap: 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:59.126870+0000 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:59.126870+0000 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.126 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.127+0000 7fcc5955c700 1 Processor -- start 2024-04-20T02:28:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.127+0000 7fcc5955c700 1 -- start start 2024-04-20T02:28:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.127+0000 7fcc5955c700 10 monclient: init 2024-04-20T02:28:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient: _add_conns ranks=[2,7,1,6,0,10,3,8,5,4,9] 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc5409c2d0 0x7fcc5409c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): picked mon.noname-b con 0x7fcc5409c2d0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc54134c40 0x7fcc5409cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): picked mon.noname-i con 0x7fcc54134c40 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcc540a1cc0 addr v1:172.21.15.179:6789/0 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc540751e0 con 0x7fcc540a1cc0 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540a0510 con 0x7fcc5409c2d0 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5406a620 con 0x7fcc54134c40 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc5955c700 10 monclient(hunting): authenticate will time out at 2418.529563s 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc52ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc5409c2d0 0x7fcc5409c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc537fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcc540a1cc0 0x7fcc540a2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:41558/0 (socket says 172.21.15.47:41558) 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc537fe700 1 -- 172.21.15.47:0/958825385 learned_addr learned my addr 172.21.15.47:0/958825385 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc52ffd700 10 monclient(hunting): get_auth_request con 0x7fcc5409c2d0 auth_method 0 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.128+0000 7fcc52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc51ffb700 1 -- 172.21.15.47:0/958825385 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653556416 0 0) 0x7fcc540751e0 con 0x7fcc540a1cc0 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc51ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:28:59.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc51ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc51ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc51ffb700 10 monclient(hunting): my global_id is 55207 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc51ffb700 1 -- 172.21.15.47:0/958825385 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc3c0035b0 con 0x7fcc540a1cc0 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_done global_id 24509 payload 274 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 1 -- 172.21.15.47:0/958825385 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc54134c40 msgr2=0x7fcc5409cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/958825385 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc54134c40 0x7fcc5409cbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 1 -- 172.21.15.47:0/958825385 >> v1:172.21.15.179:6789/0 conn(0x7fcc540a1cc0 legacy=0x7fcc540a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 1 monclient: found mon.noname-b 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 1 -- 172.21.15.47:0/958825385 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc540a0130 con 0x7fcc5409c2d0 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.129+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/958825385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc5409c2d0 0x7fcc5409c6b0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcc480098a0 tx=0x7fcc48009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed02fd3d0a772fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 1 -- 172.21.15.47:0/958825385 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc48011aa0 con 0x7fcc5409c2d0 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: dump: 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.134 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc5955c700 5 monclient: authenticate success, global_id 24509 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc5955c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 1 -- 172.21.15.47:0/958825385 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48011c00 con 0x7fcc5409c2d0 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc5955c700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 1 -- 172.21.15.47:0/958825385 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc48011d70 con 0x7fcc5409c2d0 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc5955c700 4 set_mon_vals no callback set 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.130+0000 7fcc51ffb700 10 monclient: dump: 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.136 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.137 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.137 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 10 monclient: shutdown 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 1 -- 172.21.15.47:0/958825385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc5409c2d0 msgr2=0x7fcc5409c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 1 --2- 172.21.15.47:0/958825385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc5409c2d0 0x7fcc5409c6b0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcc480098a0 tx=0x7fcc48009bb0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 1 -- 172.21.15.47:0/958825385 shutdown_connections 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 1 --2- 172.21.15.47:0/958825385 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcc5409c2d0 0x7fcc5409c6b0 secure :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcc480098a0 tx=0x7fcc48009bb0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 1 --2- 172.21.15.47:0/958825385 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc54134c40 0x7fcc5409cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.131+0000 7fcc5955c700 1 -- 172.21.15.47:0/958825385 >> 172.21.15.47:0/958825385 conn(0x7fcc540a2db0 msgr2=0x7fcc540a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.135+0000 7fcc5955c700 1 -- 172.21.15.47:0/958825385 shutdown_connections 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.135+0000 7fcc5955c700 1 -- 172.21.15.47:0/958825385 wait complete. 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.136+0000 7fcc5955c700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.136+0000 7fcc5955c700 10 monclient: monmap: 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.138 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.136+0000 7fcc5955c700 1 librados: starting msgr at 2024-04-20T02:28:59.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.136+0000 7fcc5955c700 1 librados: starting objecter 2024-04-20T02:28:59.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.138+0000 7fcc5955c700 1 Processor -- start 2024-04-20T02:28:59.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.141+0000 7fcc5955c700 1 -- start start 2024-04-20T02:28:59.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.141+0000 7fcc5955c700 1 librados: setting wanted keys 2024-04-20T02:28:59.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.141+0000 7fcc5955c700 1 librados: calling monclient init 2024-04-20T02:28:59.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.141+0000 7fcc5955c700 10 monclient: init 2024-04-20T02:28:59.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient: _add_conns ranks=[1,6,5,2,4,0,7,3,8] 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc540a1cc0 0x7fcc5408f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): picked mon.noname-f con 0x7fcc540a1cc0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcc54070500 0x7fcc5406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): picked mon.noname-d con 0x7fcc54070500 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc5406f190 0x7fcc5406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): picked mon.noname-h con 0x7fcc5406f190 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540717d0 con 0x7fcc540a1cc0 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54071d00 con 0x7fcc5406f190 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5409edf0 con 0x7fcc54070500 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.142+0000 7fcc5955c700 10 monclient(hunting): authenticate will time out at 2418.543799s 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcc54070500 0x7fcc5406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcc54070500 0x7fcc5406ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38770/0 (socket says 172.21.15.47:38770) 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 1 -- 172.21.15.47:0/919272996 learned_addr learned my addr 172.21.15.47:0/919272996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): get_auth_request con 0x7fcc54070500 auth_method 0 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc540a1cc0 0x7fcc5408f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc527fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): get_auth_request con 0x7fcc540a1cc0 auth_method 0 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.146+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 10 monclient(hunting): handle_auth_done global_id 54870 payload 824 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc5406f190 msgr2=0x7fcc5406f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc5406f190 0x7fcc5406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc540a1cc0 msgr2=0x7fcc5408f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc540a1cc0 0x7fcc5408f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 monclient: found mon.noname-d 2024-04-20T02:28:59.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:28:59.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc480094a0 con 0x7fcc54070500 2024-04-20T02:28:59.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.147+0000 7fcc527fc700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcc54070500 0x7fcc5406ec50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b450 tx=0x7fcc4400b760 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 client.?.objecter ms_handle_connect 0x7fcc54070500 2024-04-20T02:28:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc44011aa0 con 0x7fcc54070500 2024-04-20T02:28:59.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: dump: 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.160 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44011c40 con 0x7fcc54070500 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 5 monclient: authenticate success, global_id 54870 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 10 monclient: _renew_subs 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5406fe20 con 0x7fcc54070500 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc44011db0 con 0x7fcc54070500 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 10 client.54870.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc3b7fe700 10 monclient: dump: 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.162 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 10 monclient: _renew_subs 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5408fc20 con 0x7fcc54070500 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.155+0000 7fcc5955c700 1 librados: init done 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.156+0000 7fcc517fa700 4 set_mon_vals no callback set 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc5955c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc5955c700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 0x7fcc34004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34007d10 con 0x7fcc34002850 2024-04-20T02:28:59.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 0x7fcc34004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient: get_auth_request con 0x7fcc34002850 auth_method 0 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.158+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.159+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_done global_id 84109 payload 824 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.159+0000 7fcc52ffd700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc34007ed0 con 0x7fcc34002850 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.159+0000 7fcc3b7fe700 10 client.54870.objecter ms_handle_connect 0x7fcc34002850 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.159+0000 7fcc3b7fe700 10 client.54870.objecter resend_mon_ops 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.159+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 0x7fcc34004cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc48020070 con 0x7fcc34002850 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:28:59.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 10 monclient: dump: 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.165 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.166 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcc44011630 con 0x7fcc54070500 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc4003a730 0x7fcc4003cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(512..512 src has 266..512) v4 ==== 6562+0+0 (secure 0 0 0) 0x7fcc4404a7b0 con 0x7fcc54070500 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 10 client.54870.objecter ms_dispatch 0x7fcc54001550 osd_map(512..512 src has 266..512) v4 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 3 client.54870.objecter handle_osd_map got epochs [512,512] > 0 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.160+0000 7fcc3b7fe700 3 client.54870.objecter handle_osd_map decoding full epoch 512 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 20 client.54870.objecter dump_active .. 0 homeless 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc537fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc4003a730 0x7fcc4003cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc537fe700 10 monclient: get_auth_request con 0x7fcc4003a730 auth_method 0 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcc480036a0 con 0x7fcc34002850 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:59.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 msgr2=0x7fcc34004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 0x7fcc34004cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48004750 comp rx=0 tx=0).stop 2024-04-20T02:28:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 msgr2=0x7fcc34004cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34002850 0x7fcc34004cf0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48004750 comp rx=0 tx=0).stop 2024-04-20T02:28:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc3b7fe700 10 client.54870.objecter ms_handle_connect 0x7fcc4003a730 2024-04-20T02:28:59.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.161+0000 7fcc537fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc4003a730 0x7fcc4003cbf0 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0012e0 tx=0x7fcc4c0015b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.171+0000 7fcc5955c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.171+0000 7fcc5955c700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 0x7fcc34007150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.171+0000 7fcc5955c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.171+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3400a270 con 0x7fcc34006cc0 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 0x7fcc34007150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient: get_auth_request con 0x7fcc34006cc0 auth_method 0 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 10 monclient(hunting): handle_auth_done global_id 84118 payload 824 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 1 -- 172.21.15.47:0/919272996 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc34007690 con 0x7fcc34006cc0 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc52ffd700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 0x7fcc34007150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009790 tx=0x7fcc48011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc3b7fe700 10 client.54870.objecter ms_handle_connect 0x7fcc34006cc0 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc3b7fe700 10 client.54870.objecter resend_mon_ops 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc48009300 con 0x7fcc34006cc0 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc3b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc3b7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.172+0000 7fcc3b7fe700 10 monclient: dump: 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.174 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fcc48003e90 con 0x7fcc34006cc0 2024-04-20T02:28:59.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 msgr2=0x7fcc34007150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 0x7fcc34007150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009790 tx=0x7fcc48011000 comp rx=0 tx=0).stop 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 msgr2=0x7fcc34007150 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcc34006cc0 0x7fcc34007150 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009790 tx=0x7fcc48011000 comp rx=0 tx=0).stop 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:28:59.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:28:59.177 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.178 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:28:59.179 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.182 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:59.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:59.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:59.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.194 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:28:59.196 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 10 librados: watch_flush enter 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 10 librados: watch_flush exit 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 20 client.54870.objecter shutdown clearing up homeless session... 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 10 client.54870.objecter successfully canceled tick 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc4003a730 msgr2=0x7fcc4003cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc4003a730 0x7fcc4003cbf0 secure :-1 s=READY pgs=3290 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0012e0 tx=0x7fcc4c0015b0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 10 monclient: shutdown 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcc54070500 msgr2=0x7fcc5406ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 --2- 172.21.15.47:0/919272996 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fcc54070500 0x7fcc5406ec50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b450 tx=0x7fcc4400b760 comp rx=0 tx=0).stop 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc537fe700 1 -- 172.21.15.47:0/919272996 reap_dead start 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc537fe700 1 -- 172.21.15.47:0/919272996 reap_dead start 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 shutdown_connections 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 >> 172.21.15.47:0/919272996 conn(0x7fcc540a2db0 msgr2=0x7fcc5409d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 shutdown_connections 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 -- 172.21.15.47:0/919272996 wait complete. 2024-04-20T02:28:59.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.173+0000 7fcc5955c700 1 librados: shutdown 2024-04-20T02:28:59.199 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:28:59.199 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:28:59.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:28:59.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.337+0000 7fa8ea6dd700 10 monclient: get_monmap_and_config 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.337+0000 7fa8ea6dd700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.337+0000 7fa8ea6dd700 10 monclient: monmap: 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:59.338871+0000 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:59.338871+0000 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.338 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:59.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.338+0000 7fa8ea6dd700 1 Processor -- start 2024-04-20T02:28:59.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.338+0000 7fa8ea6dd700 1 -- start start 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.338+0000 7fa8ea6dd700 10 monclient: init 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient: _add_conns ranks=[9,10,4,3,8,1,2,6,5,0,7] 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8e40a4360 0x7fa8e4137b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): picked mon.noname-a con 0x7fa8e40a4360 addr v2:172.21.15.47:3300/0 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa8e41380d0 0x7fa8e41384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): picked mon.noname-f con 0x7fa8e41380d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa8e406c7a0 0x7fa8e406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): picked mon.noname-c con 0x7fa8e406c7a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e4138a10 con 0x7fa8e40a4360 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e4070370 con 0x7fa8e41380d0 2024-04-20T02:28:59.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40a4740 con 0x7fa8e406c7a0 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8ea6dd700 10 monclient(hunting): authenticate will time out at 2418.740130s 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8e40a4360 0x7fa8e4137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa8e406c7a0 0x7fa8e406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8e40a4360 0x7fa8e4137b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40506/0 (socket says 172.21.15.47:40506) 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 -- 172.21.15.47:0/1334162406 learned_addr learned my addr 172.21.15.47:0/1334162406 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): get_auth_request con 0x7fa8e40a4360 auth_method 0 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): get_auth_request con 0x7fa8e406c7a0 auth_method 0 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e37fe700 1 --2- 172.21.15.47:0/1334162406 >> v2:172.21.15.179:3300/0 conn(0x7fa8e41380d0 0x7fa8e41384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e37fe700 10 monclient(hunting): get_auth_request con 0x7fa8e41380d0 auth_method 0 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_done global_id 36315 payload 274 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 -- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa8e406c7a0 msgr2=0x7fa8e406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa8e406c7a0 0x7fa8e406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 -- 172.21.15.47:0/1334162406 >> v2:172.21.15.179:3300/0 conn(0x7fa8e41380d0 msgr2=0x7fa8e41384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:59.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/1334162406 >> v2:172.21.15.179:3300/0 conn(0x7fa8e41380d0 0x7fa8e41384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 monclient: found mon.noname-a 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 -- 172.21.15.47:0/1334162406 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e40751e0 con 0x7fa8e40a4360 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e8c7a700 1 --2- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa8e406c7a0 0x7fa8e406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8e40a4360 0x7fa8e4137b90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa8e406eb30 tx=0x7fa8d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec6ec10b04d5123 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e2ffd700 1 -- 172.21.15.47:0/1334162406 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d4011aa0 con 0x7fa8e40a4360 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.339+0000 7fa8e2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: dump: 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.353 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 5 monclient: authenticate success, global_id 36315 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 1 -- 172.21.15.47:0/1334162406 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4011c00 con 0x7fa8e40a4360 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 1 -- 172.21.15.47:0/1334162406 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d4011d70 con 0x7fa8e40a4360 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 4 set_mon_vals no callback set 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8e2ffd700 10 monclient: dump: 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.355 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 10 monclient: shutdown 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8e40a4360 msgr2=0x7fa8e4137b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8e40a4360 0x7fa8e4137b90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa8e406eb30 tx=0x7fa8d4009bd0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/1334162406 shutdown_connections 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa8e406c7a0 0x7fa8e406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/1334162406 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8e40a4360 0x7fa8e4137b90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/1334162406 >> v2:172.21.15.179:3300/0 conn(0x7fa8e41380d0 0x7fa8e41384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/1334162406 >> 172.21.15.47:0/1334162406 conn(0x7fa8e40a4a50 msgr2=0x7fa8e40a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/1334162406 shutdown_connections 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.340+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/1334162406 wait complete. 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.341+0000 7fa8ea6dd700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.341+0000 7fa8ea6dd700 10 monclient: monmap: 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:59.357 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.341+0000 7fa8ea6dd700 1 librados: starting msgr at 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.341+0000 7fa8ea6dd700 1 librados: starting objecter 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.341+0000 7fa8ea6dd700 1 Processor -- start 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 -- start start 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 librados: setting wanted keys 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 librados: calling monclient init 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient: init 2024-04-20T02:28:59.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient: _add_conns ranks=[3,2,7,5,0,6,8,4,1] 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e406c7a0 0x7fa8e40a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): picked mon.noname-g con 0x7fa8e406c7a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8e40a4360 0x7fa8e4133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): picked mon.noname-b con 0x7fa8e40a4360 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa8e41380d0 0x7fa8e41cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): picked mon.noname-i con 0x7fa8e41380d0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40720f0 con 0x7fa8e40a4360 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e406da70 con 0x7fa8e406c7a0 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4120b20 con 0x7fa8e41380d0 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8ea6dd700 10 monclient(hunting): authenticate will time out at 2418.743577s 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8e40a4360 0x7fa8e4133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8e40a4360 0x7fa8e4133a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54030/0 (socket says 172.21.15.47:54030) 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 -- 172.21.15.47:0/3915424351 learned_addr learned my addr 172.21.15.47:0/3915424351 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): get_auth_request con 0x7fa8e40a4360 auth_method 0 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e406c7a0 0x7fa8e40a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): get_auth_request con 0x7fa8e406c7a0 auth_method 0 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e8c7a700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa8e41380d0 0x7fa8e41cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e8c7a700 10 monclient(hunting): get_auth_request con 0x7fa8e41380d0 auth_method 0 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e8c7a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e8c7a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e8c7a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 10 monclient(hunting): handle_auth_done global_id 24518 payload 824 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa8e41380d0 msgr2=0x7fa8e41cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa8e41380d0 0x7fa8e41cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e406c7a0 msgr2=0x7fa8e40a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.342+0000 7fa8e37fe700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa8e406c7a0 0x7fa8e40a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e37fe700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e37fe700 1 monclient: found mon.noname-b 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e37fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e37fe700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc009a70 con 0x7fa8e40a4360 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e8c7a700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa8e41380d0 0x7fa8e41cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e37fe700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8e40a4360 0x7fa8e4133a00 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800d800 tx=0x7fa8d800db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 client.?.objecter ms_handle_connect 0x7fa8e40a4360 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d8012a60 con 0x7fa8e40a4360 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: dump: 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.362 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8012c00 con 0x7fa8e40a4360 2024-04-20T02:28:59.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 5 monclient: authenticate success, global_id 24518 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 10 monclient: _renew_subs 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d4009550 con 0x7fa8e40a4360 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d8012d70 con 0x7fa8e40a4360 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e0ff9700 10 monclient: dump: 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.364 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8e27fc700 4 set_mon_vals no callback set 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 10 client.24518.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 10 monclient: _renew_subs 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e411f7a0 con 0x7fa8e40a4360 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.343+0000 7fa8ea6dd700 1 librados: init done 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8ea6dd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 0x7fa8e41cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4068430 con 0x7fa8e4134370 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 0x7fa8e41cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient: get_auth_request con 0x7fa8e4134370 auth_method 0 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa8d80125f0 con 0x7fa8e40a4360 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa8d003a2f0 0x7fa8d003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_done global_id 54879 payload 824 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(512..512 src has 266..512) v4 ==== 6562+0+0 (secure 0 0 0) 0x7fa8d804acc0 con 0x7fa8e40a4360 2024-04-20T02:28:59.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 10 client.24518.objecter ms_dispatch 0x7fa8e4001550 osd_map(512..512 src has 266..512) v4 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 3 client.24518.objecter handle_osd_map got epochs [512,512] > 0 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 3 client.24518.objecter handle_osd_map decoding full epoch 512 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8e406e4c0 con 0x7fa8e4134370 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 20 client.24518.objecter dump_active .. 0 homeless 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 10 client.24518.objecter ms_handle_connect 0x7fa8e4134370 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e0ff9700 10 client.24518.objecter resend_mon_ops 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.344+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 0x7fa8e41cc7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8e406eb30 tx=0x7fa8d4004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e8c7a700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa8d003a2f0 0x7fa8d003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e8c7a700 10 monclient: get_auth_request con 0x7fa8d003a2f0 auth_method 0 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d4020070 con 0x7fa8e4134370 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 10 monclient: dump: 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.367 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa8d4004540 con 0x7fa8e4134370 2024-04-20T02:28:59.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 msgr2=0x7fa8e41cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 0x7fa8e41cc7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8e406eb30 tx=0x7fa8d4004750 comp rx=0 tx=0).stop 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 msgr2=0x7fa8e41cc7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4134370 0x7fa8e41cc7c0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8e406eb30 tx=0x7fa8d4004750 comp rx=0 tx=0).stop 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e0ff9700 10 client.24518.objecter ms_handle_connect 0x7fa8d003a2f0 2024-04-20T02:28:59.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.345+0000 7fa8e8c7a700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa8d003a2f0 0x7fa8d003c7b0 secure :-1 s=READY pgs=3291 cs=0 l=1 rev1=1 crypto rx=0x7fa8e41211b0 tx=0x7fa8dc00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8ea6dd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 0x7fa8e41ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8ea6dd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e406a400 con 0x7fa8e4133f40 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 0x7fa8e41ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient: get_auth_request con 0x7fa8e4133f40 auth_method 0 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 10 monclient(hunting): handle_auth_done global_id 54888 payload 824 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 1 -- 172.21.15.47:0/3915424351 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa8e406e1e0 con 0x7fa8e4133f40 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e3fff700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 0x7fa8e41ccd00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009840 tx=0x7fa8d4011000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e0ff9700 10 client.24518.objecter ms_handle_connect 0x7fa8e4133f40 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e0ff9700 10 client.24518.objecter resend_mon_ops 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa8d402cd50 con 0x7fa8e4133f40 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.353+0000 7fa8e0ff9700 10 monclient: dump: 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.372 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fa8d402c8e0 con 0x7fa8e4133f40 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 msgr2=0x7fa8e41ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 0x7fa8e41ccd00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009840 tx=0x7fa8d4011000 comp rx=0 tx=0).stop 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 msgr2=0x7fa8e41ccd00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e0ff9700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa8e4133f40 0x7fa8e41ccd00 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009840 tx=0x7fa8d4011000 comp rx=0 tx=0).stop 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:28:59.374 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.375 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.376 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:28:59.377 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.378 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:28:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:28:59.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.383 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:28:59.385 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.387 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:28:59.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.389 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:28:59.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.393 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:28:59.394 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 10 librados: watch_flush enter 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 10 librados: watch_flush exit 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 20 client.24518.objecter shutdown clearing up homeless session... 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 10 client.24518.objecter successfully canceled tick 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa8d003a2f0 msgr2=0x7fa8d003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa8d003a2f0 0x7fa8d003c7b0 secure :-1 s=READY pgs=3291 cs=0 l=1 rev1=1 crypto rx=0x7fa8e41211b0 tx=0x7fa8dc00b040 comp rx=0 tx=0).stop 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 10 monclient: shutdown 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8e40a4360 msgr2=0x7fa8e4133a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 --2- 172.21.15.47:0/3915424351 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa8e40a4360 0x7fa8e4133a00 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800d800 tx=0x7fa8d800db10 comp rx=0 tx=0).stop 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e8c7a700 1 -- 172.21.15.47:0/3915424351 reap_dead start 2024-04-20T02:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8e8c7a700 1 -- 172.21.15.47:0/3915424351 reap_dead start 2024-04-20T02:28:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 shutdown_connections 2024-04-20T02:28:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 >> 172.21.15.47:0/3915424351 conn(0x7fa8e40a4a50 msgr2=0x7fa8e40707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 shutdown_connections 2024-04-20T02:28:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 -- 172.21.15.47:0/3915424351 wait complete. 2024-04-20T02:28:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.354+0000 7fa8ea6dd700 1 librados: shutdown 2024-04-20T02:28:59.397 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:28:59.397 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:28:59.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.535+0000 7f4805b9a700 10 monclient: get_monmap_and_config 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.535+0000 7f4805b9a700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.535+0000 7f4805b9a700 10 monclient: monmap: 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:59.537075+0000 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:59.537075+0000 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:59.536 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:59.537 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.536+0000 7f4805b9a700 1 Processor -- start 2024-04-20T02:28:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.536+0000 7f4805b9a700 1 -- start start 2024-04-20T02:28:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.536+0000 7f4805b9a700 10 monclient: init 2024-04-20T02:28:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient: _add_conns ranks=[4,5,6,0,1,2,10,3,8,9,7] 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 0x7f48000a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): picked mon.noname-c con 0x7f48000a0830 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48000a01b0 0x7f48000a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): picked mon.noname-h con 0x7f48000a01b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48000a1150 0x7f48000a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): picked mon.noname-d con 0x7f48000a1150 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4800067820 con 0x7f48000a0830 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f480009deb0 con 0x7f48000a01b0 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4800067330 con 0x7f48000a1150 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f4805b9a700 10 monclient(hunting): authenticate will time out at 2418.938488s 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 0x7f48000a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48000a1150 0x7f48000a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 0x7f48000a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:45264/0 (socket says 172.21.15.47:45264) 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 -- 172.21.15.47:0/3010689775 learned_addr learned my addr 172.21.15.47:0/3010689775 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): get_auth_request con 0x7f48000a0830 auth_method 0 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): get_auth_request con 0x7f48000a1150 auth_method 0 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48000a01b0 0x7f48000a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): get_auth_request con 0x7f48000a01b0 auth_method 0 2024-04-20T02:28:59.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47feffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_done global_id 84136 payload 274 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 -- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48000a1150 msgr2=0x7f48000a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:59.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48000a1150 0x7f48000a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 -- 172.21.15.47:0/3010689775 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48000a01b0 msgr2=0x7f48000a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48000a01b0 0x7f48000a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.537+0000 7f47ff7fe700 1 monclient: found mon.noname-c 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47ff7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47ff7fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47ff7fe700 1 -- 172.21.15.47:0/3010689775 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48000751e0 con 0x7f48000a0830 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fffff700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48000a1150 0x7f48000a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 0x7f48000a0c10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009920 tx=0x7f47f4009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4258f62a33aff0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 1 -- 172.21.15.47:0/3010689775 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47f4011aa0 con 0x7f48000a0830 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: dump: 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.552 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 5 monclient: authenticate success, global_id 84136 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 1 -- 172.21.15.47:0/3010689775 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4011c00 con 0x7f48000a0830 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 4 set_mon_vals no callback set 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 1 -- 172.21.15.47:0/3010689775 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47f4011d70 con 0x7f48000a0830 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f47fe7fc700 10 monclient: dump: 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.554 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 10 monclient: shutdown 2024-04-20T02:28:59.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 1 -- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 msgr2=0x7f48000a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.538+0000 7f4805b9a700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 0x7f48000a0c10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009920 tx=0x7f47f4009c30 comp rx=0 tx=0).stop 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 -- 172.21.15.47:0/3010689775 shutdown_connections 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a0830 0x7f48000a0c10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48000a1150 0x7f48000a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 --2- 172.21.15.47:0/3010689775 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48000a01b0 0x7f48000a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 -- 172.21.15.47:0/3010689775 >> 172.21.15.47:0/3010689775 conn(0x7f48000a7570 msgr2=0x7f48000a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 -- 172.21.15.47:0/3010689775 shutdown_connections 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 1 -- 172.21.15.47:0/3010689775 wait complete. 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.539+0000 7f4805b9a700 10 monclient: monmap: 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:59.556 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 librados: starting msgr at 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 librados: starting objecter 2024-04-20T02:28:59.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 Processor -- start 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 -- start start 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 librados: setting wanted keys 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 librados: calling monclient init 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient: init 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient: _add_conns ranks=[3,1,4,6,2,5,0,7,8] 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48000a01b0 0x7f48001cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient(hunting): picked mon.noname-g con 0x7f48000a01b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48000a0830 0x7f48001bee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient(hunting): picked mon.noname-f con 0x7f48000a0830 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:28:59.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a1150 0x7f48001c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient(hunting): picked mon.noname-c con 0x7f48000a1150 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480009e440 con 0x7f48000a0830 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48001aa480 con 0x7f48000a01b0 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.540+0000 7f4805b9a700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48001abf40 con 0x7f48000a1150 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 monclient(hunting): authenticate will time out at 2418.942013s 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a1150 0x7f48001c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a1150 0x7f48001c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:45274/0 (socket says 172.21.15.47:45274) 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 -- 172.21.15.47:0/2055048335 learned_addr learned my addr 172.21.15.47:0/2055048335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): get_auth_request con 0x7f48000a1150 auth_method 0 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48000a01b0 0x7f48001cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47feffd700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48000a0830 0x7f48001bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47ff7fe700 10 monclient(hunting): get_auth_request con 0x7f48000a01b0 auth_method 0 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47ff7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47ff7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47ff7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47feffd700 10 monclient(hunting): get_auth_request con 0x7f48000a0830 auth_method 0 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47feffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47feffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47feffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient(hunting): handle_auth_done global_id 84145 payload 824 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48000a01b0 msgr2=0x7f48001cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48000a01b0 0x7f48001cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48000a0830 msgr2=0x7f48001bee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48000a0830 0x7f48001bee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 monclient: found mon.noname-c 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8009a70 con 0x7f48000a1150 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f48000a01b0 0x7f48001cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fffff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a1150 0x7f48001c8e80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f47f800ef90 tx=0x7f47f800c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 client.?.objecter ms_handle_connect 0x7f48000a1150 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47f8009cd0 con 0x7f48000a1150 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: dump: 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.562 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 5 monclient: authenticate success, global_id 84145 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800f040 con 0x7f48000a1150 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47f80148f0 con 0x7f48000a1150 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 monclient: _renew_subs 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f4009550 con 0x7f48000a1150 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47fdffb700 4 set_mon_vals no callback set 2024-04-20T02:28:59.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 client.84145.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f47e7fff700 10 monclient: dump: 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.564 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 monclient: _renew_subs 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48001ac500 con 0x7f48000a1150 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.541+0000 7f4805b9a700 1 librados: init done 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f4805b9a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f47f8014480 con 0x7f48000a1150 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f4805b9a700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 0x7f480006aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4800068310 con 0x7f48001ab270 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 0x7f480006aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47ff7fe700 10 monclient: get_auth_request con 0x7f48001ab270 auth_method 0 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47ff7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47ff7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47ff7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.542+0000 7f47e7fff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f47ec03a320 0x7f47ec03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(512..512 src has 266..512) v4 ==== 6562+0+0 (secure 0 0 0) 0x7f47f804b140 con 0x7f48000a1150 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 client.84145.objecter ms_dispatch 0x7f4800001550 osd_map(512..512 src has 266..512) v4 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 3 client.84145.objecter handle_osd_map got epochs [512,512] > 0 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 3 client.84145.objecter handle_osd_map decoding full epoch 512 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47feffd700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f47ec03a320 0x7f47ec03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47feffd700 10 monclient: get_auth_request con 0x7f47ec03a320 auth_method 0 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 20 client.84145.objecter dump_active .. 0 homeless 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_done global_id 35657 payload 824 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47ff7fe700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f480009f1c0 con 0x7f48001ab270 2024-04-20T02:28:59.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 client.84145.objecter ms_handle_connect 0x7f48001ab270 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 client.84145.objecter resend_mon_ops 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 0x7f480006aac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009920 tx=0x7f47f4004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47f40044c0 con 0x7f48001ab270 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 monclient: dump: 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.567 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47feffd700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f47ec03a320 0x7f47ec03c7e0 secure :-1 s=READY pgs=3292 cs=0 l=1 rev1=1 crypto rx=0x7f48001a9fb0 tx=0x7f47f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 client.84145.objecter ms_handle_connect 0x7f47ec03a320 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f47f40036a0 con 0x7f48001ab270 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 msgr2=0x7f480006aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 0x7f480006aac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009920 tx=0x7f47f4004750 comp rx=0 tx=0).stop 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 msgr2=0x7f480006aac0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.543+0000 7f47e7fff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001ab270 0x7f480006aac0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f4805b9a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:28:59.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f4805b9a700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 0x7f48001c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f4805b9a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f480006a400 con 0x7f48001bf340 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 0x7f48001c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient: get_auth_request con 0x7f48001bf340 auth_method 0 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.551+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47ff7fe700 10 monclient(hunting): handle_auth_done global_id 35666 payload 824 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47ff7fe700 1 -- 172.21.15.47:0/2055048335 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48000a4450 con 0x7f48001bf340 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47ff7fe700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 0x7f48001c93c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009840 tx=0x7f47f40092c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 client.84145.objecter ms_handle_connect 0x7f48001bf340 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 client.84145.objecter resend_mon_ops 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47f402ac00 con 0x7f48001bf340 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 monclient: dump: 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.572 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f47f402ad60 con 0x7f48001bf340 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 msgr2=0x7f48001c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 0x7f48001c93c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009840 tx=0x7f47f40092c0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 msgr2=0x7f48001c93c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.552+0000 7f47e7fff700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48001bf340 0x7f48001c93c0 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009840 tx=0x7f47f40092c0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:28:59.574 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.575 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.576 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:28:59.577 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.578 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.580 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:28:59.582 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:28:59.585 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.587 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.590 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.592 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.593 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:28:59.594 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 10 librados: watch_flush enter 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 10 librados: watch_flush exit 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 20 client.84145.objecter shutdown clearing up homeless session... 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 10 client.84145.objecter successfully canceled tick 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f47ec03a320 msgr2=0x7f47ec03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f47ec03a320 0x7f47ec03c7e0 secure :-1 s=READY pgs=3292 cs=0 l=1 rev1=1 crypto rx=0x7f48001a9fb0 tx=0x7f47f0008040 comp rx=0 tx=0).stop 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 10 monclient: shutdown 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a1150 msgr2=0x7f48001c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 --2- 172.21.15.47:0/2055048335 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48000a1150 0x7f48001c8e80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f47f800ef90 tx=0x7f47f800c3a0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f47fffff700 1 -- 172.21.15.47:0/2055048335 reap_dead start 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f47fffff700 1 -- 172.21.15.47:0/2055048335 reap_dead start 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 shutdown_connections 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 >> 172.21.15.47:0/2055048335 conn(0x7f48000a7570 msgr2=0x7f48000722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 shutdown_connections 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 -- 172.21.15.47:0/2055048335 wait complete. 2024-04-20T02:28:59.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.553+0000 7f4805b9a700 1 librados: shutdown 2024-04-20T02:28:59.597 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:28:59.597 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:28:59.597 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:28:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.735+0000 7f5a4a6ec700 10 monclient: get_monmap_and_config 2024-04-20T02:28:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.735+0000 7f5a4a6ec700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.735+0000 7f5a4a6ec700 10 monclient: monmap: 2024-04-20T02:28:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:28:59.736290+0000 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:28:59.736290+0000 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:28:59.736 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.735+0000 7f5a4a6ec700 1 Processor -- start 2024-04-20T02:28:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 -- start start 2024-04-20T02:28:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient: init 2024-04-20T02:28:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient: _add_conns ranks=[9,3,10,6,5,8,7,1,4,2,0] 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a440a2950 0x7f5a440a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): picked mon.noname-a con 0x7f5a440a2950 addr v2:172.21.15.47:3300/0 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a4409dd40 0x7f5a440a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): picked mon.noname-g con 0x7f5a4409dd40 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5a44137b00 0x7f5a44137f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): picked mon.noname-f con 0x7f5a44137b00 addr v2:172.21.15.179:3300/0 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4409f230 con 0x7f5a440a2950 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a440a37b0 con 0x7f5a44137b00 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440673a0 con 0x7f5a4409dd40 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a4a6ec700 10 monclient(hunting): authenticate will time out at 2419.137593s 2024-04-20T02:28:59.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a440a2950 0x7f5a440a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5a440a2950 0x7f5a440a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40508/0 (socket says 172.21.15.47:40508) 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 1 -- 172.21.15.47:0/204438499 learned_addr learned my addr 172.21.15.47:0/204438499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): get_auth_request con 0x7f5a440a2950 auth_method 0 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a437fe700 1 --2- 172.21.15.47:0/204438499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a4409dd40 0x7f5a440a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a48c89700 1 --2- 172.21.15.47:0/204438499 >> v2:172.21.15.179:3300/0 conn(0x7f5a44137b00 0x7f5a44137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a43fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a437fe700 10 monclient(hunting): get_auth_request con 0x7f5a4409dd40 auth_method 0 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a48c89700 10 monclient(hunting): get_auth_request con 0x7f5a44137b00 auth_method 0 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a48c89700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a48c89700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.736+0000 7f5a48c89700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a48c89700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a48c89700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a48c89700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 10 monclient(hunting): handle_auth_done global_id 36324 payload 274 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 -- 172.21.15.47:0/204438499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a4409dd40 msgr2=0x7f5a440a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 --2- 172.21.15.47:0/204438499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a4409dd40 0x7f5a440a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 -- 172.21.15.47:0/204438499 >> v2:172.21.15.179:3300/0 conn(0x7f5a44137b00 msgr2=0x7f5a44137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 --2- 172.21.15.47:0/204438499 >> v2:172.21.15.179:3300/0 conn(0x7f5a44137b00 0x7f5a44137f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 monclient: found mon.noname-a 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 -- 172.21.15.47:0/204438499 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440751e0 con 0x7f5a440a2950 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a43fff700 1 --2- 172.21.15.47:0/204438499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a440a2950 0x7f5a440a2d30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5a440728c0 tx=0x7f5a34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ec33f3a8ddef42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 1 -- 172.21.15.47:0/204438499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a34011aa0 con 0x7f5a440a2950 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: dump: 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.748 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 1 -- 172.21.15.47:0/204438499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a34011c00 con 0x7f5a440a2950 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a4a6ec700 5 monclient: authenticate success, global_id 36324 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a4a6ec700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:28:59.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a4a6ec700 10 monclient: get_monmap_and_config success 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 1 -- 172.21.15.47:0/204438499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a34011d70 con 0x7f5a440a2950 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a4a6ec700 4 set_mon_vals no callback set 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.737+0000 7f5a42ffd700 10 monclient: dump: 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.750 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 10 monclient: shutdown 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/204438499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a440a2950 msgr2=0x7f5a440a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/204438499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a440a2950 0x7f5a440a2d30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5a440728c0 tx=0x7f5a34009bd0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/204438499 shutdown_connections 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/204438499 >> v2:172.21.15.179:3300/0 conn(0x7f5a44137b00 0x7f5a44137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/204438499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a440a2950 0x7f5a440a2d30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/204438499 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a4409dd40 0x7f5a440a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/204438499 >> 172.21.15.47:0/204438499 conn(0x7f5a440a7570 msgr2=0x7f5a440a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/204438499 shutdown_connections 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/204438499 wait complete. 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 10 monclient: build_initial_monmap 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.738+0000 7f5a4a6ec700 10 monclient: monmap: 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:28:59.752 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 librados: starting msgr at 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 librados: starting objecter 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 Processor -- start 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 -- start start 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 librados: setting wanted keys 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 librados: calling monclient init 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 10 monclient: init 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 10 monclient: _add_conns ranks=[3,7,2,0,6,4,5,1,8] 2024-04-20T02:28:59.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a440a2950 0x7f5a441cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 10 monclient(hunting): picked mon.noname-g con 0x7f5a440a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.739+0000 7f5a4a6ec700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a44137b00 0x7f5a44136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): picked mon.noname-i con 0x7f5a44137b00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): picked mon.noname-b con 0x7f5a44137260 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44125e60 con 0x7f5a44137260 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4409e4d0 con 0x7f5a440a2950 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4409ea20 con 0x7f5a44137b00 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a4a6ec700 10 monclient(hunting): authenticate will time out at 2419.141060s 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54034/0 (socket says 172.21.15.47:54034) 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 learned_addr learned my addr 172.21.15.47:0/322892764 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): get_auth_request con 0x7f5a44137260 auth_method 0 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a43fff700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a440a2950 0x7f5a441cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a437fe700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a44137b00 0x7f5a44136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a43fff700 10 monclient(hunting): get_auth_request con 0x7f5a440a2950 auth_method 0 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a43fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a43fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a43fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a437fe700 10 monclient(hunting): get_auth_request con 0x7f5a44137b00 auth_method 0 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient(hunting): handle_auth_done global_id 24527 payload 824 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a44137b00 msgr2=0x7f5a44136d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f5a44137b00 0x7f5a44136d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a440a2950 msgr2=0x7f5a441cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a440a2950 0x7f5a441cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 monclient: found mon.noname-b 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a34009550 con 0x7f5a44137260 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a43fff700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5a440a2950 0x7f5a441cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:28:59.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 msgr2=0x7f5a441227f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:28:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 msgr2=0x7f5a441227f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:28:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00efc0 tx=0x7f5a3c00c3a0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:28:59.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.740+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 10 monclient: get_auth_request con 0x7f5a44137260 auth_method 0 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 0 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient: ms_handle_reset current mon [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient: _reopen_session rank -1 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 msgr2=0x7f5a441227f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5a44137260 0x7f5a441227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient: _add_conns ranks=[0,5,1,4,6,8] 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a30003660 0x7f5a30005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): picked mon.noname-a con 0x7f5a30003660 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a30006650 0x7f5a30008ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): picked mon.noname-h con 0x7f5a30006650 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a300096c0 0x7f5a3000bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): picked mon.noname-f con 0x7f5a300096c0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000c770 con 0x7f5a30003660 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000c8b0 con 0x7f5a300096c0 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000ca70 con 0x7f5a30006650 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a40ff9700 10 monclient(hunting): _renew_subs 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a30003660 0x7f5a30005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): get_auth_request con 0x7f5a30003660 auth_method 0 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a437fe700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a30006650 0x7f5a30008ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a300096c0 0x7f5a3000bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a437fe700 10 monclient(hunting): get_auth_request con 0x7f5a30006650 auth_method 0 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a437fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 10 monclient(hunting): get_auth_request con 0x7f5a300096c0 auth_method 0 2024-04-20T02:28:59.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient(hunting): handle_auth_done global_id 24527 payload 293 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a30006650 msgr2=0x7f5a30008ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5a30006650 0x7f5a30008ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a300096c0 msgr2=0x7f5a3000bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a300096c0 0x7f5a3000bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient: _finish_hunting 0 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 monclient: found mon.noname-a 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a43fff700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3000ccf0 con 0x7f5a30003660 2024-04-20T02:28:59.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.941+0000 7f5a48c89700 1 -- 172.21.15.47:0/322892764 reap_dead start 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a43fff700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a30003660 0x7f5a30005ab0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5a34016000 tx=0x7f5a3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 client.?.objecter ms_handle_connect 0x7f5a30003660 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a340044b0 con 0x7f5a30003660 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: dump: 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.951 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: _finish_auth 0 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a34011950 con 0x7f5a30003660 2024-04-20T02:28:59.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a34003b10 con 0x7f5a30003660 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 5 monclient: authenticate success, global_id 24527 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a40ff9700 10 monclient: dump: 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.953 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a427fc700 4 set_mon_vals no callback set 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 10 monclient: _renew_subs 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a44124330 con 0x7f5a30003660 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 10 client.24527.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:28:59.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 10 monclient: _renew_subs 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a44124c30 con 0x7f5a30003660 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.942+0000 7f5a4a6ec700 1 librados: init done 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a4a6ec700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 0x7f5a44136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44068310 con 0x7f5a4409dd40 2024-04-20T02:28:59.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5a34003c90 con 0x7f5a30003660 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a437fe700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 0x7f5a44136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a437fe700 10 monclient: get_auth_request con 0x7f5a4409dd40 auth_method 0 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.943+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5a3003e860 0x7f5a30040d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(512..512 src has 266..512) v4 ==== 6562+0+0 (secure 0 0 0) 0x7f5a3404b7b0 con 0x7f5a30003660 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 client.24527.objecter ms_dispatch 0x7f5a44001550 osd_map(512..512 src has 266..512) v4 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 3 client.24527.objecter handle_osd_map got epochs [512,512] > 0 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 3 client.24527.objecter handle_osd_map decoding full epoch 512 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 20 client.24527.objecter dump_active .. 0 homeless 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5a3003e860 0x7f5a30040d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a48c89700 10 monclient: get_auth_request con 0x7f5a3003e860 auth_method 0 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a437fe700 10 monclient(hunting): handle_auth_done global_id 55252 payload 824 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a437fe700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a440a44b0 con 0x7f5a4409dd40 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a437fe700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 0x7f5a44136d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5a440a47e0 tx=0x7f5a380138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 client.24527.objecter ms_handle_connect 0x7f5a4409dd40 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 client.24527.objecter resend_mon_ops 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 client.24527.objecter ms_handle_connect 0x7f5a3003e860 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a48c89700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5a3003e860 0x7f5a30040d20 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7f5a44123fd0 tx=0x7f5a3c014000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a380044a0 con 0x7f5a4409dd40 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.944+0000 7f5a40ff9700 10 monclient: dump: 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5a38004030 con 0x7f5a4409dd40 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 msgr2=0x7f5a44136d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 0x7f5a44136d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5a440a47e0 tx=0x7f5a380138f0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 msgr2=0x7f5a44136d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.945+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 0x7f5a44136d20 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5a440a47e0 tx=0x7f5a380138f0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a4a6ec700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 0x7f5a44126080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a4a6ec700 10 monclient(hunting): start opening mon connection 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4406a400 con 0x7f5a4406aac0 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a437fe700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 0x7f5a44126080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a437fe700 10 monclient: get_auth_request con 0x7f5a4406aac0 auth_method 0 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a437fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a437fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.953+0000 7f5a437fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:28:59.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a437fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a437fe700 10 monclient(hunting): handle_auth_done global_id 55261 payload 824 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a437fe700 1 -- 172.21.15.47:0/322892764 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5a440677a0 con 0x7f5a4406aac0 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a40ff9700 10 client.24527.objecter ms_handle_connect 0x7f5a4406aac0 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a437fe700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 0x7f5a44126080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a40ff9700 10 client.24527.objecter resend_mon_ops 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a38026070 con 0x7f5a4406aac0 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a40ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a40ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.954+0000 7f5a40ff9700 10 monclient: dump: 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:28:59.962 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f5a38016670 con 0x7f5a4406aac0 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:28:59.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 msgr2=0x7f5a44126080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 0x7f5a44126080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a380047c0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 msgr2=0x7f5a44126080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a40ff9700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 0x7f5a44126080 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a38000c00 tx=0x7f5a380047c0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:28:59.964 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.965 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:28:59.967 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:28:59.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.978 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:28:59.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:28:59.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:28:59.984 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:28:59.985 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:28:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 10 librados: watch_flush enter 2024-04-20T02:28:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 10 librados: watch_flush exit 2024-04-20T02:28:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 20 client.24527.objecter shutdown clearing up homeless session... 2024-04-20T02:28:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 10 client.24527.objecter successfully canceled tick 2024-04-20T02:28:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5a3003e860 msgr2=0x7f5a30040d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5a3003e860 0x7f5a30040d20 secure :-1 s=READY pgs=3293 cs=0 l=1 rev1=1 crypto rx=0x7f5a44123fd0 tx=0x7f5a3c014000 comp rx=0 tx=0).stop 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 10 monclient: shutdown 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a30003660 msgr2=0x7f5a30005ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a30003660 0x7f5a30005ab0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5a34016000 tx=0x7f5a3c00b040 comp rx=0 tx=0).stop 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 shutdown_connections 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5a3003e860 0x7f5a30040d20 unknown :-1 s=CLOSED pgs=3293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a30003660 0x7f5a30005ab0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4406aac0 0x7f5a44126080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 --2- 172.21.15.47:0/322892764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a4409dd40 0x7f5a44136d20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 >> 172.21.15.47:0/322892764 conn(0x7f5a440a7570 msgr2=0x7f5a441bcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 shutdown_connections 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 -- 172.21.15.47:0/322892764 wait complete. 2024-04-20T02:28:59.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:28:59.955+0000 7f5a4a6ec700 1 librados: shutdown 2024-04-20T02:28:59.989 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:28:59.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:28:59.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.133+0000 7f9f13e81700 10 monclient: get_monmap_and_config 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.133+0000 7f9f13e81700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.133+0000 7f9f13e81700 10 monclient: monmap: 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:00.134310+0000 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:00.134310+0000 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:00.134 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.133+0000 7f9f13e81700 1 Processor -- start 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 -- start start 2024-04-20T02:29:00.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient: init 2024-04-20T02:29:00.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient: _add_conns ranks=[4,7,9,3,5,8,2,1,10,0,6] 2024-04-20T02:29:00.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): picked mon.noname-c con 0x7f9f0c0a3960 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9f0c0a3d40 0x7f9f0c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): picked mon.noname-i con 0x7f9f0c0a3d40 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 0x7f9f0c134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): picked mon.noname-a con 0x7f9f0c134700 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c06ca20 con 0x7f9f0c134700 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c06e110 con 0x7f9f0c0a3960 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c06e3f0 con 0x7f9f0c0a3d40 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f13e81700 10 monclient(hunting): authenticate will time out at 2419.535593s 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 0x7f9f0c134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:45280/0 (socket says 172.21.15.47:45280) 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 0x7f9f0c134bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40526/0 (socket says 172.21.15.47:40526) 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 1 -- 172.21.15.47:0/3382802970 learned_addr learned my addr 172.21.15.47:0/3382802970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): get_auth_request con 0x7f9f0c0a3960 auth_method 0 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): get_auth_request con 0x7f9f0c134700 auth_method 0 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1141c700 1 --2- 172.21.15.47:0/3382802970 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9f0c0a3d40 0x7f9f0c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1241e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1141c700 10 monclient(hunting): get_auth_request con 0x7f9f0c0a3d40 auth_method 0 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1141c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1141c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.134+0000 7f9f1141c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f1141c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f1141c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f1141c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_done global_id 84154 payload 274 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 -- 172.21.15.47:0/3382802970 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9f0c0a3d40 msgr2=0x7f9f0c0a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/3382802970 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9f0c0a3d40 0x7f9f0c0a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 -- 172.21.15.47:0/3382802970 >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 msgr2=0x7f9f0c134bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:00.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/3382802970 >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 0x7f9f0c134bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 monclient: found mon.noname-c 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 -- 172.21.15.47:0/3382802970 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0751e0 con 0x7f9f0c0a3960 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f1241e700 1 --2- 172.21.15.47:0/3382802970 >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 0x7f9f0c134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/3382802970 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c070590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f080098f0 tx=0x7f9f08009c00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f1d959452185e728 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 1 -- 172.21.15.47:0/3382802970 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f08011aa0 con 0x7f9f0c0a3960 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: dump: 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.147 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 1 -- 172.21.15.47:0/3382802970 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f08011c00 con 0x7f9f0c0a3960 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f13e81700 5 monclient: authenticate success, global_id 84154 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f13e81700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f13e81700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f13e81700 4 set_mon_vals no callback set 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 1 -- 172.21.15.47:0/3382802970 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f08011d70 con 0x7f9f0c0a3960 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.135+0000 7f9f10c1b700 10 monclient: dump: 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 10 monclient: shutdown 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 -- 172.21.15.47:0/3382802970 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 msgr2=0x7f9f0c070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 --2- 172.21.15.47:0/3382802970 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c070590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f080098f0 tx=0x7f9f08009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 -- 172.21.15.47:0/3382802970 shutdown_connections 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 --2- 172.21.15.47:0/3382802970 >> v2:172.21.15.47:3300/0 conn(0x7f9f0c134700 0x7f9f0c134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 --2- 172.21.15.47:0/3382802970 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c070590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 --2- 172.21.15.47:0/3382802970 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f9f0c0a3d40 0x7f9f0c0a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 -- 172.21.15.47:0/3382802970 >> 172.21.15.47:0/3382802970 conn(0x7f9f0c0a8610 msgr2=0x7f9f0c0a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 -- 172.21.15.47:0/3382802970 shutdown_connections 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.136+0000 7f9f13e81700 1 -- 172.21.15.47:0/3382802970 wait complete. 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 10 monclient: monmap: 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 1 librados: starting msgr at 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 1 librados: starting objecter 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 1 Processor -- start 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 1 -- start start 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 1 librados: setting wanted keys 2024-04-20T02:29:00.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 1 librados: calling monclient init 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.137+0000 7f9f13e81700 10 monclient: init 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient: _add_conns ranks=[5,1,2,4,3,0,7,6,8] 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c0a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): picked mon.noname-h con 0x7f9f0c0a3960 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f0c0a3d40 0x7f9f0c133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): picked mon.noname-f con 0x7f9f0c0a3d40 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0c134700 0x7f9f0c133ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): picked mon.noname-b con 0x7f9f0c134700 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1cc740 con 0x7f9f0c0a3d40 2024-04-20T02:29:00.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c071f80 con 0x7f9f0c134700 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c120f40 con 0x7f9f0c0a3960 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f13e81700 10 monclient(hunting): authenticate will time out at 2419.539435s 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0c134700 0x7f9f0c133ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0c134700 0x7f9f0c133ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54064/0 (socket says 172.21.15.47:54064) 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 -- 172.21.15.47:0/2199716346 learned_addr learned my addr 172.21.15.47:0/2199716346 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): get_auth_request con 0x7f9f0c134700 auth_method 0 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c0a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f0c0a3d40 0x7f9f0c133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): get_auth_request con 0x7f9f0c0a3960 auth_method 0 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): get_auth_request con 0x7f9f0c0a3d40 auth_method 0 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1141c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient(hunting): handle_auth_done global_id 24536 payload 824 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f0c0a3960 msgr2=0x7f9f0c0a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f9f0c0a3960 0x7f9f0c0a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f0c0a3d40 msgr2=0x7f9f0c133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f0c0a3d40 0x7f9f0c133ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 monclient: found mon.noname-b 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:29:00.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.138+0000 7f9f1241e700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f04009a70 con 0x7f9f0c134700 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f1241e700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0c134700 0x7f9f0c133ff0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009e60 tx=0x7f9f0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 client.?.objecter ms_handle_connect 0x7f9f0c134700 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f04009cd0 con 0x7f9f0c134700 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: dump: 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.157 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0400f040 con 0x7f9f0c134700 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 5 monclient: authenticate success, global_id 24536 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 10 monclient: _renew_subs 2024-04-20T02:29:00.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f08009550 con 0x7f9f0c134700 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f040148f0 con 0x7f9f0c134700 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f03fff700 4 set_mon_vals no callback set 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 10 client.24536.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f027fc700 10 monclient: dump: 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.159 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 10 monclient: _renew_subs 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c134530 con 0x7f9f0c134700 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.139+0000 7f9f13e81700 1 librados: init done 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f13e81700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f13e81700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 0x7f9f0c11e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c068380 con 0x7f9f0c06aac0 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 0x7f9f0c11e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient: get_auth_request con 0x7f9f0c06aac0 auth_method 0 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_done global_id 84171 payload 824 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.140+0000 7f9f11c1d700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f0c04ec20 con 0x7f9f0c06aac0 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.141+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 0x7f9f0c11e5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f080098f0 tx=0x7f9f08004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.143+0000 7f9f027fc700 10 client.24536.objecter ms_handle_connect 0x7f9f0c06aac0 2024-04-20T02:29:00.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.143+0000 7f9f027fc700 10 client.24536.objecter resend_mon_ops 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9f040144a0 con 0x7f9f0c134700 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ef003a320 0x7f9ef003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(513..513 src has 266..513) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f9f0404ab20 con 0x7f9f0c134700 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 10 client.24536.objecter ms_dispatch 0x7f9f0c001550 osd_map(513..513 src has 266..513) v4 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 3 client.24536.objecter handle_osd_map got epochs [513,513] > 0 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 3 client.24536.objecter handle_osd_map decoding full epoch 513 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 20 client.24536.objecter dump_active .. 0 homeless 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f08020070 con 0x7f9f0c06aac0 2024-04-20T02:29:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f1141c700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ef003a320 0x7f9ef003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f027fc700 10 monclient: dump: 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.144+0000 7f9f1141c700 10 monclient: get_auth_request con 0x7f9ef003a320 auth_method 0 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9f08004250 con 0x7f9f0c06aac0 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 msgr2=0x7f9f0c11e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 0x7f9f0c11e5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f080098f0 tx=0x7f9f08004750 comp rx=0 tx=0).stop 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 msgr2=0x7f9f0c11e5a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.145+0000 7f9f027fc700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c06aac0 0x7f9f0c11e5a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.146+0000 7f9f027fc700 10 client.24536.objecter ms_handle_connect 0x7f9ef003a320 2024-04-20T02:29:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.146+0000 7f9f1141c700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ef003a320 0x7f9ef003c7e0 secure :-1 s=READY pgs=3294 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0a1b30 tx=0x7f9efc009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f13e81700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f13e81700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 0x7f9f0c11eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f13e81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1cca30 con 0x7f9f0c11eae0 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 0x7f9f0c11eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient: get_auth_request con 0x7f9f0c11eae0 auth_method 0 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.153+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f11c1d700 10 monclient(hunting): handle_auth_done global_id 84180 payload 824 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f11c1d700 1 -- 172.21.15.47:0/2199716346 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f0c0a5490 con 0x7f9f0c11eae0 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 client.24536.objecter ms_handle_connect 0x7f9f0c11eae0 2024-04-20T02:29:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f11c1d700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 0x7f9f0c11eef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009840 tx=0x7f9f08011000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 client.24536.objecter resend_mon_ops 2024-04-20T02:29:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f0802ce90 con 0x7f9f0c11eae0 2024-04-20T02:29:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 monclient: dump: 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f9f0802ca20 con 0x7f9f0c11eae0 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 msgr2=0x7f9f0c11eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 0x7f9f0c11eef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009840 tx=0x7f9f08011000 comp rx=0 tx=0).stop 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 msgr2=0x7f9f0c11eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.154+0000 7f9f027fc700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9f0c11eae0 0x7f9f0c11eef0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009840 tx=0x7f9f08011000 comp rx=0 tx=0).stop 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:00.172 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.173 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.174 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:00.175 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.176 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:00.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:00.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.185 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.187 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:00.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.189 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.190 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:00.191 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:00.192 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 10 librados: watch_flush enter 2024-04-20T02:29:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 10 librados: watch_flush exit 2024-04-20T02:29:00.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 20 client.24536.objecter shutdown clearing up homeless session... 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 10 client.24536.objecter successfully canceled tick 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ef003a320 msgr2=0x7f9ef003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9ef003a320 0x7f9ef003c7e0 secure :-1 s=READY pgs=3294 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0a1b30 tx=0x7f9efc009210 comp rx=0 tx=0).stop 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 10 monclient: shutdown 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0c134700 msgr2=0x7f9f0c133ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 --2- 172.21.15.47:0/2199716346 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0c134700 0x7f9f0c133ff0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009e60 tx=0x7f9f0400c3a0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f1241e700 1 -- 172.21.15.47:0/2199716346 reap_dead start 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f1241e700 1 -- 172.21.15.47:0/2199716346 reap_dead start 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 shutdown_connections 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 >> 172.21.15.47:0/2199716346 conn(0x7f9f0c0a8610 msgr2=0x7f9f0c070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 shutdown_connections 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 -- 172.21.15.47:0/2199716346 wait complete. 2024-04-20T02:29:00.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.155+0000 7f9f13e81700 1 librados: shutdown 2024-04-20T02:29:00.196 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:00.196 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:29:00.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:29:00.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.334+0000 7f7fd00de700 10 monclient: get_monmap_and_config 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.334+0000 7f7fd00de700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.334+0000 7f7fd00de700 10 monclient: monmap: 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:00.335787+0000 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:00.335787+0000 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.336 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.336 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:00.337 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.335+0000 7f7fd00de700 1 Processor -- start 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.335+0000 7f7fd00de700 1 -- start start 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.335+0000 7f7fd00de700 10 monclient: init 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient: _add_conns ranks=[8,3,9,2,6,0,10,4,1,7,5] 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 0x7f7fc80a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): picked mon.noname-e con 0x7f7fc80a2950 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc8137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): picked mon.noname-g con 0x7f7fc80a3270 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 0x7f7fc81384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): picked mon.noname-a con 0x7f7fc8138040 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc8072830 con 0x7f7fc8138040 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc809f150 con 0x7f7fc80a3270 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc809f710 con 0x7f7fc80a2950 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fd00de700 10 monclient(hunting): authenticate will time out at 2419.737106s 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 0x7f7fc80a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 0x7f7fc81384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 0x7f7fc80a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48548/0 (socket says 172.21.15.47:48548) 2024-04-20T02:29:00.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 0x7f7fc81384f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40542/0 (socket says 172.21.15.47:40542) 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 -- 172.21.15.47:0/157131147 learned_addr learned my addr 172.21.15.47:0/157131147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): get_auth_request con 0x7f7fc80a2950 auth_method 0 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): get_auth_request con 0x7f7fc8138040 auth_method 0 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcd679700 1 --2- 172.21.15.47:0/157131147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc8137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcd679700 10 monclient(hunting): get_auth_request con 0x7f7fc80a3270 auth_method 0 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcd679700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcd679700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcd679700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_done global_id 35675 payload 274 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 -- 172.21.15.47:0/157131147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 msgr2=0x7f7fc8137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/157131147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc8137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 -- 172.21.15.47:0/157131147 >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 msgr2=0x7f7fc81384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/157131147 >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 0x7f7fc81384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 monclient: found mon.noname-e 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 -- 172.21.15.47:0/157131147 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80751e0 con 0x7f7fc80a2950 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fce67b700 1 --2- 172.21.15.47:0/157131147 >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 0x7f7fc81384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/157131147 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 0x7f7fc80a2d30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=94c48f57962d99ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.336+0000 7f7fcce78700 1 -- 172.21.15.47:0/157131147 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc011aa0 con 0x7f7fc80a2950 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:00.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: dump: 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.361 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 5 monclient: authenticate success, global_id 35675 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 1 -- 172.21.15.47:0/157131147 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc011c00 con 0x7f7fc80a2950 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 1 -- 172.21.15.47:0/157131147 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc011d70 con 0x7f7fc80a2950 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 4 set_mon_vals no callback set 2024-04-20T02:29:00.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fcce78700 10 monclient: dump: 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 10 monclient: shutdown 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 -- 172.21.15.47:0/157131147 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 msgr2=0x7f7fc80a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 --2- 172.21.15.47:0/157131147 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 0x7f7fc80a2d30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 -- 172.21.15.47:0/157131147 shutdown_connections 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 --2- 172.21.15.47:0/157131147 >> v2:172.21.15.47:3300/0 conn(0x7f7fc8138040 0x7f7fc81384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 --2- 172.21.15.47:0/157131147 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7fc80a2950 0x7f7fc80a2d30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 --2- 172.21.15.47:0/157131147 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc8137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 -- 172.21.15.47:0/157131147 >> 172.21.15.47:0/157131147 conn(0x7f7fc80a7570 msgr2=0x7f7fc80a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 -- 172.21.15.47:0/157131147 shutdown_connections 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.337+0000 7f7fd00de700 1 -- 172.21.15.47:0/157131147 wait complete. 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 10 monclient: monmap: 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:00.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 1 librados: starting msgr at 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 1 librados: starting objecter 2024-04-20T02:29:00.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 1 Processor -- start 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 1 -- start start 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 1 librados: setting wanted keys 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 1 librados: calling monclient init 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.338+0000 7f7fd00de700 10 monclient: init 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient: _add_conns ranks=[1,3,4,8,2,6,5,7,0] 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc80a2950 0x7f7fc80716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): picked mon.noname-f con 0x7f7fc80a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc81371d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): picked mon.noname-g con 0x7f7fc80a3270 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7fc8138040 0x7f7fc81c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): picked mon.noname-c con 0x7f7fc8138040 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc809dde0 con 0x7f7fc80a2950 2024-04-20T02:29:00.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81242f0 con 0x7f7fc80a3270 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81245d0 con 0x7f7fc8138040 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fd00de700 10 monclient(hunting): authenticate will time out at 2419.740355s 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7fc8138040 0x7f7fc81c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7fc8138040 0x7f7fc81c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:45286/0 (socket says 172.21.15.47:45286) 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 -- 172.21.15.47:0/370701594 learned_addr learned my addr 172.21.15.47:0/370701594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): get_auth_request con 0x7f7fc8138040 auth_method 0 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc81371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc80a2950 0x7f7fc80716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): get_auth_request con 0x7f7fc80a2950 auth_method 0 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): get_auth_request con 0x7f7fc80a3270 auth_method 0 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fcd679700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient(hunting): handle_auth_done global_id 84163 payload 824 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 msgr2=0x7f7fc81371d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7fc80a3270 0x7f7fc81371d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc80a2950 msgr2=0x7f7fc80716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fc80a2950 0x7f7fc80716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 monclient: found mon.noname-c 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.339+0000 7f7fce67b700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0009a70 con 0x7f7fc8138040 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fce67b700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7fc8138040 0x7f7fc81c8d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000efc0 tx=0x7f7fc000c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 client.?.objecter ms_handle_connect 0x7f7fc8138040 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fc0014480 con 0x7f7fc8138040 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: dump: 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.372 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 5 monclient: authenticate success, global_id 84163 2024-04-20T02:29:00.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000f040 con 0x7f7fc8138040 2024-04-20T02:29:00.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fc0014a70 con 0x7f7fc8138040 2024-04-20T02:29:00.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 10 monclient: _renew_subs 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc009550 con 0x7f7fc8138040 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fbbfff700 4 set_mon_vals no callback set 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fba7fc700 10 monclient: dump: 2024-04-20T02:29:00.375 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.376 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.376 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.376 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.376 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.376 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.376 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.378 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 10 client.84163.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 10 monclient: _renew_subs 2024-04-20T02:29:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:00.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc81cc710 con 0x7f7fc8138040 2024-04-20T02:29:00.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.340+0000 7f7fd00de700 1 librados: init done 2024-04-20T02:29:00.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fd00de700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fd00de700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 0x7f7fc802bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7fc0014bd0 con 0x7f7fc8138040 2024-04-20T02:29:00.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc8068310 con 0x7f7fc806aac0 2024-04-20T02:29:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7fac03a2f0 0x7f7fac03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 0x7f7fc802bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient: get_auth_request con 0x7f7fc806aac0 auth_method 0 2024-04-20T02:29:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(513..513 src has 266..513) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f7fc004ac60 con 0x7f7fc8138040 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 10 client.84163.objecter ms_dispatch 0x7f7fc8001550 osd_map(513..513 src has 266..513) v4 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 3 client.84163.objecter handle_osd_map got epochs [513,513] > 0 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 3 client.84163.objecter handle_osd_map decoding full epoch 513 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fba7fc700 20 client.84163.objecter dump_active .. 0 homeless 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcd679700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7fac03a2f0 0x7f7fac03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcd679700 10 monclient: get_auth_request con 0x7f7fac03a2f0 auth_method 0 2024-04-20T02:29:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_done global_id 84137 payload 824 2024-04-20T02:29:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.341+0000 7f7fcde7a700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fc8124460 con 0x7f7fc806aac0 2024-04-20T02:29:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 client.84163.objecter ms_handle_connect 0x7f7fc806aac0 2024-04-20T02:29:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 client.84163.objecter resend_mon_ops 2024-04-20T02:29:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 0x7f7fc802bbc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc01f070 con 0x7f7fc806aac0 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fcd679700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7fac03a2f0 0x7f7fac03c7b0 secure :-1 s=READY pgs=3295 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8071df0 tx=0x7f7fc4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 monclient: dump: 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.383 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.384 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.385 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 client.84163.objecter ms_handle_connect 0x7f7fac03a2f0 2024-04-20T02:29:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7fbc0044f0 con 0x7f7fc806aac0 2024-04-20T02:29:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 msgr2=0x7f7fc802bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 0x7f7fc802bbc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc004750 comp rx=0 tx=0).stop 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 msgr2=0x7f7fc802bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.342+0000 7f7fba7fc700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc806aac0 0x7f7fc802bbc0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098f0 tx=0x7f7fbc004750 comp rx=0 tx=0).stop 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fd00de700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fd00de700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 0x7f7fc802c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fd00de700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc81cc900 con 0x7f7fc81235b0 2024-04-20T02:29:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 0x7f7fc802c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fcde7a700 10 monclient: get_auth_request con 0x7f7fc81235b0 auth_method 0 2024-04-20T02:29:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fcde7a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fcde7a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.350+0000 7f7fcde7a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fcde7a700 10 monclient(hunting): handle_auth_done global_id 84146 payload 824 2024-04-20T02:29:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fcde7a700 1 -- 172.21.15.47:0/370701594 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7fc809ee70 con 0x7f7fc81235b0 2024-04-20T02:29:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 client.84163.objecter ms_handle_connect 0x7f7fc81235b0 2024-04-20T02:29:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fcde7a700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 0x7f7fc802c100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009130 tx=0x7f7fbc011000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 client.84163.objecter resend_mon_ops 2024-04-20T02:29:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc0093d0 con 0x7f7fc81235b0 2024-04-20T02:29:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 monclient: dump: 2024-04-20T02:29:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.393 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f7fbc003d40 con 0x7f7fc81235b0 2024-04-20T02:29:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 msgr2=0x7f7fc802c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 0x7f7fc802c100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009130 tx=0x7f7fbc011000 comp rx=0 tx=0).stop 2024-04-20T02:29:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 msgr2=0x7f7fc802c100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.351+0000 7f7fba7fc700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7fc81235b0 0x7f7fc802c100 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009130 tx=0x7f7fbc011000 comp rx=0 tx=0).stop 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:00.395 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:00.396 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.397 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.398 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.399 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:00.400 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:00.401 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.402 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.403 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.404 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.405 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.407 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:00.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:00.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:00.412 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.415 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.417 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:00.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:00.419 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 10 librados: watch_flush enter 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 10 librados: watch_flush exit 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 20 client.84163.objecter shutdown clearing up homeless session... 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 10 client.84163.objecter successfully canceled tick 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7fac03a2f0 msgr2=0x7f7fac03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7fac03a2f0 0x7f7fac03c7b0 secure :-1 s=READY pgs=3295 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8071df0 tx=0x7f7fc4008040 comp rx=0 tx=0).stop 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 10 monclient: shutdown 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7fc8138040 msgr2=0x7f7fc81c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 --2- 172.21.15.47:0/370701594 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7fc8138040 0x7f7fc81c8d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000efc0 tx=0x7f7fc000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fce67b700 1 -- 172.21.15.47:0/370701594 reap_dead start 2024-04-20T02:29:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fce67b700 1 -- 172.21.15.47:0/370701594 reap_dead start 2024-04-20T02:29:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 shutdown_connections 2024-04-20T02:29:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 >> 172.21.15.47:0/370701594 conn(0x7f7fc80a7570 msgr2=0x7f7fc809c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 shutdown_connections 2024-04-20T02:29:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 -- 172.21.15.47:0/370701594 wait complete. 2024-04-20T02:29:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.352+0000 7f7fd00de700 1 librados: shutdown 2024-04-20T02:29:00.422 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:00.422 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:29:00.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.561+0000 7fc008884700 10 monclient: get_monmap_and_config 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.561+0000 7fc008884700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.561+0000 7fc008884700 10 monclient: monmap: 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:00.563092+0000 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:00.563092+0000 2024-04-20T02:29:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:00.563 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.562+0000 7fc008884700 1 Processor -- start 2024-04-20T02:29:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.562+0000 7fc008884700 1 -- start start 2024-04-20T02:29:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.562+0000 7fc008884700 10 monclient: init 2024-04-20T02:29:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient: _add_conns ranks=[1,0,2,9,3,8,7,10,4,5,6] 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc0040a1510 addr v1:172.21.15.179:6789/0 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc00409f9a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 0x7fc00413e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient(hunting): picked mon.noname-b con 0x7fc00409fe00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040751e0 con 0x7fc00409f9a0 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040720a0 con 0x7fc0040a1510 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc004070640 con 0x7fc00409fe00 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc008884700 10 monclient(hunting): authenticate will time out at 2419.964416s 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 0x7fc00413e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc001d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc00409f9a0 0x7fc00413a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47248/0 (socket says 172.21.15.47:47248) 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 0x7fc00413e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54068/0 (socket says 172.21.15.47:54068) 2024-04-20T02:29:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc001d9b700 1 -- 172.21.15.47:0/754008516 learned_addr learned my addr 172.21.15.47:0/754008516 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): get_auth_request con 0x7fc00409fe00 auth_method 0 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient(hunting): handle_auth_done global_id 24545 payload 274 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 1 -- 172.21.15.47:0/754008516 >> v1:172.21.15.179:6789/0 conn(0x7fc0040a1510 legacy=0x7fc0040a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 1 -- 172.21.15.47:0/754008516 >> v1:172.21.15.47:6789/0 conn(0x7fc00409f9a0 legacy=0x7fc00413a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 1 monclient: found mon.noname-b 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc00159a700 1 -- 172.21.15.47:0/754008516 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730094171 0 0) 0x7fc0040751e0 con 0x7fc00409f9a0 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc002d9d700 1 -- 172.21.15.47:0/754008516 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004071d30 con 0x7fc00409fe00 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.563+0000 7fc00159a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc002d9d700 1 --2- 172.21.15.47:0/754008516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 0x7fc00413e080 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009dc0 tx=0x7fbffc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db7dcfae53a7c593 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 1 -- 172.21.15.47:0/754008516 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbffc00e070 con 0x7fc00409fe00 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: dump: 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 1 -- 172.21.15.47:0/754008516 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc00bbd0 con 0x7fc00409fe00 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 5 monclient: authenticate success, global_id 24545 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 1 -- 172.21.15.47:0/754008516 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbffc00bd40 con 0x7fc00409fe00 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 4 set_mon_vals no callback set 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc00159a700 10 monclient: dump: 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.578 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 10 monclient: shutdown 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 1 -- 172.21.15.47:0/754008516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 msgr2=0x7fc00413e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 1 --2- 172.21.15.47:0/754008516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 0x7fc00413e080 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009dc0 tx=0x7fbffc00b330 comp rx=0 tx=0).stop 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.564+0000 7fc008884700 1 -- 172.21.15.47:0/754008516 shutdown_connections 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.565+0000 7fc008884700 1 --2- 172.21.15.47:0/754008516 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc00409fe00 0x7fc00413e080 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.565+0000 7fc008884700 1 -- 172.21.15.47:0/754008516 >> 172.21.15.47:0/754008516 conn(0x7fc0040a8510 msgr2=0x7fc0040a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.565+0000 7fc008884700 1 -- 172.21.15.47:0/754008516 shutdown_connections 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.565+0000 7fc008884700 1 -- 172.21.15.47:0/754008516 wait complete. 2024-04-20T02:29:00.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.565+0000 7fc008884700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.565+0000 7fc008884700 10 monclient: monmap: 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.580 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 librados: starting msgr at 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 librados: starting objecter 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 Processor -- start 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 -- start start 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 librados: setting wanted keys 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 librados: calling monclient init 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient: init 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient: _add_conns ranks=[8,3,6,0,4,7,5,2,1] 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 0x7fc0041cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): picked mon.noname-e con 0x7fc00409f9a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc00409fe00 0x7fc0041ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): picked mon.noname-g con 0x7fc00409fe00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:00.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0040a1510 0x7fc004124220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): picked mon.noname-d con 0x7fc0040a1510 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc004127ae0 con 0x7fc00409fe00 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040a2130 con 0x7fc0040a1510 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041268a0 con 0x7fc00409f9a0 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc008884700 10 monclient(hunting): authenticate will time out at 2419.967789s 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 0x7fc0041cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0040a1510 0x7fc004124220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 0x7fc0041cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48564/0 (socket says 172.21.15.47:48564) 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 1 -- 172.21.15.47:0/1867303546 learned_addr learned my addr 172.21.15.47:0/1867303546 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc002d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0040a1510 0x7fc004124220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38802/0 (socket says 172.21.15.47:38802) 2024-04-20T02:29:00.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 10 monclient(hunting): get_auth_request con 0x7fc00409f9a0 auth_method 0 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc00259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc002d9d700 10 monclient(hunting): get_auth_request con 0x7fc0040a1510 auth_method 0 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc002d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc002d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.566+0000 7fc002d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc00409fe00 0x7fc0041ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc002d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc002d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc002d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): get_auth_request con 0x7fc00409fe00 auth_method 0 2024-04-20T02:29:00.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 10 monclient(hunting): handle_auth_done global_id 35684 payload 824 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0040a1510 msgr2=0x7fc004124220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0040a1510 0x7fc004124220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc00409fe00 msgr2=0x7fc0041ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc00409fe00 0x7fc0041ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 monclient: found mon.noname-e 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:00.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009a70 con 0x7fc00409f9a0 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc002d9d700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc0040a1510 0x7fc004124220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc00259c700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 0x7fc0041cc7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbff400c450 tx=0x7fbff400c760 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 client.?.objecter ms_handle_connect 0x7fc00409f9a0 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff400ce60 con 0x7fc00409f9a0 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: dump: 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.585 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4005540 con 0x7fc00409f9a0 2024-04-20T02:29:00.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 5 monclient: authenticate success, global_id 35684 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 10 monclient: _renew_subs 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004127800 con 0x7fc00409f9a0 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff4003af0 con 0x7fc00409f9a0 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fbff2ffd700 10 monclient: dump: 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.587 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc000d99700 4 set_mon_vals no callback set 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 10 client.35684.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 10 monclient: _renew_subs 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc004125cd0 con 0x7fc00409f9a0 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.567+0000 7fc008884700 1 librados: init done 2024-04-20T02:29:00.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc008884700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc008884700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 0x7fc004124760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc008884700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040682f0 con 0x7fc00406aac0 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbff4003c70 con 0x7fc00409f9a0 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 0x7fc004124760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient: get_auth_request con 0x7fc00406aac0 auth_method 0 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fbff2ffd700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfe403a320 0x7fbfe403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(513..513 src has 266..513) v4 ==== 6557+0+0 (secure 0 0 0) 0x7fbff4049720 con 0x7fc00409f9a0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fbff2ffd700 10 client.35684.objecter ms_dispatch 0x7fc004001550 osd_map(513..513 src has 266..513) v4 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fbff2ffd700 3 client.35684.objecter handle_osd_map got epochs [513,513] > 0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.568+0000 7fbff2ffd700 3 client.35684.objecter handle_osd_map decoding full epoch 513 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 20 client.35684.objecter dump_active .. 0 homeless 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fc002d9d700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfe403a320 0x7fbfe403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fc001d9b700 10 monclient(hunting): handle_auth_done global_id 84148 payload 824 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fc001d9b700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0040a7ac0 con 0x7fc00406aac0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fc002d9d700 10 monclient: get_auth_request con 0x7fbfe403a320 auth_method 0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 client.35684.objecter ms_handle_connect 0x7fc00406aac0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fc001d9b700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 0x7fc004124760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbff8007fd0 tx=0x7fbff80018f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 client.35684.objecter resend_mon_ops 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff8017070 con 0x7fc00406aac0 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:00.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 monclient: dump: 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.591 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 client.35684.objecter ms_handle_connect 0x7fbfe403a320 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fc002d9d700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfe403a320 0x7fbfe403c7e0 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b370 tx=0x7fbffc011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbff8012560 con 0x7fc00406aac0 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 msgr2=0x7fc004124760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 0x7fc004124760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbff8007fd0 tx=0x7fbff80018f0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 msgr2=0x7fc004124760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:00.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.569+0000 7fbff2ffd700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc00406aac0 0x7fc004124760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.577+0000 7fc008884700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.577+0000 7fc008884700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 0x7fc0041250b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.577+0000 7fc008884700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.577+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc00406a400 con 0x7fc004124ca0 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 0x7fc0041250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient: get_auth_request con 0x7fc004124ca0 auth_method 0 2024-04-20T02:29:00.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 10 monclient(hunting): handle_auth_done global_id 84157 payload 824 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 1 -- 172.21.15.47:0/1867303546 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc00406af90 con 0x7fc004124ca0 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fc001d9b700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 0x7fc0041250b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc00409f5f0 tx=0x7fbff8000ed0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fbff2ffd700 10 client.35684.objecter ms_handle_connect 0x7fc004124ca0 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fbff2ffd700 10 client.35684.objecter resend_mon_ops 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff8022070 con 0x7fc004124ca0 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fbff2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fbff2ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.578+0000 7fbff2ffd700 10 monclient: dump: 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fbff8013600 con 0x7fc004124ca0 2024-04-20T02:29:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 msgr2=0x7fc0041250b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 0x7fc0041250b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc00409f5f0 tx=0x7fbff8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 msgr2=0x7fc0041250b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fbff2ffd700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc004124ca0 0x7fc0041250b0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc00409f5f0 tx=0x7fbff8000ed0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:00.597 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:00.598 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.599 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:00.600 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:00.601 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.603 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:00.604 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.606 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:00.607 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.609 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:00.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.611 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.612 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:00.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:00.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.616 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:00.617 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 10 librados: watch_flush enter 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 10 librados: watch_flush exit 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 20 client.35684.objecter shutdown clearing up homeless session... 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 10 client.35684.objecter successfully canceled tick 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfe403a320 msgr2=0x7fbfe403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbfe403a320 0x7fbfe403c7e0 secure :-1 s=READY pgs=3296 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b370 tx=0x7fbffc011000 comp rx=0 tx=0).stop 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 10 monclient: shutdown 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 msgr2=0x7fc0041cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 0x7fc0041cc7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbff400c450 tx=0x7fbff400c760 comp rx=0 tx=0).stop 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc002d9d700 1 -- 172.21.15.47:0/1867303546 reap_dead start 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 shutdown_connections 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 --2- 172.21.15.47:0/1867303546 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc00409f9a0 0x7fc0041cc7e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 >> 172.21.15.47:0/1867303546 conn(0x7fc0040a8510 msgr2=0x7fc0040a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 shutdown_connections 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 -- 172.21.15.47:0/1867303546 wait complete. 2024-04-20T02:29:00.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.579+0000 7fc008884700 1 librados: shutdown 2024-04-20T02:29:00.620 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:00.620 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:29:00.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:29:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.757+0000 7fc8fba44700 10 monclient: get_monmap_and_config 2024-04-20T02:29:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.757+0000 7fc8fba44700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.757+0000 7fc8fba44700 10 monclient: monmap: 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:00.758544+0000 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:00.758544+0000 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.758 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:00.759 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:00.759 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 Processor -- start 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 -- start start 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient: init 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient: _add_conns ranks=[2,1,6,3,4,10,7,5,8,9,0] 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f40a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): picked mon.noname-b con 0x7fc8f40a5080 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc8f40a1a80 addr v1:172.21.15.179:6789/0 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 0x7fc8f413da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): picked mon.noname-d con 0x7fc8f40a3f90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f40751e0 con 0x7fc8f40a1a80 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40a5460 con 0x7fc8f40a5080 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4070b10 con 0x7fc8f40a3f90 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8fba44700 10 monclient(hunting): authenticate will time out at 2420.159878s 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8f9fe1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 0x7fc8f413da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.758+0000 7fc8f97e0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f40a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 0x7fc8f413da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38806/0 (socket says 172.21.15.47:38806) 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f40a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54084/0 (socket says 172.21.15.47:54084) 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 -- 172.21.15.47:0/2977499167 learned_addr learned my addr 172.21.15.47:0/2977499167 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): get_auth_request con 0x7fc8f40a3f90 auth_method 0 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): get_auth_request con 0x7fc8f40a5080 auth_method 0 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_done global_id 54915 payload 274 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 -- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 msgr2=0x7fc8f40a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 --2- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f40a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 -- 172.21.15.47:0/2977499167 >> v1:172.21.15.179:6789/0 conn(0x7fc8f40a1a80 legacy=0x7fc8f40a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 monclient: found mon.noname-d 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 -- 172.21.15.47:0/2977499167 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4136370 con 0x7fc8f40a3f90 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f97e0700 1 --2- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f40a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8f9fe1700 1 --2- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 0x7fc8f413da50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009e80 tx=0x7fc8ec00b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=316b14601461b9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 1 -- 172.21.15.47:0/2977499167 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8ec00e070 con 0x7fc8f40a3f90 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: dump: 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.771 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8fba44700 5 monclient: authenticate success, global_id 54915 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 1 -- 172.21.15.47:0/2977499167 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec0099b0 con 0x7fc8f40a3f90 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8fba44700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8fba44700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:00.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8fba44700 4 set_mon_vals no callback set 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 1 -- 172.21.15.47:0/2977499167 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8ec00bd80 con 0x7fc8f40a3f90 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.759+0000 7fc8ebfff700 10 monclient: dump: 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.773 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 10 monclient: shutdown 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 -- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 msgr2=0x7fc8f413da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 0x7fc8f413da50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009e80 tx=0x7fc8ec00b330 comp rx=0 tx=0).stop 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 -- 172.21.15.47:0/2977499167 shutdown_connections 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8f40a3f90 0x7fc8f413da50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2977499167 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f40a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 -- 172.21.15.47:0/2977499167 >> 172.21.15.47:0/2977499167 conn(0x7fc8f40a84a0 msgr2=0x7fc8f40a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 -- 172.21.15.47:0/2977499167 shutdown_connections 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.760+0000 7fc8fba44700 1 -- 172.21.15.47:0/2977499167 wait complete. 2024-04-20T02:29:00.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 10 monclient: build_initial_monmap 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 10 monclient: monmap: 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:00.775 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 1 librados: starting msgr at 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 1 librados: starting objecter 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 1 Processor -- start 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 1 -- start start 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 1 librados: setting wanted keys 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 1 librados: calling monclient init 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.761+0000 7fc8fba44700 10 monclient: init 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient: _add_conns ranks=[0,3,2,6,7,4,5,1,8] 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 0x7fc8f4134b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): picked mon.noname-a con 0x7fc8f40a1a80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8f40a3f90 0x7fc8f41c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): picked mon.noname-g con 0x7fc8f40a3f90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f41c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): picked mon.noname-b con 0x7fc8f40a5080 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4123f70 con 0x7fc8f40a1a80 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40712d0 con 0x7fc8f40a5080 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f406e260 con 0x7fc8f40a3f90 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8fba44700 10 monclient(hunting): authenticate will time out at 2420.163286s 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 0x7fc8f4134b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 0x7fc8f4134b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40558/0 (socket says 172.21.15.47:40558) 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 -- 172.21.15.47:0/2468880529 learned_addr learned my addr 172.21.15.47:0/2468880529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): get_auth_request con 0x7fc8f40a1a80 auth_method 0 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f41c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): get_auth_request con 0x7fc8f40a5080 auth_method 0 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f8fdf700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8f40a3f90 0x7fc8f41c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f8fdf700 10 monclient(hunting): get_auth_request con 0x7fc8f40a3f90 auth_method 0 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f8fdf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f8fdf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:00.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f8fdf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient(hunting): handle_auth_done global_id 36360 payload 824 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 -- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8f40a3f90 msgr2=0x7fc8f41c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8f40a3f90 0x7fc8f41c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 -- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 msgr2=0x7fc8f41c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f41c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient: _finish_hunting 0 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 monclient: found mon.noname-a 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 -- 172.21.15.47:0/2468880529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00094a0 con 0x7fc8f40a1a80 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f8fdf700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8f40a3f90 0x7fc8f41c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f9fe1700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f41c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8f97e0700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 0x7fc8f4134b30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000ebe0 tx=0x7fc8f000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.762+0000 7fc8e9ffb700 10 client.?.objecter ms_handle_connect 0x7fc8f40a1a80 2024-04-20T02:29:00.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8f0009800 con 0x7fc8f40a1a80 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: dump: 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.780 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f000cd80 con 0x7fc8f40a1a80 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 5 monclient: authenticate success, global_id 36360 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 10 monclient: _renew_subs 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8f00138e0 con 0x7fc8f40a1a80 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:00.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec009b30 con 0x7fc8f40a1a80 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8eb7fe700 4 set_mon_vals no callback set 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8e9ffb700 10 monclient: dump: 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:00.782 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 10 client.36360.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 10 monclient: _renew_subs 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f4123c90 con 0x7fc8f40a1a80 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.763+0000 7fc8fba44700 1 librados: init done 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc8f0013470 con 0x7fc8f40a1a80 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc8d803a2f0 0x7fc8d803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(513..513 src has 266..513) v4 ==== 6557+0+0 (secure 0 0 0) 0x7fc8f004a960 con 0x7fc8f40a1a80 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 10 client.36360.objecter ms_dispatch 0x7fc8f4001550 osd_map(513..513 src has 266..513) v4 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 3 client.36360.objecter handle_osd_map got epochs [513,513] > 0 2024-04-20T02:29:00.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 3 client.36360.objecter handle_osd_map decoding full epoch 513 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8fba44700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8fba44700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8fba44700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8e9ffb700 20 client.36360.objecter dump_active .. 0 homeless 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f4124190 con 0x7fc8f40a1a80 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8f8fdf700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc8d803a2f0 0x7fc8d803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.764+0000 7fc8f8fdf700 10 monclient: get_auth_request con 0x7fc8d803a2f0 auth_method 0 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.765+0000 7fc8e9ffb700 10 client.36360.objecter ms_handle_connect 0x7fc8d803a2f0 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.765+0000 7fc8f8fdf700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc8d803a2f0 0x7fc8d803c7b0 secure :-1 s=READY pgs=3297 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406dde0 tx=0x7fc8e4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.769+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc8f0013720 con 0x7fc8f40a1a80 2024-04-20T02:29:00.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.769+0000 7fc8e9ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:00.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:00.769+0000 7fc8e9ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:01.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.080+0000 7fc8fba44700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:29:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.080+0000 7fc8fba44700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:29:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.080+0000 7fc8fba44700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:01.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.080+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fc8f41243e0 con 0x7fc8f40a1a80 2024-04-20T02:29:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.082+0000 7fc8e9ffb700 1 -- 172.21.15.47:0/2468880529 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fc8f0017070 con 0x7fc8f40a1a80 2024-04-20T02:29:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.082+0000 7fc8e9ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:29:01.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.082+0000 7fc8e9ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=10,logm=61} crc {auth=1670437890,config=692619927,health=2746330634,kv=1530195301,logm=3266753296}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=2242190795,config=692619927,health=3443741831,kv=1530195301,logm=2700436821}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=3674541701,config=692619927,health=3827132391,kv=1530195301,logm=1174165966}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=15,config=2,health=12,kv=11,logm=60} crc {auth=3132556660,config=692619927,health=131042378,kv=1530195301,logm=887955414}) 2024-04-20T02:29:01.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=1,health=12,kv=11,logm=60} crc {auth=1526417203,config=3678701417,health=131042378,kv=1530195301,logm=2960615280}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=287791761,config=692619927,health=2102423746,kv=3859881480,logm=1745324053}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=17,config=1,health=10,kv=10,logm=62} crc {auth=287791761,config=3678701417,health=1089931575,kv=3093236402,logm=1723646161}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=287791761,config=692619927,health=131042378,kv=2592093630,logm=3929567454}) 2024-04-20T02:29:01.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.083+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=11,kv=10,logm=60} crc {auth=287791761,config=692619927,health=1356226261,kv=435135402,logm=1059061225}) 2024-04-20T02:29:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 10 librados: watch_flush enter 2024-04-20T02:29:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 10 librados: watch_flush exit 2024-04-20T02:29:01.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 20 client.36360.objecter shutdown clearing up homeless session... 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 10 client.36360.objecter successfully canceled tick 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc8d803a2f0 msgr2=0x7fc8d803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc8d803a2f0 0x7fc8d803c7b0 secure :-1 s=READY pgs=3297 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406dde0 tx=0x7fc8e4009040 comp rx=0 tx=0).stop 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 10 monclient: shutdown 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 msgr2=0x7fc8f4134b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 0x7fc8f4134b30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000ebe0 tx=0x7fc8f000eef0 comp rx=0 tx=0).stop 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 shutdown_connections 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc8d803a2f0 0x7fc8d803c7b0 unknown :-1 s=CLOSED pgs=3297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8f40a1a80 0x7fc8f4134b30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8f40a5080 0x7fc8f41c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 --2- 172.21.15.47:0/2468880529 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc8f40a3f90 0x7fc8f41c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:01.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 >> 172.21.15.47:0/2468880529 conn(0x7fc8f40a84a0 msgr2=0x7fc8f41bd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:01.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2461034602}) 2024-04-20T02:29:01.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 shutdown_connections 2024-04-20T02:29:01.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fc8fba44700 1 -- 172.21.15.47:0/2468880529 wait complete. 2024-04-20T02:29:01.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4206300346}) 2024-04-20T02:29:01.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.085+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fc8fba44700 1 librados: shutdown 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3017234779}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3480234617}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1164205384}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3276625689}) 2024-04-20T02:29:01.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3130843866}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1296491510}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.086+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1074334250}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=61595559}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3832577829}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1257885676}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1913355798}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1454106541}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=498159754}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3928578909}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3581122826}) 2024-04-20T02:29:01.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.088+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3346086930}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1742115998}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=662270355}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2802311257}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3696673254}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2845694587}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2637069161}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=704856379}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1338102179}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.091+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1041964539}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=758353355}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=68759830}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=853118370}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2880285006}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2453757941}) 2024-04-20T02:29:01.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=411101673}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=827402894}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3835687333}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.094+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3681728895}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4870169}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1248353417}) 2024-04-20T02:29:01.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4045281833}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4017280280}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4074692387}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1933760825}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1966399871}) 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4077932365}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.096+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3459410840}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3641202494}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2840251833}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=69216578}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=887137322}) 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503584798}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3423724647}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3828800085}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=591861505}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.098+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1240596781}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1292763176}) 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2246285108}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3270868030}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=163035524}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3956340252}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4231792405}) 2024-04-20T02:29:01.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2266423035}) 2024-04-20T02:29:01.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3373965514}) 2024-04-20T02:29:01.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.099+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1276230800}) 2024-04-20T02:29:01.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681389928}) 2024-04-20T02:29:01.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=430804482}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3199303265}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3985886511}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3377504085}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3620716362}) 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2822756860}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2740553813}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.101+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1509538551}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1798828645}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785206}) 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.103+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1727158741}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3875840999}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4073289388}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3383255688}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1270592878}) 2024-04-20T02:29:01.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2756303396}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.104+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3279908253}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.105+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.105+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.105+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3676468493}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.105+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.105+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4023113870}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2360369127}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2325687365}) 2024-04-20T02:29:01.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1142969653}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=861967262}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3435293742}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531518038}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.106+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=807317303}) 2024-04-20T02:29:01.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3321054064}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3806856254}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2573498635}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4282633789}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4033582038}) 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3446646234}) 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.107+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=172797831}) 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.108+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.108+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575617398}) 2024-04-20T02:29:01.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.108+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=698561091}) 2024-04-20T02:29:01.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:29:01.235 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.235 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1799072266}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2644315403}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3855917792}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3216707333}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=704385368}) 2024-04-20T02:29:01.236 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3860596830}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=973550241}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=814654786}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.109+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=150488684}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4176756580}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.237 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3345349997}) 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2682907134}) 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=610423276}) 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.238 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988166611}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=840816390}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2792003328}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852797709}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.111+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647399641}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=176481780}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.239 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4083819858}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=689490178}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=986554046}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1569188004}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3295783549}) 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.240 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4022428638}) 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2141566481}) 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.113+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3849784724}) 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.241 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3887040992}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=952358959}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3131438558}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=418323309}) 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.242 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1568119946}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3290670374}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=108204848}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231730208}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.114+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4246399822}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568887817}) 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.243 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1625432908}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1652802620}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3514833015}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1762556601}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4005266912}) 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.244 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2837823459}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3148011178}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.116+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=245000352}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1288284946}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1661105995}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4126545844}) 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.245 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1467906435}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=670968992}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1641744845}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4246069098}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3207908123}) 2024-04-20T02:29:01.246 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.118+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1454094780}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230667614}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2507130246}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2669746363}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3935139784}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.247 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=86264758}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3059139220}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1894398992}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1779509299}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.119+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1269999744}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3211234983}) 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.248 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1988005882}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=429107629}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2298602241}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2018443795}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1690867818}) 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.249 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1695239868}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3682399672}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.121+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=412549285}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1555506510}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=605257051}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1100506077}) 2024-04-20T02:29:01.250 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3206496457}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4081827022}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=260461997}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1901141248}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730082578}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.122+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3114274080}) 2024-04-20T02:29:01.251 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3644639753}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=394179347}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2860772488}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2281253468}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2596784440}) 2024-04-20T02:29:01.252 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3613169581}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1582962562}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.124+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739247593}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.125+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2415773424}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1383769446}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4057896348}) 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.253 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=58093124}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3429377191}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=31385214}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2331409114}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2654978431}) 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.254 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284359549}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.126+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=262918317}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=242728413}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2837257388}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.127+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1037284911}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1349317325}) 2024-04-20T02:29:01.255 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1946396456}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=20972269}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1344807189}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323185093}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.128+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=724332404}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1030706761}) 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.256 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2059844196}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2341190674}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=919481725}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2916180277}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2631615402}) 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.257 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=621230632}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2091959707}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.129+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=332197549}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2665328881}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4089974310}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820741614}) 2024-04-20T02:29:01.258 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.131+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=943724867}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=318509035}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=161137148}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3534031707}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1994482031}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.132+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692181089}) 2024-04-20T02:29:01.259 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4011792997}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2515101199}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1113436361}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1018730108}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=190592781}) 2024-04-20T02:29:01.260 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702425160}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2203054718}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1035954322}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.133+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066913905}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1879054886}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=407456244}) 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.261 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2908541721}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=436630015}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3388858910}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1909423802}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2089407412}) 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.262 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1354665901}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.135+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2648936882}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4055872168}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1752647689}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3598154949}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3415723982}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.263 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2950852644}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=753734955}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.136+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3151995257}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2997805796}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.137+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1430690584}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4269923441}) 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.264 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2860142967}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1576587159}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=110403611}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1205718014}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3304709463}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.265 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1609658155}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1639386473}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.139+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3161887987}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=956373609}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2033440116}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=254449913}) 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.266 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=928838833}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=299073782}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2754297541}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3964510095}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1198578762}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.141+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2830239614}) 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.267 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1828047998}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=319853813}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2210587630}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2304691456}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=612083974}) 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.268 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1689092068}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.146+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=410753371}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1194929510}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.147+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2692540216}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1895000976}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3064646419}) 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.148+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.269 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2901789588}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2799065790}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2884602319}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3893927384}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1956771015}) 2024-04-20T02:29:01.270 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148771148}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.149+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4054596024}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3041642877}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3703917951}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2336394982}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=913591582}) 2024-04-20T02:29:01.271 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3615570223}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2345733265}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.150+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3487525861}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4023276422}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.151+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4259541783}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1731921093}) 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.272 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3476478275}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=938967184}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=439364346}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3176041663}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3428298841}) 2024-04-20T02:29:01.273 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2278795357}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938658954}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.152+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1414078496}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662001832}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2499233162}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2080305112}) 2024-04-20T02:29:01.274 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=960671143}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1566269415}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3412328305}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1765588904}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2492831272}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.154+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=308260253}) 2024-04-20T02:29:01.275 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1092280217}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3485133119}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4119088340}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1732920881}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=437949277}) 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.276 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=450858325}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4121636053}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929069585}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.155+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=873361282}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=286051162}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=942143187}) 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.277 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=644848423}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4170995243}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=912748496}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3326675587}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=549971131}) 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.278 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716005176}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.157+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337945704}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2907857402}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3243770699}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.158+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=197248225}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3728437907}) 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.279 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2404259081}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4178135686}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1435123545}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3304475403}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.159+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1067602620}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1919605159}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.280 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2858793052}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1977627953}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3236277052}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.160+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171023289}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1921718926}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=388206080}) 2024-04-20T02:29:01.281 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2652880940}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.161+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=699988029}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295999537}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3414278105}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=645831590}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.282 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1467242070}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2527242170}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1968638228}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1650900272}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1540838324}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.162+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3569516138}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.283 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2833868940}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3372119795}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2834329394}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=299287327}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4269381637}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.284 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=942654877}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=861720820}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3683461729}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.164+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=962839357}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3156523454}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4194447559}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=967752553}) 2024-04-20T02:29:01.285 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.165+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=90707228}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2301280042}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4159339713}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=959269212}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165767262}) 2024-04-20T02:29:01.286 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.166+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3503282645}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1561771753}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884240094}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=461855567}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1543326346}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.287 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3314382007}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=215471740}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1945769477}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486596150}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.167+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2377058011}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2350601060}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.288 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=229795570}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2682884762}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3295877477}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=325288318}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4147559091}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.289 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=806831069}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3298273842}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.169+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2640909924}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863648772}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2190875412}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367019207}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.290 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=271347314}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1144040436}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2035138099}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1158619061}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1431997327}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.171+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=482119274}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.291 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=174939604}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=651494882}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1933104986}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2339987823}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1789198891}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.292 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1964143438}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1706790066}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3239468562}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.173+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=284171313}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3776014779}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3272657542}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.293 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2704570111}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.174+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=782263898}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3608311607}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3975286886}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2843704694}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2048033719}) 2024-04-20T02:29:01.294 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.175+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691473513}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2515437101}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3291683320}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.176+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2416609259}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4028915867}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1949580691}) 2024-04-20T02:29:01.295 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1893272843}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1801478249}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=321986469}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.177+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2603131580}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717708691}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.296 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1914982975}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2263176358}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=660196183}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3750688142}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4008095382}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3109741791}) 2024-04-20T02:29:01.297 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2329267277}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.178+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3037815090}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3360249285}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=339171580}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=584207531}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.298 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1233285193}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2196919305}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1310419208}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4073552462}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417556496}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.180+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3173014534}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092842416}) 2024-04-20T02:29:01.299 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=197711808}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=182749240}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1266381961}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=542286985}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=306071589}) 2024-04-20T02:29:01.300 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.181+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2779685680}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=163888711}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.182+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1244909003}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2838203462}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239467520}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.301 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1097000004}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3057032785}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1510089858}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=791867462}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003801779}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.183+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1295789731}) 2024-04-20T02:29:01.302 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.184+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=491196799}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608827039}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3870710743}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1554653664}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1400587446}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.303 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2228331118}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4090678632}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=409889850}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244620912}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.186+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4270672256}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3093816784}) 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.304 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2748035891}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4109469906}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2681281904}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2961881672}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4251446991}) 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.305 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3297311609}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395677573}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.189+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2469411513}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3056154679}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2411493089}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3463087428}) 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.306 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1379265831}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3117047323}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=447939959}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3882354716}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2908561939}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.191+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1448102456}) 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.307 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284135692}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2163968666}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2684495771}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2343185243}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=858608842}) 2024-04-20T02:29:01.308 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2488301142}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2514199314}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4057825777}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.193+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2532869091}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=340007522}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.309 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3844307171}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3567392918}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3542807613}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=961121899}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3622928061}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.310 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1890821214}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1510597784}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.195+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556114674}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2115313979}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3917950751}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=5681682}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.311 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3660945793}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1393040300}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=318352626}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=120036226}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1635231828}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.197+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1533570188}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.312 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2256486525}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1639883669}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3076574122}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3604375737}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4096097374}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.313 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=809138588}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.199+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2501948875}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.200+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.200+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276091295}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.200+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2563117501}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3593598292}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3980559552}) 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.314 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=421867054}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3432711734}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.202+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2761909350}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=70361803}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2043416579}) 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.315 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1547625410}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.203+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=932486126}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.209+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2818087668}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3568226001}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562177689}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4088421280}) 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.316 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3464744493}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1573123151}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=509214437}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=687469177}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.210+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3008442828}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4148923088}) 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.317 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1319833952}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4028776597}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3249194405}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3101541495}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1638845417}) 2024-04-20T02:29:01.318 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4147668682}) 2024-04-20T02:29:01.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949411888}) 2024-04-20T02:29:01.319 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.213+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=642674389}) 2024-04-20T02:29:01.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.423 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=960860000}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2735280160}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=884325278}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3358183037}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.424 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2639290379}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2688281697}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2436197411}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=976803712}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.237+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=295131832}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879117576}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.425 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2014490250}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2401682648}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3465189719}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14141065}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1470881290}) 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.426 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1495750448}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4291445175}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.239+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2212691360}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3237055657}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1302941803}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4195575345}) 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.427 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1046449374}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3354368784}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1142766229}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.240+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.241+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.241+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=131509648}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.241+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.241+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1436171581}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.241+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2139128617}) 2024-04-20T02:29:01.428 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2319530052}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1525792730}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=603946986}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.242+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1268528421}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3614977105}) 2024-04-20T02:29:01.429 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=159969194}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=61745068}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2679676629}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.243+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4007422187}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.244+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.244+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.244+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1417865576}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.244+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.244+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.430 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.244+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=943394039}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=651068244}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4133025314}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2842213004}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2504295778}) 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.431 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2241358995}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2796941211}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.245+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=66499550}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2524609246}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4265760173}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2822837168}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.432 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3396219843}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2942019019}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=956051049}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2053760115}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2939020027}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.246+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2019680363}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.433 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=411945187}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1427105685}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=911267883}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2879225996}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4195970920}) 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.434 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2939743224}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2502655769}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2354136309}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.248+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2714250041}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=657261839}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1025788952}) 2024-04-20T02:29:01.435 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.249+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4086453074}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1711936330}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=767948285}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1151819648}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.436 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3371053475}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4223560073}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.250+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=358518024}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4069308795}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=703885087}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3839617371}) 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.437 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3505401046}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2058992896}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=67477886}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=740022565}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072495304}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.251+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3779585254}) 2024-04-20T02:29:01.438 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=597256916}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2013990729}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=439722635}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2043472356}) 2024-04-20T02:29:01.439 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811666294}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3026193289}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3346638391}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395048875}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.253+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1525570075}) 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.440 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2766061860}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3734373900}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1775126728}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3146913945}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3950312136}) 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.441 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2528950059}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.254+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4291566127}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.255+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.255+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232799254}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.255+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2651677263}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3024516359}) 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.442 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1400162241}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4142198979}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3117906826}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4180503393}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1367116429}) 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.443 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2366055317}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2041499523}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.256+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3302634727}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.257+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4047883718}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1524241829}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1791750998}) 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.444 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2243683582}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561839847}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=521910816}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=722849692}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=530482429}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.258+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3524389863}) 2024-04-20T02:29:01.445 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.259+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.259+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.259+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=565472505}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.259+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.259+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.259+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2716620854}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=777448988}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=817646797}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1722924061}) 2024-04-20T02:29:01.446 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2565953179}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3217870662}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4143366922}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.260+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=792967166}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1703281576}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3214842953}) 2024-04-20T02:29:01.447 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1008097052}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=348283377}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.261+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=171987114}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1108388669}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2166091354}) 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.448 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302743903}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.262+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433621610}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=273661768}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2439579036}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3594895212}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2531597700}) 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.449 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743257054}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1760415715}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3745092513}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.263+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=620765801}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.264+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1978380986}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3516004883}) 2024-04-20T02:29:01.450 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1412393504}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2829901295}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3833857054}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=183669286}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2252758851}) 2024-04-20T02:29:01.451 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2845617049}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1272657055}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.265+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2558609991}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1813305202}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=492641542}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.452 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2642604042}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3917165917}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1918629727}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=855806338}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3705196260}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3919494877}) 2024-04-20T02:29:01.453 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.267+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3086741919}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.268+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1924762007}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2929007912}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1299650023}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=79274133}) 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.454 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3384114503}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2613629659}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=865592324}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=904267920}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.269+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3485906621}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3181896319}) 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.455 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=311869935}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=272569861}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4238213152}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1453000383}) 2024-04-20T02:29:01.456 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3429558383}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.270+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.271+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3243730131}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.271+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.271+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001696899}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.271+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1993318799}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1907926343}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.457 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2653859608}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=394648473}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4180946072}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=245856371}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1010825598}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.272+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.458 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.273+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=319027523}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.273+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.273+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2865078431}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.273+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1847345372}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906444872}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178374765}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=160360829}) 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.459 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=801828347}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1044629611}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2460228738}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2384768830}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778801026}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.274+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2670697579}) 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.275+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.460 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.275+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.275+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588576587}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=52203568}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2733859096}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3011004179}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3402633451}) 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.461 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2401373571}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2507793218}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522861680}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.276+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2236067143}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1034276813}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=978064818}) 2024-04-20T02:29:01.462 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4004986134}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.277+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3286280101}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424918225}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1566757575}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.463 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4249625453}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3417366453}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.278+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1962002946}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3204220386}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1627647984}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.279+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3031101653}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.464 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1749169669}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2565933187}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043885826}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2899539868}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4113907376}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.280+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3782210473}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.281+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.465 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=845325417}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=257121007}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3969563961}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3298957832}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3055440233}) 2024-04-20T02:29:01.466 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=764068575}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2409248362}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3558228068}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.282+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3722171982}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2106382827}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.467 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1853258603}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1292648013}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3964630866}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4077684480}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1054220345}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1969973207}) 2024-04-20T02:29:01.468 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=167635827}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.284+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1363364518}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=918286623}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=374245962}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2383033850}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1058793113}) 2024-04-20T02:29:01.469 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3074003438}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2715933769}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.286+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.287+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.287+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=873792648}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.287+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.287+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3593782705}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.287+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1982263826}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2508116954}) 2024-04-20T02:29:01.470 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3039147156}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3809590286}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.288+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3478779182}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1066137085}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1183601922}) 2024-04-20T02:29:01.471 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1387390654}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=933942374}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.289+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=153281954}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1293638548}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4235366847}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2147684274}) 2024-04-20T02:29:01.472 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3924746567}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1629352192}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1264293523}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2938675823}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=600092123}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.290+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4280935491}) 2024-04-20T02:29:01.473 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2801491157}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1876636795}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1827193661}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3327450887}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4142315470}) 2024-04-20T02:29:01.474 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=37884902}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2609141427}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3157040970}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.292+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2455134325}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.293+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.293+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.293+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1792941659}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.293+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.293+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.293+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2463969388}) 2024-04-20T02:29:01.475 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=31097309}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2347015179}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1819701375}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365845397}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.476 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3962452302}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021863847}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.294+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3636194302}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3138711820}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=413713107}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3231388891}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.477 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1556153592}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=999270971}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3888911439}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2233150843}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3713846366}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.295+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1488105078}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.478 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3195382672}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1582211771}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4140233931}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3264154831}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3748790020}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.479 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2336325654}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466847657}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137449298}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.297+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3736590670}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3956152852}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4202702257}) 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.480 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2617867190}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2704245663}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.298+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233558724}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2484706583}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4189063800}) 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.481 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297945918}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.299+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3323303394}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470411105}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=486687114}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1815705574}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2907895555}) 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.482 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=829673449}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=536557411}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1405765408}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=28687689}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.300+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2193776979}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885053292}) 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.483 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2098827809}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2426263066}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2093018402}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2025128482}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2967716373}) 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.484 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=325972172}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=719198656}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.302+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1581053306}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.303+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.303+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.303+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385956167}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2145520913}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3102956378}) 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.485 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2443273524}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=283299031}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771255575}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4176422033}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3633670246}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.304+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970364817}) 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.486 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1740352161}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1685346758}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=352842590}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2384369917}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4260063515}) 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.305+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.487 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3171094095}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=725691484}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2744839058}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.306+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=64116491}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3467396567}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2469674152}) 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.488 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=939606788}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2523088081}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=190076224}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=569666072}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1836726591}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410266829}) 2024-04-20T02:29:01.489 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.307+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399278192}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1749017031}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2434747894}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=826372532}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1177933548}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.490 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2107559214}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=846353999}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3481284800}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3603843289}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.308+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1180801773}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.309+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3074288924}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.491 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2411452811}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1641461443}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=470412179}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1197237842}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=205465735}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.492 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2503908206}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2753516887}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.310+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2002348633}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3574251784}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884362984}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.493 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3093081399}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2340601378}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=253850696}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2651871851}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=686265115}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41606150}) 2024-04-20T02:29:01.494 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.311+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4006374614}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3943290921}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3937777475}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1214241852}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1331998317}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.495 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1624641224}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1453834189}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=151124379}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.312+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3442085711}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=384030098}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2652243221}) 2024-04-20T02:29:01.496 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.313+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2694451940}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=411151410}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3326699080}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184544764}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4230393500}) 2024-04-20T02:29:01.497 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2311847866}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290357722}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.314+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492371855}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.316+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=15,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37,osdmap=1} crc {logm=835454430,mdsmap=1531151339,mgr=1637162916,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=2968193071,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=2240665469,osdmap=3033740934}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=2127998366}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.498 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=16,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=33,osdmap=5} crc {logm=2804493015,mdsmap=1531151339,mgr=1969983733,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=2071145081,osdmap=2334359777}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=14,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=33,osdmap=9} crc {logm=2331050846,mdsmap=3140514009,mgr=1637162916,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2566794439,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=3686280955,osdmap=2477586838}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37,osdmap=4} crc {logm=218644252,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3065552487,mon_config_key=3396760964,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=2240665469,osdmap=3189862891}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=13,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=5} crc {logm=505087238,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045236,mgr_metadata=2530826944,mgrstat=413190140,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=4249318536,osdmap=2334359777}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=14,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=3} crc {logm=2687875740,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=831412822,osdmap=1376488926}) 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.499 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37} crc {logm=2804493015,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=3954964383,monmap=3385036551,osd_metadata=1440774329,osd_pg_creating=2914245765,osd_snap=2240665469}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.317+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36} crc {logm=2804493015,mdsmap=1531151340,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2557708641,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=2914245765,osd_snap=3101172573}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.318+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2252753582}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2051748132}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1437646805}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3688792854}) 2024-04-20T02:29:01.500 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1068776441}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2735664798}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1193162940}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2781776939}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.319+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2524222135}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.501 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=816854765}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3644314752}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4277392111}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=181217573}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3329187369}) 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.502 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=415323591}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=745748757}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1858158660}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.321+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1402322081}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3673081792}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.503 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2054474834}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3071200082}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2584742655}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2141823595}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.323+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2333525160}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.504 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=758512497}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2437714825}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.324+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=402294714}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.334+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.334+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.334+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=70} crc {osdmap=3093320482}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.334+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=4135584264}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=73} crc {osdmap=378362979}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=74} crc {osdmap=3247646083}) 2024-04-20T02:29:01.505 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=73} crc {osdmap=2958041246}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=71} crc {osdmap=129195549}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=70} crc {osdmap=2883649934}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=74} crc {osdmap=2878663322}) 2024-04-20T02:29:01.506 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:29:01.335+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=73} crc {osdmap=1739587435}) 2024-04-20T02:29:02.129 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:29:02.129 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:29:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.308+0000 7fed59a84700 10 monclient: get_monmap_and_config 2024-04-20T02:29:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.308+0000 7fed59a84700 10 monclient: build_initial_monmap 2024-04-20T02:29:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.308+0000 7fed59a84700 10 monclient: monmap: 2024-04-20T02:29:02.309 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:02.309511+0000 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:02.309511+0000 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:02.310 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:02.311 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:02.311 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.310+0000 7fed59a84700 1 Processor -- start 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.310+0000 7fed59a84700 1 -- start start 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.310+0000 7fed59a84700 10 monclient: init 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient: _add_conns ranks=[1,9,7,3,6,5,4,10,0,8,2] 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fed54137c90 addr v1:172.21.15.179:6789/0 2024-04-20T02:29:02.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fed5409b850 0x7fed5413cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): picked mon.noname-a con 0x7fed5409b850 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed5413d150 0x7fed5413f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): picked mon.noname-i con 0x7fed5413d150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed540751e0 con 0x7fed54137c90 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed541378d0 con 0x7fed5409b850 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fed5409c5a0 con 0x7fed5413d150 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed59a84700 10 monclient(hunting): authenticate will time out at 2421.712449s 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed537fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fed54137c90 0x7fed54138070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:41588/0 (socket says 172.21.15.47:41588) 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed537fe700 1 -- 172.21.15.47:0/2814082903 learned_addr learned my addr 172.21.15.47:0/2814082903 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed53fff700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed5413d150 0x7fed5413f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed53fff700 10 monclient(hunting): get_auth_request con 0x7fed5413d150 auth_method 0 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed53fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed53fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed53fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed52ffd700 1 --2- 172.21.15.47:0/2814082903 >> v2:172.21.15.47:3300/0 conn(0x7fed5409b850 0x7fed5413cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed52ffd700 10 monclient(hunting): get_auth_request con 0x7fed5409b850 auth_method 0 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45086547 0 0) 0x7fed540751e0 con 0x7fed54137c90 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed527fc700 10 monclient(hunting): _negotiate 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed527fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed527fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.311+0000 7fed527fc700 10 monclient(hunting): my global_id is 55297 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed3c0035b0 con 0x7fed54137c90 2024-04-20T02:29:02.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323382417 0 0) 0x7fed3c0035b0 con 0x7fed54137c90 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed5413d150 msgr2=0x7fed5413f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed5413d150 0x7fed5413f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 >> v2:172.21.15.47:3300/0 conn(0x7fed5409b850 msgr2=0x7fed5413cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 --2- 172.21.15.47:0/2814082903 >> v2:172.21.15.47:3300/0 conn(0x7fed5409b850 0x7fed5413cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: _finish_hunting 0 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:29:02.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5409c2c0 con 0x7fed54137c90 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7fed4c001c60 con 0x7fed54137c90 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: dump: 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.319 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 >> v1:172.21.15.179:6789/0 conn(0x7fed54137c90 legacy=0x7fed54138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient: _add_conns ranks=[4,8,3,1,6,5,0,7,2] 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fed3c004230 0x7fed3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): picked mon.c con 0x7fed3c004230 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fed3c004d90 0x7fed3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): picked mon.e con 0x7fed3c004d90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed3c009ac0 0x7fed3c00bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): picked mon.g con 0x7fed3c009ac0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed541378d0 con 0x7fed3c009ac0 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed5409c5a0 con 0x7fed3c004230 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c00c860 con 0x7fed3c004d90 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed527fc700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fed3c004230 0x7fed3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed3c009ac0 0x7fed3c00bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): get_auth_request con 0x7fed3c004230 auth_method 0 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): get_auth_request con 0x7fed3c009ac0 auth_method 0 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed53fff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient(hunting): handle_auth_done global_id 55297 payload 293 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 -- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fed3c004d90 msgr2=0x7fed3c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fed3c004d90 0x7fed3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 -- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed3c009ac0 msgr2=0x7fed3c00bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:02.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed3c009ac0 0x7fed3c00bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient: _finish_hunting 0 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 monclient: found mon.c 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 1 -- 172.21.15.47:0/2814082903 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5409c2c0 con 0x7fed3c004230 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed537fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed59a84700 5 monclient: authenticate success, global_id 55297 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.312+0000 7fed59a84700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed53fff700 1 -- 172.21.15.47:0/2814082903 reap_dead start 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed537fe700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fed3c004230 0x7fed3c004850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed4c002230 tx=0x7fed3c005880 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a894d2316296d0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed4c007540 con 0x7fed3c004230 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: dump: 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.324 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.325 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0076e0 con 0x7fed3c004230 2024-04-20T02:29:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed59a84700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:02.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed59a84700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed59a84700 4 set_mon_vals no callback set 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 1 -- 172.21.15.47:0/2814082903 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed4c010300 con 0x7fed3c004230 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.313+0000 7fed527fc700 10 monclient: dump: 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.326 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 10 monclient: shutdown 2024-04-20T02:29:02.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 -- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fed3c004230 msgr2=0x7fed3c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fed3c004230 0x7fed3c004850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed4c002230 tx=0x7fed3c005880 comp rx=0 tx=0).stop 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 -- 172.21.15.47:0/2814082903 shutdown_connections 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 --2- 172.21.15.47:0/2814082903 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fed3c004230 0x7fed3c004850 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 -- 172.21.15.47:0/2814082903 >> 172.21.15.47:0/2814082903 conn(0x7fed540a6480 msgr2=0x7fed540a2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 -- 172.21.15.47:0/2814082903 shutdown_connections 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 1 -- 172.21.15.47:0/2814082903 wait complete. 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 10 monclient: build_initial_monmap 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.314+0000 7fed59a84700 10 monclient: monmap: 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:02.328 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.315+0000 7fed59a84700 1 librados: starting msgr at 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.315+0000 7fed59a84700 1 librados: starting objecter 2024-04-20T02:29:02.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.315+0000 7fed59a84700 1 Processor -- start 2024-04-20T02:29:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 -- start start 2024-04-20T02:29:02.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 librados: setting wanted keys 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 librados: calling monclient init 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient: init 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient: _add_conns ranks=[1,5,7,0,8,4,6,2,3] 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed3c004230 0x7fed541d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): picked mon.noname-f con 0x7fed3c004230 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:02.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fed54133e80 0x7fed541342e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): picked mon.noname-h con 0x7fed54133e80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed54138dc0 0x7fed54139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): picked mon.noname-i con 0x7fed54138dc0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5413c4d0 con 0x7fed3c004230 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed5409fe70 con 0x7fed54133e80 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fed5409f9b0 con 0x7fed54138dc0 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.316+0000 7fed59a84700 10 monclient(hunting): authenticate will time out at 2421.717747s 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed54138dc0 0x7fed54139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed54138dc0 0x7fed54139240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:39492/0 (socket says 172.21.15.47:39492) 2024-04-20T02:29:02.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 1 -- 172.21.15.47:0/581473297 learned_addr learned my addr 172.21.15.47:0/581473297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 10 monclient(hunting): get_auth_request con 0x7fed54138dc0 auth_method 0 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed53fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fed54133e80 0x7fed541342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): get_auth_request con 0x7fed54133e80 auth_method 0 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed3c004230 0x7fed541d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): get_auth_request con 0x7fed3c004230 auth_method 0 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed537fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient(hunting): handle_auth_done global_id 84155 payload 824 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 -- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed54138dc0 msgr2=0x7fed54139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed54138dc0 0x7fed54139240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 -- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed3c004230 msgr2=0x7fed541d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed3c004230 0x7fed541d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 monclient: found mon.noname-h 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 -- 172.21.15.47:0/581473297 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5413cd70 con 0x7fed54133e80 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed52ffd700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fed54133e80 0x7fed541342e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed4800e7a0 tx=0x7fed4800eab0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed3bfff700 10 client.?.objecter ms_handle_connect 0x7fed54133e80 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.317+0000 7fed3bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed48013af0 con 0x7fed54133e80 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: dump: 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.337 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.338 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: _finish_auth 0 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 5 monclient: authenticate success, global_id 84155 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed48013c90 con 0x7fed54133e80 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 10 monclient: _renew_subs 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5413b0f0 con 0x7fed54133e80 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 10 client.84155.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 10 monclient: _renew_subs 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54134820 con 0x7fed54133e80 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed59a84700 1 librados: init done 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:02.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed48013e00 con 0x7fed54133e80 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed3bfff700 10 monclient: dump: 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.340 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.318+0000 7fed51ffb700 4 set_mon_vals no callback set 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed59a84700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed59a84700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed59a84700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed340050e0 con 0x7fed54133e80 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fed48013680 con 0x7fed54133e80 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fed4003a320 0x7fed4003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(515..515 src has 266..515) v4 ==== 6557+0+0 (secure 0 0 0) 0x7fed4804a4b0 con 0x7fed54133e80 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 10 client.84155.objecter ms_dispatch 0x7fed54001550 osd_map(515..515 src has 266..515) v4 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 3 client.84155.objecter handle_osd_map got epochs [515,515] > 0 2024-04-20T02:29:02.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 3 client.84155.objecter handle_osd_map decoding full epoch 515 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.319+0000 7fed3bfff700 20 client.84155.objecter dump_active .. 0 homeless 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.320+0000 7fed537fe700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fed4003a320 0x7fed4003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.320+0000 7fed537fe700 10 monclient: get_auth_request con 0x7fed4003a320 auth_method 0 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.320+0000 7fed537fe700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fed4003a320 0x7fed4003c7e0 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7fed4c002230 tx=0x7fed4c009070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.320+0000 7fed3bfff700 10 client.84155.objecter ms_handle_connect 0x7fed4003a320 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.324+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fed48013930 con 0x7fed54133e80 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.324+0000 7fed3bfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.324+0000 7fed3bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:02.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.631+0000 7fed59a84700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:29:02.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.631+0000 7fed59a84700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:29:02.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.631+0000 7fed59a84700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:29:02.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.631+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fed34005910 con 0x7fed54133e80 2024-04-20T02:29:02.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.632+0000 7fed3bfff700 1 -- 172.21.15.47:0/581473297 <== mon.5 v2:172.21.15.179:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fed48017c80 con 0x7fed54133e80 2024-04-20T02:29:02.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.632+0000 7fed3bfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:29:02.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.632+0000 7fed3bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:02.633 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":88,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 10 librados: watch_flush enter 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 10 librados: watch_flush exit 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 20 client.84155.objecter shutdown clearing up homeless session... 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 10 client.84155.objecter successfully canceled tick 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fed4003a320 msgr2=0x7fed4003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fed4003a320 0x7fed4003c7e0 secure :-1 s=READY pgs=3298 cs=0 l=1 rev1=1 crypto rx=0x7fed4c002230 tx=0x7fed4c009070 comp rx=0 tx=0).stop 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 10 monclient: shutdown 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fed54133e80 msgr2=0x7fed541342e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fed54133e80 0x7fed541342e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed4800e7a0 tx=0x7fed4800eab0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 shutdown_connections 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fed54138dc0 0x7fed54139240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fed4003a320 0x7fed4003c7e0 unknown :-1 s=CLOSED pgs=3298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed3c004230 0x7fed541d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 --2- 172.21.15.47:0/581473297 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fed54133e80 0x7fed541342e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 >> 172.21.15.47:0/581473297 conn(0x7fed540a6480 msgr2=0x7fed5409e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 shutdown_connections 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 -- 172.21.15.47:0/581473297 wait complete. 2024-04-20T02:29:02.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.634+0000 7fed59a84700 1 librados: shutdown 2024-04-20T02:29:02.648 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:29:02.648 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:29:02.648 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:29:02.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.784+0000 7f17c4970700 10 monclient: get_monmap_and_config 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.784+0000 7f17c4970700 10 monclient: build_initial_monmap 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.784+0000 7f17c4970700 10 monclient: monmap: 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:02.785726+0000 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:02.785726+0000 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.785 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 Processor -- start 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 -- start start 2024-04-20T02:29:02.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient: init 2024-04-20T02:29:02.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient: _add_conns ranks=[9,6,8,3,2,0,4,7,10,5,1] 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17c0071730 0x7f17c0071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): picked mon.noname-a con 0x7f17c0071730 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 0x7f17c00a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): picked mon.noname-d con 0x7f17c006f020 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c00a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): picked mon.noname-e con 0x7f17c00a2db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c009fea0 con 0x7f17c0071730 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f17c009e7b0 con 0x7f17c006f020 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f17c006d190 con 0x7f17c00a2db0 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.785+0000 7f17c4970700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17c4970700 10 monclient(hunting): authenticate will time out at 2422.187013s 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c00a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 0x7f17c00a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17c0071730 0x7f17c0071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 0x7f17c00a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:38812/0 (socket says 172.21.15.47:38812) 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c00a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48586/0 (socket says 172.21.15.47:48586) 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): get_auth_request con 0x7f17c0071730 auth_method 0 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 -- 172.21.15.47:0/363460067 learned_addr learned my addr 172.21.15.47:0/363460067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): get_auth_request con 0x7f17c006f020 auth_method 0 2024-04-20T02:29:02.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): get_auth_request con 0x7f17c00a2db0 auth_method 0 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_done global_id 54924 payload 274 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 -- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 msgr2=0x7f17c00a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 --2- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c00a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 -- 172.21.15.47:0/363460067 >> v2:172.21.15.47:3300/0 conn(0x7f17c0071730 msgr2=0x7f17c0071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 --2- 172.21.15.47:0/363460067 >> v2:172.21.15.47:3300/0 conn(0x7f17c0071730 0x7f17c0071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 monclient: found mon.noname-d 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 -- 172.21.15.47:0/363460067 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00751e0 con 0x7f17c006f020 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17be59c700 1 --2- 172.21.15.47:0/363460067 >> v2:172.21.15.47:3300/0 conn(0x7f17c0071730 0x7f17c0071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bed9d700 1 --2- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c00a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bdd9b700 1 --2- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 0x7f17c00a2870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17b40098f0 tx=0x7f17b4009c00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b2c2bd23d316dcf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 1 -- 172.21.15.47:0/363460067 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17b4011aa0 con 0x7f17c006f020 2024-04-20T02:29:02.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: dump: 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.804 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: _finish_auth 0 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: _check_auth_tickets 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 1 -- 172.21.15.47:0/363460067 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b4011c00 con 0x7f17c006f020 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17c4970700 5 monclient: authenticate success, global_id 54924 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17c4970700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17c4970700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 1 -- 172.21.15.47:0/363460067 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17b4011d70 con 0x7f17c006f020 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17c4970700 4 set_mon_vals no callback set 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.786+0000 7f17bd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17bd59a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17bd59a700 10 monclient: dump: 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.806 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 10 monclient: shutdown 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 -- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 msgr2=0x7f17c00a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 --2- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 0x7f17c00a2870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17b40098f0 tx=0x7f17b4009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 -- 172.21.15.47:0/363460067 shutdown_connections 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 --2- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c00a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 --2- 172.21.15.47:0/363460067 >> v2:172.21.15.47:3300/0 conn(0x7f17c0071730 0x7f17c0071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 --2- 172.21.15.47:0/363460067 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f17c006f020 0x7f17c00a2870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 -- 172.21.15.47:0/363460067 >> 172.21.15.47:0/363460067 conn(0x7f17c00a3f90 msgr2=0x7f17c006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 -- 172.21.15.47:0/363460067 shutdown_connections 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.787+0000 7f17c4970700 1 -- 172.21.15.47:0/363460067 wait complete. 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 10 monclient: build_initial_monmap 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 10 monclient: monmap: 2024-04-20T02:29:02.807 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 1 librados: starting msgr at 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 1 librados: starting objecter 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 1 Processor -- start 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 1 -- start start 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 1 librados: setting wanted keys 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 1 librados: calling monclient init 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.788+0000 7f17c4970700 10 monclient: init 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient: _add_conns ranks=[1,7,8,4,2,3,6,5,0] 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17c006f020 0x7f17c01cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): picked mon.noname-f con 0x7f17c006f020 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17c0071730 0x7f17c01ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): picked mon.noname-i con 0x7f17c0071730 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c0123f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): picked mon.noname-e con 0x7f17c00a2db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0127f40 con 0x7f17c006f020 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0072410 con 0x7f17c0071730 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f17c009f1b0 con 0x7f17c00a2db0 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17c4970700 10 monclient(hunting): authenticate will time out at 2422.190393s 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c0123f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c0123f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48600/0 (socket says 172.21.15.47:48600) 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 -- 172.21.15.47:0/1508997649 learned_addr learned my addr 172.21.15.47:0/1508997649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): get_auth_request con 0x7f17c00a2db0 auth_method 0 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17c0071730 0x7f17c01ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): get_auth_request con 0x7f17c0071730 auth_method 0 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17c006f020 0x7f17c01cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): get_auth_request con 0x7f17c006f020 auth_method 0 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient(hunting): handle_auth_done global_id 35702 payload 824 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17c0071730 msgr2=0x7f17c01ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17c0071730 0x7f17c01ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17c006f020 msgr2=0x7f17c01cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17c006f020 0x7f17c01cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 monclient: found mon.noname-e 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:02.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.789+0000 7f17bed9d700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b000b740 con 0x7f17c00a2db0 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17bed9d700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c0123f50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009e60 tx=0x7f17b800c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 client.?.objecter ms_handle_connect 0x7f17c00a2db0 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17b8009cd0 con 0x7f17c00a2db0 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: dump: 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.813 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: _finish_auth 0 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: _check_auth_tickets 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b800f040 con 0x7f17c00a2db0 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 5 monclient: authenticate success, global_id 35702 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 10 monclient: _renew_subs 2024-04-20T02:29:02.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b8009a70 con 0x7f17c00a2db0 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17b80148f0 con 0x7f17c00a2db0 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17aeffd700 10 monclient: dump: 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.815 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17bcd99700 4 set_mon_vals no callback set 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 10 client.35702.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 10 monclient: _renew_subs 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b4009550 con 0x7f17c00a2db0 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.790+0000 7f17c4970700 1 librados: init done 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17c4970700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17c4970700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 0x7f17c0124490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0068280 con 0x7f17c006aac0 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f17b8014480 con 0x7f17c00a2db0 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 0x7f17c0124490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient: get_auth_request con 0x7f17c006aac0 auth_method 0 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f17a003a2f0 0x7f17a003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(515..515 src has 266..515) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f17b804ab70 con 0x7f17c00a2db0 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 10 client.35702.objecter ms_dispatch 0x7f17c0001550 osd_map(515..515 src has 266..515) v4 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 3 client.35702.objecter handle_osd_map got epochs [515,515] > 0 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 3 client.35702.objecter handle_osd_map decoding full epoch 515 2024-04-20T02:29:02.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 10 monclient(hunting): handle_auth_done global_id 36387 payload 824 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17c0127850 con 0x7f17c006aac0 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 20 client.35702.objecter dump_active .. 0 homeless 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17bdd9b700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f17a003a2f0 0x7f17a003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17bdd9b700 10 monclient: get_auth_request con 0x7f17a003a2f0 auth_method 0 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17be59c700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 0x7f17c0124490 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17b0011d90 tx=0x7f17b000f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 10 client.35702.objecter ms_handle_connect 0x7f17c006aac0 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.791+0000 7f17aeffd700 10 client.35702.objecter resend_mon_ops 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17b000fea0 con 0x7f17c006aac0 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 10 monclient: dump: 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 10 client.35702.objecter ms_handle_connect 0x7f17a003a2f0 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17bdd9b700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f17a003a2f0 0x7f17a003c7b0 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7f17b40098f0 tx=0x7f17b4008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f17b0004030 con 0x7f17c006aac0 2024-04-20T02:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 msgr2=0x7f17c0124490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 0x7f17c0124490 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17b0011d90 tx=0x7f17b000f6b0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 msgr2=0x7f17c0124490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.792+0000 7f17aeffd700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c006aac0 0x7f17c0124490 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17b0011d90 tx=0x7f17b000f6b0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17c4970700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17c4970700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 0x7f17c0124de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17c4970700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c006a400 con 0x7f17c01249d0 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 0x7f17c0124de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient: get_auth_request con 0x7f17c01249d0 auth_method 0 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 10 monclient(hunting): handle_auth_done global_id 36396 payload 824 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 1 -- 172.21.15.47:0/1508997649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f17c0127da0 con 0x7f17c01249d0 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17aeffd700 10 client.35702.objecter ms_handle_connect 0x7f17c01249d0 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17aeffd700 10 client.35702.objecter resend_mon_ops 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17be59c700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 0x7f17c0124de0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f17b0011d90 tx=0x7f17b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17b0003c40 con 0x7f17c01249d0 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17aeffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.800+0000 7f17aeffd700 10 monclient: dump: 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f17b0023070 con 0x7f17c01249d0 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 msgr2=0x7f17c0124de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 0x7f17c0124de0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f17b0011d90 tx=0x7f17b0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 msgr2=0x7f17c0124de0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17aeffd700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17c01249d0 0x7f17c0124de0 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f17b0011d90 tx=0x7f17b0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:02.825 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:02.826 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:02.827 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:02.828 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:02.829 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:02.831 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:02.832 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.833 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:02.834 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:02.835 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.836 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.837 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.838 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.839 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.840 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.841 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:02.842 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:02.844 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:02.845 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:29:02.845 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:02.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:02.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:02.845 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:02.845 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 10 librados: watch_flush enter 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 10 librados: watch_flush exit 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 20 client.35702.objecter shutdown clearing up homeless session... 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 10 client.35702.objecter successfully canceled tick 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f17a003a2f0 msgr2=0x7f17a003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f17a003a2f0 0x7f17a003c7b0 secure :-1 s=READY pgs=3299 cs=0 l=1 rev1=1 crypto rx=0x7f17b40098f0 tx=0x7f17b4008030 comp rx=0 tx=0).stop 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 10 monclient: shutdown 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 msgr2=0x7f17c0123f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 --2- 172.21.15.47:0/1508997649 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17c00a2db0 0x7f17c0123f50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009e60 tx=0x7f17b800c3a0 comp rx=0 tx=0).stop 2024-04-20T02:29:02.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17bed9d700 1 -- 172.21.15.47:0/1508997649 reap_dead start 2024-04-20T02:29:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17bed9d700 1 -- 172.21.15.47:0/1508997649 reap_dead start 2024-04-20T02:29:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 shutdown_connections 2024-04-20T02:29:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 >> 172.21.15.47:0/1508997649 conn(0x7f17c00a3f90 msgr2=0x7f17c0134f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 shutdown_connections 2024-04-20T02:29:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 -- 172.21.15.47:0/1508997649 wait complete. 2024-04-20T02:29:02.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.801+0000 7f17c4970700 1 librados: shutdown 2024-04-20T02:29:02.848 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:02.848 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:29:02.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:29:02.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.987+0000 7fd7d9236700 10 monclient: get_monmap_and_config 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.987+0000 7fd7d9236700 10 monclient: build_initial_monmap 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.987+0000 7fd7d9236700 10 monclient: monmap: 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:02.988892+0000 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:02.988892+0000 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:02.988 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:02.989 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:02.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 1 Processor -- start 2024-04-20T02:29:02.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 1 -- start start 2024-04-20T02:29:02.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 10 monclient: init 2024-04-20T02:29:02.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:02.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 10 monclient: _add_conns ranks=[9,3,0,2,7,10,1,5,6,8,4] 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7d409f690 0x7fd7d40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.988+0000 7fd7d9236700 10 monclient(hunting): picked mon.noname-a con 0x7fd7d409f690 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d40a3ea0 0x7fd7d40a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 10 monclient(hunting): picked mon.noname-g con 0x7fd7d40a3ea0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd7d41357f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d40751e0 con 0x7fd7d41357f0 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d4070ce0 con 0x7fd7d409f690 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4072780 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d9236700 10 monclient(hunting): authenticate will time out at 2422.390089s 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd7d409f690 0x7fd7d40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d359e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd7d41357f0 0x7fd7d4135c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47262/0 (socket says 172.21.15.47:47262) 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d359e700 1 -- 172.21.15.47:0/4106147514 learned_addr learned my addr 172.21.15.47:0/4106147514 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): get_auth_request con 0x7fd7d409f690 auth_method 0 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d259c700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d40a3ea0 0x7fd7d40a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_done global_id 36405 payload 274 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 -- 172.21.15.47:0/4106147514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d40a3ea0 msgr2=0x7fd7d40a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d40a3ea0 0x7fd7d40a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 -- 172.21.15.47:0/4106147514 >> v1:172.21.15.47:6789/0 conn(0x7fd7d41357f0 legacy=0x7fd7d4135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 monclient: found mon.noname-a 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:03.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 -- 172.21.15.47:0/4106147514 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d409fa70 con 0x7fd7d409f690 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d259c700 10 monclient: get_auth_request con 0x7fd7d40a3ea0 auth_method 0 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d259c700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d40a3ea0 0x7fd7d40a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d2d9d700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7d409f690 0x7fd7d40a3960 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009840 tx=0x7fd7c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1175afa6451e2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 1 -- 172.21.15.47:0/4106147514 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7c4011aa0 con 0x7fd7d409f690 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 10 monclient: dump: 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.004 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.989+0000 7fd7d1d9b700 1 -- 172.21.15.47:0/4106147514 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4011c00 con 0x7fd7d409f690 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 5 monclient: authenticate success, global_id 36405 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d1d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 4 set_mon_vals no callback set 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d1d9b700 1 -- 172.21.15.47:0/4106147514 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7c4011d70 con 0x7fd7d409f690 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d1d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d1d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d1d9b700 10 monclient: dump: 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.006 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.007 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 10 monclient: shutdown 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 -- 172.21.15.47:0/4106147514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7d409f690 msgr2=0x7fd7d40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7d409f690 0x7fd7d40a3960 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009840 tx=0x7fd7c4009b50 comp rx=0 tx=0).stop 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 -- 172.21.15.47:0/4106147514 shutdown_connections 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7d409f690 0x7fd7d40a3960 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 --2- 172.21.15.47:0/4106147514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d40a3ea0 0x7fd7d40a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 -- 172.21.15.47:0/4106147514 >> 172.21.15.47:0/4106147514 conn(0x7fd7d40a8610 msgr2=0x7fd7d40a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 -- 172.21.15.47:0/4106147514 shutdown_connections 2024-04-20T02:29:03.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.990+0000 7fd7d9236700 1 -- 172.21.15.47:0/4106147514 wait complete. 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.991+0000 7fd7d9236700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.991+0000 7fd7d9236700 10 monclient: monmap: 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.009 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.991+0000 7fd7d9236700 1 librados: starting msgr at 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.991+0000 7fd7d9236700 1 librados: starting objecter 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.992+0000 7fd7d9236700 1 Processor -- start 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.992+0000 7fd7d9236700 1 -- start start 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.992+0000 7fd7d9236700 1 librados: setting wanted keys 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.992+0000 7fd7d9236700 1 librados: calling monclient init 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.992+0000 7fd7d9236700 10 monclient: init 2024-04-20T02:29:03.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient: _add_conns ranks=[2,8,3,4,7,6,1,5,0] 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d409f690 0x7fd7d41cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): picked mon.noname-b con 0x7fd7d409f690 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd7d40a3ea0 0x7fd7d4134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): picked mon.noname-e con 0x7fd7d40a3ea0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d41357f0 0x7fd7d41c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): picked mon.noname-g con 0x7fd7d41357f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d41cd260 con 0x7fd7d409f690 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4135580 con 0x7fd7d41357f0 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4071380 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d9236700 10 monclient(hunting): authenticate will time out at 2422.394221s 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d409f690 0x7fd7d41cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd7d40a3ea0 0x7fd7d4134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d409f690 0x7fd7d41cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:54098/0 (socket says 172.21.15.47:54098) 2024-04-20T02:29:03.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 1 -- 172.21.15.47:0/1685976668 learned_addr learned my addr 172.21.15.47:0/1685976668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): get_auth_request con 0x7fd7d40a3ea0 auth_method 0 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): get_auth_request con 0x7fd7d409f690 auth_method 0 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d359e700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d41357f0 0x7fd7d41c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient(hunting): handle_auth_done global_id 35711 payload 824 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d41357f0 msgr2=0x7fd7d41c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d41357f0 0x7fd7d41c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d409f690 msgr2=0x7fd7d41cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d409f690 0x7fd7d41cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 monclient: found mon.noname-e 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40094a0 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d359e700 10 monclient: get_auth_request con 0x7fd7d41357f0 auth_method 0 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d359e700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd7d41357f0 0x7fd7d41c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7d259c700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd7d40a3ea0 0x7fd7d4134a80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4124be0 tx=0x7fd7c800caa0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7bb7fe700 10 client.?.objecter ms_handle_connect 0x7fd7d40a3ea0 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.993+0000 7fd7bb7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7c80044a0 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: dump: 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.015 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80036a0 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 5 monclient: authenticate success, global_id 35711 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 10 monclient: _renew_subs 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7c8003c80 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d409edc0 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d159a700 4 set_mon_vals no callback set 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 10 client.35711.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:03.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7bb7fe700 10 monclient: dump: 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.018 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 10 monclient: _renew_subs 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d4125ec0 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 1 librados: init done 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.994+0000 7fd7d9236700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d9236700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 0x7fd7d41c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4068380 con 0x7fd7d406aac0 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd7c8003810 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 0x7fd7d41c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient: get_auth_request con 0x7fd7d406aac0 auth_method 0 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd7c003a2f0 0x7fd7c003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_done global_id 24581 payload 824 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7d40721c0 con 0x7fd7d406aac0 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(515..515 src has 266..515) v4 ==== 6557+0+0 (secure 0 0 0) 0x7fd7c804a4e0 con 0x7fd7d40a3ea0 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 10 client.35711.objecter ms_dispatch 0x7fd7d4001550 osd_map(515..515 src has 266..515) v4 2024-04-20T02:29:03.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 3 client.35711.objecter handle_osd_map got epochs [515,515] > 0 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 3 client.35711.objecter handle_osd_map decoding full epoch 515 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d2d9d700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 0x7fd7d41c93b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40090b0 tx=0x7fd7c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 20 client.35711.objecter dump_active .. 0 homeless 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 10 client.35711.objecter ms_handle_connect 0x7fd7d406aac0 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 10 client.35711.objecter resend_mon_ops 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7c4020070 con 0x7fd7d406aac0 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d359e700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd7c003a2f0 0x7fd7c003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7bb7fe700 10 monclient: dump: 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.021 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.022 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.995+0000 7fd7d359e700 10 monclient: get_auth_request con 0x7fd7c003a2f0 auth_method 0 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd7c40045b0 con 0x7fd7d406aac0 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 msgr2=0x7fd7d41c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 0x7fd7d41c93b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40090b0 tx=0x7fd7c4004750 comp rx=0 tx=0).stop 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 msgr2=0x7fd7d41c93b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d406aac0 0x7fd7d41c93b0 secure :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40090b0 tx=0x7fd7c4004750 comp rx=0 tx=0).stop 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7bb7fe700 10 client.35711.objecter ms_handle_connect 0x7fd7c003a2f0 2024-04-20T02:29:03.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:02.996+0000 7fd7d359e700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd7c003a2f0 0x7fd7c003c7b0 secure :-1 s=READY pgs=3300 cs=0 l=1 rev1=1 crypto rx=0x7fd7d406dd20 tx=0x7fd7cc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.003+0000 7fd7d9236700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:03.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.003+0000 7fd7d9236700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 0x7fd7d41c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.003+0000 7fd7d9236700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.003+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d406a400 con 0x7fd7d4124c80 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.003+0000 7fd7d2d9d700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 0x7fd7d41c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient: get_auth_request con 0x7fd7d4124c80 auth_method 0 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 10 monclient(hunting): handle_auth_done global_id 24590 payload 824 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 1 -- 172.21.15.47:0/1685976668 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd7d40724a0 con 0x7fd7d4124c80 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 client.35711.objecter ms_handle_connect 0x7fd7d4124c80 2024-04-20T02:29:03.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 client.35711.objecter resend_mon_ops 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7d2d9d700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 0x7fd7d41c98f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4027000 tx=0x7fd7c4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7c4003d40 con 0x7fd7d4124c80 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 monclient: dump: 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.027 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fd7c401b430 con 0x7fd7d4124c80 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 msgr2=0x7fd7d41c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 0x7fd7d41c98f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4027000 tx=0x7fd7c4011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 msgr2=0x7fd7d41c98f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.004+0000 7fd7bb7fe700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd7d4124c80 0x7fd7d41c98f0 secure :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4027000 tx=0x7fd7c4011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:03.029 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 7, 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:03.030 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:03.031 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.032 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:03.033 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.034 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.035 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.036 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.040 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.041 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:03.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:03.043 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.043 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:03.043 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.044 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:03.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:03.048 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.052 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.053 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:03.054 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 10 librados: watch_flush enter 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 10 librados: watch_flush exit 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 20 client.35711.objecter shutdown clearing up homeless session... 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 10 client.35711.objecter successfully canceled tick 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd7c003a2f0 msgr2=0x7fd7c003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd7c003a2f0 0x7fd7c003c7b0 secure :-1 s=READY pgs=3300 cs=0 l=1 rev1=1 crypto rx=0x7fd7d406dd20 tx=0x7fd7cc009040 comp rx=0 tx=0).stop 2024-04-20T02:29:03.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 10 monclient: shutdown 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd7d40a3ea0 msgr2=0x7fd7d4134a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd7d40a3ea0 0x7fd7d4134a80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4124be0 tx=0x7fd7c800caa0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d359e700 1 -- 172.21.15.47:0/1685976668 reap_dead start 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 shutdown_connections 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 --2- 172.21.15.47:0/1685976668 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd7d40a3ea0 0x7fd7d4134a80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 >> 172.21.15.47:0/1685976668 conn(0x7fd7d40a8610 msgr2=0x7fd7d40a05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 shutdown_connections 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 -- 172.21.15.47:0/1685976668 wait complete. 2024-04-20T02:29:03.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.005+0000 7fd7d9236700 1 librados: shutdown 2024-04-20T02:29:03.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:03.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:29:03.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.197+0000 7f79eb5bf700 10 monclient: get_monmap_and_config 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.197+0000 7f79eb5bf700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.197+0000 7f79eb5bf700 10 monclient: monmap: 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:03.198918+0000 2024-04-20T02:29:03.198 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:03.198918+0000 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.199 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:03.200 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:03.200 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.198+0000 7f79eb5bf700 1 Processor -- start 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.198+0000 7f79eb5bf700 1 -- start start 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.198+0000 7f79eb5bf700 10 monclient: init 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient: _add_conns ranks=[8,6,0,4,10,2,7,5,9,1,3] 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e40719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): picked mon.noname-e con 0x7f79e4071610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:03.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f79e4071f30 0x7f79e40a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): picked mon.noname-d con 0x7f79e4071f30 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f79e40a2e90 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e40751e0 con 0x7f79e40a2e90 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40727b0 con 0x7f79e4071f30 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40a0880 con 0x7f79e4071610 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79eb5bf700 10 monclient(hunting): authenticate will time out at 2422.600229s 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e40719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e9b5c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f79e40a2e90 0x7f79e413b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47278/0 (socket says 172.21.15.47:47278) 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f79e4071f30 0x7f79e40a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e9b5c700 1 -- 172.21.15.47:0/1647106213 learned_addr learned my addr 172.21.15.47:0/1647106213 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e40719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48622/0 (socket says 172.21.15.47:48622) 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): get_auth_request con 0x7f79e4071610 auth_method 0 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): get_auth_request con 0x7f79e4071f30 auth_method 0 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient(hunting): handle_auth_done global_id 35720 payload 274 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 -- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f79e4071f30 msgr2=0x7f79e40a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 --2- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f79e4071f30 0x7f79e40a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 -- 172.21.15.47:0/1647106213 >> v1:172.21.15.47:6789/0 conn(0x7f79e40a2e90 legacy=0x7f79e413b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 monclient: found mon.noname-e 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79dbfff700 1 -- 172.21.15.47:0/1647106213 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932104456 0 0) 0x7f79e40751e0 con 0x7f79e40a2e90 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 -- 172.21.15.47:0/1647106213 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e409e8f0 con 0x7f79e4071610 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79dbfff700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e8b5a700 1 --2- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f79e4071f30 0x7f79e40a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.199+0000 7f79e935b700 1 --2- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e40719f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009870 tx=0x7f79e0009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=760b10efc8df76b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 1 -- 172.21.15.47:0/1647106213 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e0011aa0 con 0x7f79e4071610 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: dump: 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.221 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 1 -- 172.21.15.47:0/1647106213 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0011c00 con 0x7f79e4071610 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 5 monclient: authenticate success, global_id 35720 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 4 set_mon_vals no callback set 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 1 -- 172.21.15.47:0/1647106213 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e0011d70 con 0x7f79e4071610 2024-04-20T02:29:03.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79dbfff700 10 monclient: dump: 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.224 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 10 monclient: shutdown 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 -- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 msgr2=0x7f79e40719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e40719f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009870 tx=0x7f79e0009b80 comp rx=0 tx=0).stop 2024-04-20T02:29:03.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 -- 172.21.15.47:0/1647106213 shutdown_connections 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e40719f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/1647106213 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f79e4071f30 0x7f79e40a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 -- 172.21.15.47:0/1647106213 >> 172.21.15.47:0/1647106213 conn(0x7f79e40a7570 msgr2=0x7f79e40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 -- 172.21.15.47:0/1647106213 shutdown_connections 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.200+0000 7f79eb5bf700 1 -- 172.21.15.47:0/1647106213 wait complete. 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.201+0000 7f79eb5bf700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.201+0000 7f79eb5bf700 10 monclient: monmap: 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:03.226 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.201+0000 7f79eb5bf700 1 librados: starting msgr at 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.201+0000 7f79eb5bf700 1 librados: starting objecter 2024-04-20T02:29:03.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.201+0000 7f79eb5bf700 1 Processor -- start 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 -- start start 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 librados: setting wanted keys 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 librados: calling monclient init 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient: init 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient: _add_conns ranks=[8,2,4,1,7,6,5,3,0] 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e41cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): picked mon.noname-e con 0x7f79e4071610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79e4071f30 0x7f79e41ba850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): picked mon.noname-b con 0x7f79e4071f30 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e40a2e90 0x7f79e41bad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): picked mon.noname-c con 0x7f79e40a2e90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:03.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e41ab9b0 con 0x7f79e4071f30 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40723f0 con 0x7f79e40a2e90 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f79e409f260 con 0x7f79e4071610 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79eb5bf700 10 monclient(hunting): authenticate will time out at 2422.603551s 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e41cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79e4071f30 0x7f79e41ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e41cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48632/0 (socket says 172.21.15.47:48632) 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): get_auth_request con 0x7f79e4071f30 auth_method 0 2024-04-20T02:29:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 1 -- 172.21.15.47:0/2640705021 learned_addr learned my addr 172.21.15.47:0/2640705021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): get_auth_request con 0x7f79e4071610 auth_method 0 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e40a2e90 0x7f79e41bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): get_auth_request con 0x7f79e40a2e90 auth_method 0 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e9b5c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 10 monclient(hunting): handle_auth_done global_id 24599 payload 824 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.202+0000 7f79e8b5a700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 msgr2=0x7f79e41cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e41cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e40a2e90 msgr2=0x7f79e41bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e40a2e90 0x7f79e41bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 1 monclient: found mon.noname-b 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40094a0 con 0x7f79e4071f30 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e935b700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f79e4071610 0x7f79e41cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79e8b5a700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79e4071f30 0x7f79e41ba850 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f79d400e980 tx=0x7f79d400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 client.?.objecter ms_handle_connect 0x7f79e4071f30 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79d400cb00 con 0x7f79e4071f30 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: dump: 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.235 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400cc60 con 0x7f79e4071f30 2024-04-20T02:29:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 5 monclient: authenticate success, global_id 24599 2024-04-20T02:29:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 10 monclient: _renew_subs 2024-04-20T02:29:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:29:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e00094a0 con 0x7f79e4071f30 2024-04-20T02:29:03.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79d400cdd0 con 0x7f79e4071f30 2024-04-20T02:29:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79d9ffb700 10 monclient: dump: 2024-04-20T02:29:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.237 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79daffd700 4 set_mon_vals no callback set 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 10 client.24599.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 10 monclient: _renew_subs 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:29:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e41a9e80 con 0x7f79e4071f30 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.203+0000 7f79eb5bf700 1 librados: init done 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79eb5bf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 0x7f79e41abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4068310 con 0x7f79e406aac0 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f79d400c690 con 0x7f79e4071f30 2024-04-20T02:29:03.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 0x7f79e41abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient: get_auth_request con 0x7f79e406aac0 auth_method 0 2024-04-20T02:29:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.204+0000 7f79d9ffb700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79c803a2f0 0x7f79c803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(515..515 src has 266..515) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f79d404a8f0 con 0x7f79e4071f30 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 client.24599.objecter ms_dispatch 0x7f79e4001550 osd_map(515..515 src has 266..515) v4 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 3 client.24599.objecter handle_osd_map got epochs [515,515] > 0 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 3 client.24599.objecter handle_osd_map decoding full epoch 515 2024-04-20T02:29:03.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79e935b700 10 monclient(hunting): handle_auth_done global_id 84181 payload 824 2024-04-20T02:29:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79e935b700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79e409ffe0 con 0x7f79e406aac0 2024-04-20T02:29:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 20 client.24599.objecter dump_active .. 0 homeless 2024-04-20T02:29:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 client.24599.objecter ms_handle_connect 0x7f79e406aac0 2024-04-20T02:29:03.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 client.24599.objecter resend_mon_ops 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79e935b700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 0x7f79e41abbd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79e00090b0 tx=0x7f79e0004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79e9b5c700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79c803a2f0 0x7f79c803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79e9b5c700 10 monclient: get_auth_request con 0x7f79c803a2f0 auth_method 0 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e0020070 con 0x7f79e406aac0 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:03.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 monclient: dump: 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.246 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.247 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f79e00038b0 con 0x7f79e406aac0 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 msgr2=0x7f79e41abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 0x7f79e41abbd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79e00090b0 tx=0x7f79e0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 msgr2=0x7f79e41abbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e406aac0 0x7f79e41abbd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79e9b5c700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79c803a2f0 0x7f79c803c7b0 secure :-1 s=READY pgs=3301 cs=0 l=1 rev1=1 crypto rx=0x7f79dc001c70 tx=0x7f79dc001c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.205+0000 7f79d9ffb700 10 client.24599.objecter ms_handle_connect 0x7f79c803a2f0 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79eb5bf700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 0x7f79e41a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79eb5bf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79e406a400 con 0x7f79e41a83e0 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79e935b700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 0x7f79e41a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79e935b700 10 monclient: get_auth_request con 0x7f79e41a83e0 auth_method 0 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79e935b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79e935b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.213+0000 7f79e935b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79e935b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79e935b700 10 monclient(hunting): handle_auth_done global_id 84190 payload 824 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79e935b700 1 -- 172.21.15.47:0/2640705021 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f79e40a02c0 con 0x7f79e41a83e0 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79e935b700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 0x7f79e41a87f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79e0027000 tx=0x7f79e0011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 client.24599.objecter ms_handle_connect 0x7f79e41a83e0 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 client.24599.objecter resend_mon_ops 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79e001b9e0 con 0x7f79e41a83e0 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 monclient: dump: 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.250 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f79e001b570 con 0x7f79e41a83e0 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 msgr2=0x7f79e41a87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 0x7f79e41a87f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79e0027000 tx=0x7f79e0011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 msgr2=0x7f79e41a87f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.214+0000 7f79d9ffb700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79e41a83e0 0x7f79e41a87f0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79e0027000 tx=0x7f79e0011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:03.252 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:03.253 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 8, 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.254 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.255 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.256 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:03.257 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:03.258 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.260 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.262 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:03.263 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.266 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.268 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:03.271 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:03.272 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.275 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:03.276 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.277 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:03.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:03.279 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:03.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 10 librados: watch_flush enter 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 10 librados: watch_flush exit 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 20 client.24599.objecter shutdown clearing up homeless session... 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 10 client.24599.objecter successfully canceled tick 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79c803a2f0 msgr2=0x7f79c803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79c803a2f0 0x7f79c803c7b0 secure :-1 s=READY pgs=3301 cs=0 l=1 rev1=1 crypto rx=0x7f79dc001c70 tx=0x7f79dc001c00 comp rx=0 tx=0).stop 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 10 monclient: shutdown 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79e4071f30 msgr2=0x7f79e41ba850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 --2- 172.21.15.47:0/2640705021 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79e4071f30 0x7f79e41ba850 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f79d400e980 tx=0x7f79d400ec90 comp rx=0 tx=0).stop 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79e9b5c700 1 -- 172.21.15.47:0/2640705021 reap_dead start 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79e9b5c700 1 -- 172.21.15.47:0/2640705021 reap_dead start 2024-04-20T02:29:03.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 shutdown_connections 2024-04-20T02:29:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 >> 172.21.15.47:0/2640705021 conn(0x7f79e40a7570 msgr2=0x7f79e4067730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 shutdown_connections 2024-04-20T02:29:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 -- 172.21.15.47:0/2640705021 wait complete. 2024-04-20T02:29:03.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.215+0000 7f79eb5bf700 1 librados: shutdown 2024-04-20T02:29:03.283 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:03.283 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:29:03.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:29:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.420+0000 7fe19829d700 10 monclient: get_monmap_and_config 2024-04-20T02:29:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.420+0000 7fe19829d700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.420+0000 7fe19829d700 10 monclient: monmap: 2024-04-20T02:29:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:03.422071+0000 2024-04-20T02:29:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:03.422071+0000 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:03.424 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.421+0000 7fe19829d700 1 Processor -- start 2024-04-20T02:29:03.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.421+0000 7fe19829d700 1 -- start start 2024-04-20T02:29:03.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.421+0000 7fe19829d700 10 monclient: init 2024-04-20T02:29:03.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient: _add_conns ranks=[6,9,7,1,5,8,4,10,2,0,3] 2024-04-20T02:29:03.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1900a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): picked mon.noname-d con 0x7fe190137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:03.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 0x7fe1900a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): picked mon.noname-a con 0x7fe190138170 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe1900a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): picked mon.noname-i con 0x7fe1901385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19009fa40 con 0x7fe190138170 2024-04-20T02:29:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe19009dfc0 con 0x7fe190137d90 2024-04-20T02:29:03.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1900a7bc0 con 0x7fe1901385b0 2024-04-20T02:29:03.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19829d700 10 monclient(hunting): authenticate will time out at 2422.823387s 2024-04-20T02:29:03.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1900a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 0x7fe1900a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1900a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59116/0 (socket says 172.21.15.47:59116) 2024-04-20T02:29:03.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 0x7fe1900a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45804/0 (socket says 172.21.15.47:45804) 2024-04-20T02:29:03.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 -- 172.21.15.47:0/548999952 learned_addr learned my addr 172.21.15.47:0/548999952 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): get_auth_request con 0x7fe190137d90 auth_method 0 2024-04-20T02:29:03.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): get_auth_request con 0x7fe190138170 auth_method 0 2024-04-20T02:29:03.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19683a700 1 --2- 172.21.15.47:0/548999952 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe1900a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19683a700 10 monclient(hunting): get_auth_request con 0x7fe1901385b0 auth_method 0 2024-04-20T02:29:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19683a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19683a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe19683a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient(hunting): handle_auth_done global_id 54942 payload 274 2024-04-20T02:29:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 -- 172.21.15.47:0/548999952 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 msgr2=0x7fe1900a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 --2- 172.21.15.47:0/548999952 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe1900a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 -- 172.21.15.47:0/548999952 >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 msgr2=0x7fe1900a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 --2- 172.21.15.47:0/548999952 >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 0x7fe1900a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 monclient: found mon.noname-d 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe196039700 1 -- 172.21.15.47:0/548999952 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900751e0 con 0x7fe190137d90 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.422+0000 7fe195838700 1 --2- 172.21.15.47:0/548999952 >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 0x7fe1900a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe196039700 1 --2- 172.21.15.47:0/548999952 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1900a09e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009920 tx=0x7fe18c009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4dc93b0e7564d414 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 1 -- 172.21.15.47:0/548999952 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe18c011aa0 con 0x7fe190137d90 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: dump: 2024-04-20T02:29:03.451 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.452 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 1 -- 172.21.15.47:0/548999952 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c011c00 con 0x7fe190137d90 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 1 -- 172.21.15.47:0/548999952 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe18c011d70 con 0x7fe190137d90 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 5 monclient: authenticate success, global_id 54942 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 4 set_mon_vals no callback set 2024-04-20T02:29:03.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe195037700 10 monclient: dump: 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.454 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 10 monclient: shutdown 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 -- 172.21.15.47:0/548999952 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 msgr2=0x7fe1900a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 --2- 172.21.15.47:0/548999952 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1900a09e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009920 tx=0x7fe18c009c30 comp rx=0 tx=0).stop 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 -- 172.21.15.47:0/548999952 shutdown_connections 2024-04-20T02:29:03.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 --2- 172.21.15.47:0/548999952 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe1900a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 --2- 172.21.15.47:0/548999952 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1900a09e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 --2- 172.21.15.47:0/548999952 >> v2:172.21.15.47:3300/0 conn(0x7fe190138170 0x7fe1900a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.423+0000 7fe19829d700 1 -- 172.21.15.47:0/548999952 >> 172.21.15.47:0/548999952 conn(0x7fe1900aa8b0 msgr2=0x7fe1900a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.424+0000 7fe19829d700 1 -- 172.21.15.47:0/548999952 shutdown_connections 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.424+0000 7fe19829d700 1 -- 172.21.15.47:0/548999952 wait complete. 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.424+0000 7fe19829d700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.424+0000 7fe19829d700 10 monclient: monmap: 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:03.456 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.424+0000 7fe19829d700 1 librados: starting msgr at 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.424+0000 7fe19829d700 1 librados: starting objecter 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 Processor -- start 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 -- start start 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 librados: setting wanted keys 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 librados: calling monclient init 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient: init 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient: _add_conns ranks=[6,8,7,4,3,5,2,1,0] 2024-04-20T02:29:03.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1901cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): picked mon.noname-d con 0x7fe190137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe190138170 0x7fe190137050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): picked mon.noname-e con 0x7fe190138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe190120b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): picked mon.noname-i con 0x7fe1901385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901cd160 con 0x7fe190137d90 2024-04-20T02:29:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901cc780 con 0x7fe1901385b0 2024-04-20T02:29:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe19009e9e0 con 0x7fe190138170 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19829d700 10 monclient(hunting): authenticate will time out at 2422.826709s 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1901cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe195838700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe190138170 0x7fe190137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1901cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59126/0 (socket says 172.21.15.47:59126) 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 1 -- 172.21.15.47:0/1751422987 learned_addr learned my addr 172.21.15.47:0/1751422987 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): get_auth_request con 0x7fe190137d90 auth_method 0 2024-04-20T02:29:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe195838700 10 monclient(hunting): get_auth_request con 0x7fe190138170 auth_method 0 2024-04-20T02:29:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe195838700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe195838700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe195838700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19683a700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe190120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19683a700 10 monclient(hunting): get_auth_request con 0x7fe1901385b0 auth_method 0 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19683a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19683a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe19683a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.425+0000 7fe196039700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 msgr2=0x7fe1901cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe196039700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 msgr2=0x7fe1901cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe196039700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1901cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe196039700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1901cc240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19683a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19683a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19683a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 10 monclient(hunting): handle_auth_done global_id 35738 payload 824 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 msgr2=0x7fe190120b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe1901385b0 0x7fe190120b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 msgr2=0x7fe1901cc240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190137d90 0x7fe1901cc240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 monclient: found mon.noname-e 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840094a0 con 0x7fe190138170 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe195838700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe190138170 0x7fe190137050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe18400e980 tx=0x7fe18400ec90 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 client.?.objecter ms_handle_connect 0x7fe190138170 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe1840036a0 con 0x7fe190138170 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: dump: 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.466 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 5 monclient: authenticate success, global_id 35738 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 10 monclient: _renew_subs 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184003820 con 0x7fe190138170 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c009550 con 0x7fe190138170 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe184003e00 con 0x7fe190138170 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 10 client.35738.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe194836700 4 set_mon_vals no callback set 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe182ffd700 10 monclient: dump: 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.468 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 10 monclient: _renew_subs 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:03.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190122fb0 con 0x7fe190138170 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.426+0000 7fe19829d700 1 librados: init done 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe19829d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe19829d700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 0x7fe190137590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe190068180 con 0x7fe19006aac0 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe184003990 con 0x7fe190138170 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 0x7fe190137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient: get_auth_request con 0x7fe19006aac0 auth_method 0 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe17403a2f0 0x7fe17403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(516..516 src has 266..516) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fe18404a7a0 con 0x7fe190138170 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 10 client.35738.objecter ms_dispatch 0x7fe190001550 osd_map(516..516 src has 266..516) v4 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 3 client.35738.objecter handle_osd_map got epochs [516,516] > 0 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 3 client.35738.objecter handle_osd_map decoding full epoch 516 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 10 monclient(hunting): handle_auth_done global_id 54960 payload 824 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe19009f760 con 0x7fe19006aac0 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 20 client.35738.objecter dump_active .. 0 homeless 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe19683a700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe17403a2f0 0x7fe17403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe196039700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 0x7fe190137590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009840 tx=0x7fe18c004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe19683a700 10 monclient: get_auth_request con 0x7fe17403a2f0 auth_method 0 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.427+0000 7fe182ffd700 10 client.35738.objecter ms_handle_connect 0x7fe19006aac0 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 client.35738.objecter resend_mon_ops 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe18c020070 con 0x7fe19006aac0 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 monclient: dump: 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe18c0043b0 con 0x7fe19006aac0 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe19683a700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe17403a2f0 0x7fe17403c7b0 secure :-1 s=READY pgs=3302 cs=0 l=1 rev1=1 crypto rx=0x7fe190124b00 tx=0x7fe188009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 msgr2=0x7fe190137590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 0x7fe190137590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009840 tx=0x7fe18c004750 comp rx=0 tx=0).stop 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 msgr2=0x7fe190137590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe19006aac0 0x7fe190137590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.428+0000 7fe182ffd700 10 client.35738.objecter ms_handle_connect 0x7fe17403a2f0 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe19829d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe19829d700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 0x7fe190121490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe19829d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe19006a400 con 0x7fe190121080 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 0x7fe190121490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient: get_auth_request con 0x7fe190121080 auth_method 0 2024-04-20T02:29:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 10 monclient(hunting): handle_auth_done global_id 54969 payload 824 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 1 -- 172.21.15.47:0/1751422987 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe190124690 con 0x7fe190121080 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe182ffd700 10 client.35738.objecter ms_handle_connect 0x7fe190121080 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe182ffd700 10 client.35738.objecter resend_mon_ops 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe196039700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 0x7fe190121490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009130 tx=0x7fe18c011000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe18c02cb40 con 0x7fe190121080 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe182ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe182ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.436+0000 7fe182ffd700 10 monclient: dump: 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.476 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fe18c02cca0 con 0x7fe190121080 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 msgr2=0x7fe190121490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 0x7fe190121490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009130 tx=0x7fe18c011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 msgr2=0x7fe190121490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe182ffd700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe190121080 0x7fe190121490 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009130 tx=0x7fe18c011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:03.478 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 8, 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.479 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.480 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:03.481 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:03.482 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.483 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.485 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.487 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.491 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.493 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:03.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.495 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.497 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.498 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.500 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.501 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:29:03.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:03.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:03.504 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 10 librados: watch_flush enter 2024-04-20T02:29:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 10 librados: watch_flush exit 2024-04-20T02:29:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 20 client.35738.objecter shutdown clearing up homeless session... 2024-04-20T02:29:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 10 client.35738.objecter successfully canceled tick 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe17403a2f0 msgr2=0x7fe17403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe17403a2f0 0x7fe17403c7b0 secure :-1 s=READY pgs=3302 cs=0 l=1 rev1=1 crypto rx=0x7fe190124b00 tx=0x7fe188009780 comp rx=0 tx=0).stop 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 10 monclient: shutdown 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe190138170 msgr2=0x7fe190137050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe190138170 0x7fe190137050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe18400e980 tx=0x7fe18400ec90 comp rx=0 tx=0).stop 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19683a700 1 -- 172.21.15.47:0/1751422987 reap_dead start 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 shutdown_connections 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 --2- 172.21.15.47:0/1751422987 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe190138170 0x7fe190137050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 >> 172.21.15.47:0/1751422987 conn(0x7fe1900aa8b0 msgr2=0x7fe1901bf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 shutdown_connections 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 -- 172.21.15.47:0/1751422987 wait complete. 2024-04-20T02:29:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.437+0000 7fe19829d700 1 librados: shutdown 2024-04-20T02:29:03.508 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:03.508 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:29:03.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:29:03.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.646+0000 7fdc8f410700 10 monclient: get_monmap_and_config 2024-04-20T02:29:03.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.646+0000 7fdc8f410700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.646+0000 7fdc8f410700 10 monclient: monmap: 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:03.647586+0000 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:03.647586+0000 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.647 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:03.648 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:03.648 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 Processor -- start 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 -- start start 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient: init 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient: _add_conns ranks=[9,10,4,5,2,6,8,0,7,3,1] 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 0x7fdc880a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): picked mon.noname-a con 0x7fdc880a1510 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdc8809f9a0 0x7fdc880a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): picked mon.noname-f con 0x7fdc8809f9a0 addr v2:172.21.15.179:3300/0 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 0x7fdc880a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): picked mon.noname-c con 0x7fdc8809fe00 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc880720a0 con 0x7fdc880a1510 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc88070640 con 0x7fdc8809f9a0 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88071d30 con 0x7fdc8809fe00 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.647+0000 7fdc8f410700 10 monclient(hunting): authenticate will time out at 2423.048891s 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 0x7fdc880a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 0x7fdc880a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 0x7fdc880a3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54864/0 (socket says 172.21.15.47:54864) 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 0x7fdc880a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45820/0 (socket says 172.21.15.47:45820) 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/550694363 learned_addr learned my addr 172.21.15.47:0/550694363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): get_auth_request con 0x7fdc8809fe00 auth_method 0 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): get_auth_request con 0x7fdc880a1510 auth_method 0 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8c9ab700 1 --2- 172.21.15.47:0/550694363 >> v2:172.21.15.179:3300/0 conn(0x7fdc8809f9a0 0x7fdc880a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8c9ab700 10 monclient(hunting): get_auth_request con 0x7fdc8809f9a0 auth_method 0 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8c9ab700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8c9ab700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8c9ab700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_done global_id 84199 payload 274 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/550694363 >> v2:172.21.15.179:3300/0 conn(0x7fdc8809f9a0 msgr2=0x7fdc880a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/550694363 >> v2:172.21.15.179:3300/0 conn(0x7fdc8809f9a0 0x7fdc880a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/550694363 >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 msgr2=0x7fdc880a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/550694363 >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 0x7fdc880a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 monclient: found mon.noname-c 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/550694363 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880751e0 con 0x7fdc8809fe00 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d1ac700 1 --2- 172.21.15.47:0/550694363 >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 0x7fdc880a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/550694363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 0x7fdc880a3e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc880a8fa0 tx=0x7fdc8000b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=df7d2c7da7f95cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 1 -- 172.21.15.47:0/550694363 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc8000e070 con 0x7fdc8809fe00 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: dump: 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.662 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8f410700 5 monclient: authenticate success, global_id 84199 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 1 -- 172.21.15.47:0/550694363 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8000bd80 con 0x7fdc8809fe00 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8f410700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8f410700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc8f410700 4 set_mon_vals no callback set 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 1 -- 172.21.15.47:0/550694363 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc80012860 con 0x7fdc8809fe00 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.648+0000 7fdc7ffff700 10 monclient: dump: 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.664 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 10 monclient: shutdown 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 -- 172.21.15.47:0/550694363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 msgr2=0x7fdc880a3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 --2- 172.21.15.47:0/550694363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 0x7fdc880a3e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc880a8fa0 tx=0x7fdc8000b330 comp rx=0 tx=0).stop 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 -- 172.21.15.47:0/550694363 shutdown_connections 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 --2- 172.21.15.47:0/550694363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809fe00 0x7fdc880a3e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 --2- 172.21.15.47:0/550694363 >> v2:172.21.15.47:3300/0 conn(0x7fdc880a1510 0x7fdc880a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 --2- 172.21.15.47:0/550694363 >> v2:172.21.15.179:3300/0 conn(0x7fdc8809f9a0 0x7fdc880a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 -- 172.21.15.47:0/550694363 >> 172.21.15.47:0/550694363 conn(0x7fdc880a8510 msgr2=0x7fdc880a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 -- 172.21.15.47:0/550694363 shutdown_connections 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.649+0000 7fdc8f410700 1 -- 172.21.15.47:0/550694363 wait complete. 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 10 monclient: monmap: 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:03.666 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 1 librados: starting msgr at 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 1 librados: starting objecter 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 1 Processor -- start 2024-04-20T02:29:03.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 1 -- start start 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 1 librados: setting wanted keys 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 1 librados: calling monclient init 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.650+0000 7fdc8f410700 10 monclient: init 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient: _add_conns ranks=[4,6,2,7,0,3,1,8,5] 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 0x7fdc881cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): picked mon.noname-c con 0x7fdc8809f9a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fdc8809fe00 0x7fdc881359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): picked mon.noname-d con 0x7fdc8809fe00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:03.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc880a1510 0x7fdc88135f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): picked mon.noname-b con 0x7fdc880a1510 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880a2130 con 0x7fdc880a1510 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88070ff0 con 0x7fdc8809f9a0 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc881253c0 con 0x7fdc8809fe00 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8f410700 10 monclient(hunting): authenticate will time out at 2423.052131s 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 0x7fdc881cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 0x7fdc881cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54878/0 (socket says 172.21.15.47:54878) 2024-04-20T02:29:03.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 -- 172.21.15.47:0/1734258347 learned_addr learned my addr 172.21.15.47:0/1734258347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): get_auth_request con 0x7fdc8809f9a0 auth_method 0 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc880a1510 0x7fdc88135f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fdc8809fe00 0x7fdc881359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): get_auth_request con 0x7fdc880a1510 auth_method 0 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): get_auth_request con 0x7fdc8809fe00 auth_method 0 2024-04-20T02:29:03.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8c9ab700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient(hunting): handle_auth_done global_id 84208 payload 824 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fdc8809fe00 msgr2=0x7fdc881359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fdc8809fe00 0x7fdc881359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc880a1510 msgr2=0x7fdc88135f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc880a1510 0x7fdc88135f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 monclient: found mon.noname-c 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc840094a0 con 0x7fdc8809f9a0 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc8d1ac700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 0x7fdc881cc170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400ebe0 tx=0x7fdc8400eef0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 client.?.objecter ms_handle_connect 0x7fdc8809f9a0 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc84009760 con 0x7fdc8809f9a0 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 monclient: dump: 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.672 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.651+0000 7fdc7dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8400ccc0 con 0x7fdc8809f9a0 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 5 monclient: authenticate success, global_id 84208 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc8400ce30 con 0x7fdc8809f9a0 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7f7fe700 4 set_mon_vals no callback set 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 10 monclient: dump: 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.674 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 10 monclient: _renew_subs 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc80009b30 con 0x7fdc8809f9a0 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 10 client.84208.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:03.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 10 monclient: _renew_subs 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc88124b20 con 0x7fdc8809f9a0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc8f410700 1 librados: init done 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.652+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fdc8400c850 con 0x7fdc8809f9a0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8f410700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc6c03a2f0 0x7fdc6c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8f410700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 0x7fdc8806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880a7cb0 con 0x7fdc8806aac0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(516..516 src has 266..516) v4 ==== 6549+0+0 (secure 0 0 0) 0x7fdc8404a950 con 0x7fdc8809f9a0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 10 client.84208.objecter ms_dispatch 0x7fdc88001550 osd_map(516..516 src has 266..516) v4 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 3 client.84208.objecter handle_osd_map got epochs [516,516] > 0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 3 client.84208.objecter handle_osd_map decoding full epoch 516 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 0x7fdc8806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient: get_auth_request con 0x7fdc8806aac0 auth_method 0 2024-04-20T02:29:03.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 20 client.84208.objecter dump_active .. 0 homeless 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8c9ab700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc6c03a2f0 0x7fdc6c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8c9ab700 10 monclient: get_auth_request con 0x7fdc6c03a2f0 auth_method 0 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_done global_id 35747 payload 824 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc880a1e30 con 0x7fdc8806aac0 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 10 client.84208.objecter ms_handle_connect 0x7fdc8806aac0 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 10 client.84208.objecter resend_mon_ops 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 0x7fdc8806c7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc880a8fa0 tx=0x7fdc80004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc800047d0 con 0x7fdc8806aac0 2024-04-20T02:29:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc8c9ab700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc6c03a2f0 0x7fdc6c03c7b0 secure :-1 s=READY pgs=3303 cs=0 l=1 rev1=1 crypto rx=0x7fdc88124a50 tx=0x7fdc78008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.653+0000 7fdc7dffb700 10 monclient: dump: 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 10 client.84208.objecter ms_handle_connect 0x7fdc6c03a2f0 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdc80004930 con 0x7fdc8806aac0 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 msgr2=0x7fdc8806c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 0x7fdc8806c7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc880a8fa0 tx=0x7fdc80004ad0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 msgr2=0x7fdc8806c7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.654+0000 7fdc7dffb700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806aac0 0x7fdc8806c7a0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc880a8fa0 tx=0x7fdc80004ad0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8f410700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8f410700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 0x7fdc8806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8f410700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88124cb0 con 0x7fdc8806cce0 2024-04-20T02:29:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 0x7fdc8806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient: get_auth_request con 0x7fdc8806cce0 auth_method 0 2024-04-20T02:29:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 10 monclient(hunting): handle_auth_done global_id 35756 payload 824 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/1734258347 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdc88070960 con 0x7fdc8806cce0 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc7dffb700 10 client.84208.objecter ms_handle_connect 0x7fdc8806cce0 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc8d9ad700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 0x7fdc8806d0f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc881ccc20 tx=0x7fdc80009810 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc7dffb700 10 client.84208.objecter resend_mon_ops 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc8001d890 con 0x7fdc8806cce0 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc7dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc7dffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.662+0000 7fdc7dffb700 10 monclient: dump: 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.684 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fdc8001d420 con 0x7fdc8806cce0 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 msgr2=0x7fdc8806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 0x7fdc8806d0f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc881ccc20 tx=0x7fdc80009810 comp rx=0 tx=0).stop 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 msgr2=0x7fdc8806d0f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc7dffb700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdc8806cce0 0x7fdc8806d0f0 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc881ccc20 tx=0x7fdc80009810 comp rx=0 tx=0).stop 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:03.686 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 8, 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.687 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.688 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:03.689 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.690 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:03.691 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.692 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:03.694 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:03.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.698 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.701 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:03.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.706 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:03.707 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 10 librados: watch_flush enter 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 10 librados: watch_flush exit 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 20 client.84208.objecter shutdown clearing up homeless session... 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 10 client.84208.objecter successfully canceled tick 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc6c03a2f0 msgr2=0x7fdc6c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdc6c03a2f0 0x7fdc6c03c7b0 secure :-1 s=READY pgs=3303 cs=0 l=1 rev1=1 crypto rx=0x7fdc88124a50 tx=0x7fdc78008040 comp rx=0 tx=0).stop 2024-04-20T02:29:03.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 10 monclient: shutdown 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 msgr2=0x7fdc881cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 0x7fdc881cc170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400ebe0 tx=0x7fdc8400eef0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8d9ad700 1 -- 172.21.15.47:0/1734258347 reap_dead start 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 shutdown_connections 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 --2- 172.21.15.47:0/1734258347 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc8809f9a0 0x7fdc881cc170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 >> 172.21.15.47:0/1734258347 conn(0x7fdc880a8510 msgr2=0x7fdc880a6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 shutdown_connections 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 -- 172.21.15.47:0/1734258347 wait complete. 2024-04-20T02:29:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.663+0000 7fdc8f410700 1 librados: shutdown 2024-04-20T02:29:03.711 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:03.711 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:29:03.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:29:03.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.849+0000 7f6c6d216700 10 monclient: get_monmap_and_config 2024-04-20T02:29:03.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.849+0000 7f6c6d216700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.849+0000 7f6c6d216700 10 monclient: monmap: 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:03.850656+0000 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:03.850656+0000 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.850 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.851 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.851 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:03.851 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:03.851 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 Processor -- start 2024-04-20T02:29:03.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 -- start start 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient: init 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient: _add_conns ranks=[1,6,4,10,7,2,0,9,8,5,3] 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6c680a0830 addr v1:172.21.15.179:6789/0 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c6813cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): picked mon.noname-d con 0x7f6c680a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c6813d280 0x7f6c6813f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): picked mon.noname-c con 0x7f6c6813d280 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680751e0 con 0x7f6c680a0830 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6809dee0 con 0x7f6c6813d280 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68067820 con 0x7f6c680a01b0 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6d216700 10 monclient(hunting): authenticate will time out at 2423.251724s 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c6813cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6759e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c6813d280 0x7f6c6813f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c6813cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59162/0 (socket says 172.21.15.47:59162) 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 1 -- 172.21.15.47:0/384747293 learned_addr learned my addr 172.21.15.47:0/384747293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 10 monclient(hunting): get_auth_request con 0x7f6c680a01b0 auth_method 0 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6759e700 10 monclient(hunting): get_auth_request con 0x7f6c6813d280 auth_method 0 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.850+0000 7f6c6759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 10 monclient(hunting): handle_auth_done global_id 84217 payload 274 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 1 -- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 msgr2=0x7f6c6813cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 1 --2- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c6813cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 1 -- 172.21.15.47:0/384747293 >> v1:172.21.15.179:6789/0 conn(0x7f6c680a0830 legacy=0x7f6c680a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 1 monclient: found mon.noname-c 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 1 -- 172.21.15.47:0/384747293 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6809eee0 con 0x7f6c6813d280 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6659c700 1 --2- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c6813cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:03.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6759e700 1 --2- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c6813d280 0x7f6c6813f670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009df0 tx=0x7f6c6000b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d5f7d4c5ab883f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 1 -- 172.21.15.47:0/384747293 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c6000e070 con 0x7f6c6813d280 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: dump: 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.864 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 1 -- 172.21.15.47:0/384747293 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c60012430 con 0x7f6c6813d280 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 1 -- 172.21.15.47:0/384747293 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c60012a10 con 0x7f6c6813d280 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6d216700 5 monclient: authenticate success, global_id 84217 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6d216700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c6d216700 4 set_mon_vals no callback set 2024-04-20T02:29:03.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.851+0000 7f6c65d9b700 10 monclient: dump: 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.866 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 10 monclient: shutdown 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 -- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c6813d280 msgr2=0x7f6c6813f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 --2- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c6813d280 0x7f6c6813f670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009df0 tx=0x7f6c6000b330 comp rx=0 tx=0).stop 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 -- 172.21.15.47:0/384747293 shutdown_connections 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 --2- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c6813d280 0x7f6c6813f670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 --2- 172.21.15.47:0/384747293 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c6813cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 -- 172.21.15.47:0/384747293 >> 172.21.15.47:0/384747293 conn(0x7f6c680a7570 msgr2=0x7f6c680a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 -- 172.21.15.47:0/384747293 shutdown_connections 2024-04-20T02:29:03.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.852+0000 7f6c6d216700 1 -- 172.21.15.47:0/384747293 wait complete. 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 10 monclient: build_initial_monmap 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 10 monclient: monmap: 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:03.868 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 1 librados: starting msgr at 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 1 librados: starting objecter 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 1 Processor -- start 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 1 -- start start 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 1 librados: setting wanted keys 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 1 librados: calling monclient init 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.853+0000 7f6c6d216700 10 monclient: init 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient: _add_conns ranks=[6,4,1,0,2,5,7,3,8] 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c681cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): picked mon.noname-d con 0x7f6c680a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c680a0830 0x7f6c681cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): picked mon.noname-c con 0x7f6c680a0830 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:03.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6813d280 0x7f6c681cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): picked mon.noname-f con 0x7f6c6813d280 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6806ff20 con 0x7f6c6813d280 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6809e900 con 0x7f6c680a0830 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6809e440 con 0x7f6c680a01b0 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6d216700 10 monclient(hunting): authenticate will time out at 2423.255278s 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c680a0830 0x7f6c681cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c681cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c680a0830 0x7f6c681cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54898/0 (socket says 172.21.15.47:54898) 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 1 -- 172.21.15.47:0/3821922194 learned_addr learned my addr 172.21.15.47:0/3821922194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:03.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): get_auth_request con 0x7f6c680a01b0 auth_method 0 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): get_auth_request con 0x7f6c680a0830 auth_method 0 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6759e700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6813d280 0x7f6c681cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6759e700 10 monclient(hunting): get_auth_request con 0x7f6c6813d280 auth_method 0 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient(hunting): handle_auth_done global_id 54996 payload 824 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c680a0830 msgr2=0x7f6c681cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c680a0830 0x7f6c681cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6813d280 msgr2=0x7f6c681cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6813d280 0x7f6c681cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 monclient: found mon.noname-d 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009550 con 0x7f6c680a01b0 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c6659c700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6c680a0830 0x7f6c681cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:03.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c66d9d700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c681cc410 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c440 tx=0x7f6c58001e70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c4f7fe700 10 client.?.objecter ms_handle_connect 0x7f6c680a01b0 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.854+0000 7f6c4f7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c5800ce90 con 0x7f6c680a01b0 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: dump: 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.873 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58001120 con 0x7f6c680a01b0 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 5 monclient: authenticate success, global_id 54996 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 10 monclient: _renew_subs 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c60009a70 con 0x7f6c680a01b0 2024-04-20T02:29:03.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c58005560 con 0x7f6c680a01b0 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6559a700 4 set_mon_vals no callback set 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c4f7fe700 10 monclient: dump: 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.875 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 10 client.54996.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 10 monclient: _renew_subs 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6813cb50 con 0x7f6c680a01b0 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.855+0000 7f6c6d216700 1 librados: init done 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6d216700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6d216700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 0x7f6c6813a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c68068310 con 0x7f6c6806aac0 2024-04-20T02:29:03.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6c580050d0 con 0x7f6c680a01b0 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 0x7f6c6813a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient: get_auth_request con 0x7f6c6806aac0 auth_method 0 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6c5403a2f0 0x7f6c5403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(516..516 src has 266..516) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f6c58049be0 con 0x7f6c680a01b0 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 10 client.54996.objecter ms_dispatch 0x7f6c68001550 osd_map(516..516 src has 266..516) v4 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 3 client.54996.objecter handle_osd_map got epochs [516,516] > 0 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 3 client.54996.objecter handle_osd_map decoding full epoch 516 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 20 client.54996.objecter dump_active .. 0 homeless 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6759e700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6c5403a2f0 0x7f6c5403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6759e700 10 monclient: get_auth_request con 0x7f6c5403a2f0 auth_method 0 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 10 monclient(hunting): handle_auth_done global_id 55351 payload 824 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c680672f0 con 0x7f6c6806aac0 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 10 client.54996.objecter ms_handle_connect 0x7f6c6806aac0 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c6659c700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 0x7f6c6813a600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006d50 tx=0x7f6c5c00f800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.856+0000 7f6c4f7fe700 10 client.54996.objecter resend_mon_ops 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c5c009ea0 con 0x7f6c6806aac0 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 10 monclient: dump: 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.878 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c6759e700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6c5403a2f0 0x7f6c5403c7b0 secure :-1 s=READY pgs=3304 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009df0 tx=0x7f6c60011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 10 client.54996.objecter ms_handle_connect 0x7f6c5403a2f0 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6c5c014440 con 0x7f6c6806aac0 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:03.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 msgr2=0x7f6c6813a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 0x7f6c6813a600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006d50 tx=0x7f6c5c00f800 comp rx=0 tx=0).stop 2024-04-20T02:29:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 msgr2=0x7f6c6813a600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.857+0000 7f6c4f7fe700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c6806aac0 0x7f6c6813a600 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006d50 tx=0x7f6c5c00f800 comp rx=0 tx=0).stop 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6d216700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6d216700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 0x7f6c6813ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6d216700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6806a400 con 0x7f6c68070be0 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 0x7f6c6813ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient: get_auth_request con 0x7f6c68070be0 auth_method 0 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:03.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 10 monclient(hunting): handle_auth_done global_id 55360 payload 824 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.865+0000 7f6c6659c700 1 -- 172.21.15.47:0/3821922194 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6c6809f1c0 con 0x7f6c68070be0 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 client.54996.objecter ms_handle_connect 0x7f6c68070be0 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 client.54996.objecter resend_mon_ops 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6659c700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 0x7f6c6813ab40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c6806aec0 tx=0x7f6c5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c5c022c00 con 0x7f6c68070be0 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 monclient: dump: 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:03.883 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f6c5c022d60 con 0x7f6c68070be0 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 msgr2=0x7f6c6813ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 0x7f6c6813ab40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c6806aec0 tx=0x7f6c5c004750 comp rx=0 tx=0).stop 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 msgr2=0x7f6c6813ab40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:03.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c4f7fe700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c68070be0 0x7f6c6813ab40 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c6806aec0 tx=0x7f6c5c004750 comp rx=0 tx=0).stop 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:03.885 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 8, 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.886 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.887 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:03.888 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:03.889 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.891 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:03.892 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.893 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.894 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.895 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.896 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.897 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.898 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.899 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.900 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.901 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:03.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.903 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:03.904 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:03.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6d216700 10 librados: watch_flush enter 2024-04-20T02:29:03.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6d216700 10 librados: watch_flush exit 2024-04-20T02:29:03.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6d216700 20 client.54996.objecter shutdown clearing up homeless session... 2024-04-20T02:29:03.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6d216700 10 client.54996.objecter successfully canceled tick 2024-04-20T02:29:03.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6c5403a2f0 msgr2=0x7f6c5403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.866+0000 7f6c6d216700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6c5403a2f0 0x7f6c5403c7b0 secure :-1 s=READY pgs=3304 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009df0 tx=0x7f6c60011000 comp rx=0 tx=0).stop 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 10 monclient: shutdown 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 msgr2=0x7f6c681cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 --2- 172.21.15.47:0/3821922194 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f6c680a01b0 0x7f6c681cc410 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c440 tx=0x7f6c58001e70 comp rx=0 tx=0).stop 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6759e700 1 -- 172.21.15.47:0/3821922194 reap_dead start 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6759e700 1 -- 172.21.15.47:0/3821922194 reap_dead start 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 shutdown_connections 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 >> 172.21.15.47:0/3821922194 conn(0x7f6c680a7570 msgr2=0x7f6c680a27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 shutdown_connections 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 -- 172.21.15.47:0/3821922194 wait complete. 2024-04-20T02:29:03.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:03.867+0000 7f6c6d216700 1 librados: shutdown 2024-04-20T02:29:03.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:03.908 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:29:03.908 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.045+0000 7f87ee28a700 10 monclient: get_monmap_and_config 2024-04-20T02:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.045+0000 7f87ee28a700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.045+0000 7f87ee28a700 10 monclient: monmap: 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:04.046624+0000 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:04.046624+0000 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:04.047 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 Processor -- start 2024-04-20T02:29:04.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 -- start start 2024-04-20T02:29:04.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient: init 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient: _add_conns ranks=[7,6,9,10,4,0,5,3,8,2,1] 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87e80a19d0 0x7f87e80a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): picked mon.noname-i con 0x7f87e80a19d0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 0x7f87e80701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): picked mon.noname-d con 0x7f87e80a42d0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87e8135db0 0x7f87e80706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): picked mon.noname-a con 0x7f87e8135db0 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e8136810 con 0x7f87e8135db0 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8072020 con 0x7f87e80a42d0 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8072300 con 0x7f87e80a19d0 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.046+0000 7f87ee28a700 10 monclient(hunting): authenticate will time out at 2423.447882s 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 0x7f87e80701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87e8135db0 0x7f87e80706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 0x7f87e80701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59184/0 (socket says 172.21.15.47:59184) 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 -- 172.21.15.47:0/1429791740 learned_addr learned my addr 172.21.15.47:0/1429791740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): get_auth_request con 0x7f87e80a42d0 auth_method 0 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): get_auth_request con 0x7f87e8135db0 auth_method 0 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e7fff700 1 --2- 172.21.15.47:0/1429791740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87e80a19d0 0x7f87e80a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e7fff700 10 monclient(hunting): get_auth_request con 0x7f87e80a19d0 auth_method 0 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient(hunting): handle_auth_done global_id 55005 payload 274 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 -- 172.21.15.47:0/1429791740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87e80a19d0 msgr2=0x7f87e80a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:04.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 --2- 172.21.15.47:0/1429791740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87e80a19d0 0x7f87e80a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 -- 172.21.15.47:0/1429791740 >> v2:172.21.15.47:3300/0 conn(0x7f87e8135db0 msgr2=0x7f87e80706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 --2- 172.21.15.47:0/1429791740 >> v2:172.21.15.47:3300/0 conn(0x7f87e8135db0 0x7f87e80706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 monclient: found mon.noname-d 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 -- 172.21.15.47:0/1429791740 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80751e0 con 0x7f87e80a42d0 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ec827700 1 --2- 172.21.15.47:0/1429791740 >> v2:172.21.15.47:3300/0 conn(0x7f87e8135db0 0x7f87e80706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e77fe700 1 --2- 172.21.15.47:0/1429791740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 0x7f87e80701b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098f0 tx=0x7f87dc009c00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1957a4572c6ea371 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 1 -- 172.21.15.47:0/1429791740 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87dc011aa0 con 0x7f87e80a42d0 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: dump: 2024-04-20T02:29:04.061 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.062 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ee28a700 5 monclient: authenticate success, global_id 55005 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ee28a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 1 -- 172.21.15.47:0/1429791740 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc011c00 con 0x7f87e80a42d0 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ee28a700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87ee28a700 4 set_mon_vals no callback set 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 1 -- 172.21.15.47:0/1429791740 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87dc011d70 con 0x7f87e80a42d0 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:04.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.047+0000 7f87e6ffd700 10 monclient: dump: 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.064 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 10 monclient: shutdown 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 -- 172.21.15.47:0/1429791740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 msgr2=0x7f87e80701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 --2- 172.21.15.47:0/1429791740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 0x7f87e80701b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098f0 tx=0x7f87dc009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 -- 172.21.15.47:0/1429791740 shutdown_connections 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 --2- 172.21.15.47:0/1429791740 >> v2:172.21.15.47:3300/0 conn(0x7f87e8135db0 0x7f87e80706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 --2- 172.21.15.47:0/1429791740 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f87e80a19d0 0x7f87e80a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 --2- 172.21.15.47:0/1429791740 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f87e80a42d0 0x7f87e80701b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 -- 172.21.15.47:0/1429791740 >> 172.21.15.47:0/1429791740 conn(0x7f87e80a4a50 msgr2=0x7f87e80a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 -- 172.21.15.47:0/1429791740 shutdown_connections 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.048+0000 7f87ee28a700 1 -- 172.21.15.47:0/1429791740 wait complete. 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 10 monclient: monmap: 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.065 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 1 librados: starting msgr at 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 1 librados: starting objecter 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 1 Processor -- start 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 1 -- start start 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 1 librados: setting wanted keys 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 1 librados: calling monclient init 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.049+0000 7f87ee28a700 10 monclient: init 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient: _add_conns ranks=[5,1,4,2,7,6,8,0,3] 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f87e80a19d0 0x7f87e81c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): picked mon.noname-h con 0x7f87e80a19d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80a42d0 0x7f87e8134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): picked mon.noname-f con 0x7f87e80a42d0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87e8135db0 0x7f87e8135220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): picked mon.noname-c con 0x7f87e8135db0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:04.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8071230 con 0x7f87e80a42d0 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87e81241b0 con 0x7f87e8135db0 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80715f0 con 0x7f87e80a19d0 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ee28a700 10 monclient(hunting): authenticate will time out at 2423.451187s 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87e8135db0 0x7f87e8135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87e8135db0 0x7f87e8135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54902/0 (socket says 172.21.15.47:54902) 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 -- 172.21.15.47:0/3091773557 learned_addr learned my addr 172.21.15.47:0/3091773557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): get_auth_request con 0x7f87e8135db0 auth_method 0 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80a42d0 0x7f87e8134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f87e80a19d0 0x7f87e81c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): get_auth_request con 0x7f87e80a42d0 auth_method 0 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): get_auth_request con 0x7f87e80a19d0 auth_method 0 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient(hunting): handle_auth_done global_id 84235 payload 824 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f87e80a19d0 msgr2=0x7f87e81c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f87e80a19d0 0x7f87e81c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80a42d0 msgr2=0x7f87e8134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87e80a42d0 0x7f87e8134ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 monclient: found mon.noname-c 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e0009a70 con 0x7f87e8135db0 2024-04-20T02:29:04.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87ec827700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87e8135db0 0x7f87e8135220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87e000efc0 tx=0x7f87e000c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e4ff9700 10 client.?.objecter ms_handle_connect 0x7f87e8135db0 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.050+0000 7f87e4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87e0014480 con 0x7f87e8135db0 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: dump: 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.071 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e000f040 con 0x7f87e8135db0 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 5 monclient: authenticate success, global_id 84235 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 monclient: _renew_subs 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc009550 con 0x7f87e8135db0 2024-04-20T02:29:04.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87e0014a90 con 0x7f87e8135db0 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 client.84235.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e5ffb700 4 set_mon_vals no callback set 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87e4ff9700 10 monclient: dump: 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.073 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 monclient: _renew_subs 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e8123e40 con 0x7f87e8135db0 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 1 librados: init done 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 0x7f87e81cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.051+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8068430 con 0x7f87e806aac0 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 0x7f87e81cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient: get_auth_request con 0x7f87e806aac0 auth_method 0 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f87e0014bf0 con 0x7f87e8135db0 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 10 monclient(hunting): handle_auth_done global_id 84225 payload 824 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87e8067f90 con 0x7f87e806aac0 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87d403a320 0x7f87d403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(516..516 src has 266..516) v4 ==== 6549+0+0 (secure 0 0 0) 0x7f87e004acc0 con 0x7f87e8135db0 2024-04-20T02:29:04.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 10 client.84235.objecter ms_dispatch 0x7f87e8001550 osd_map(516..516 src has 266..516) v4 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 3 client.84235.objecter handle_osd_map got epochs [516,516] > 0 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 3 client.84235.objecter handle_osd_map decoding full epoch 516 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e7fff700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 0x7f87e81cc1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87d8011600 tx=0x7f87d80119c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 20 client.84235.objecter dump_active .. 0 homeless 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 10 client.84235.objecter ms_handle_connect 0x7f87e806aac0 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 10 client.84235.objecter resend_mon_ops 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e77fe700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87d403a320 0x7f87d403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e77fe700 10 monclient: get_auth_request con 0x7f87d403a320 auth_method 0 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87d800c070 con 0x7f87e806aac0 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.052+0000 7f87e4ff9700 10 monclient: dump: 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.076 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f87d8004030 con 0x7f87e806aac0 2024-04-20T02:29:04.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 msgr2=0x7f87e81cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 0x7f87e81cc1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87d8011600 tx=0x7f87d80119c0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 msgr2=0x7f87e81cc1f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e806aac0 0x7f87e81cc1f0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87d8011600 tx=0x7f87d80119c0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e4ff9700 10 client.84235.objecter ms_handle_connect 0x7f87d403a320 2024-04-20T02:29:04.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.053+0000 7f87e77fe700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87d403a320 0x7f87d403c7e0 secure :-1 s=READY pgs=3305 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098f0 tx=0x7f87dc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87ee28a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87ee28a700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 0x7f87e81ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87ee28a700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e806a400 con 0x7f87e81cc730 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 0x7f87e81ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient: get_auth_request con 0x7f87e81cc730 auth_method 0 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 10 monclient(hunting): handle_auth_done global_id 84234 payload 824 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.061+0000 7f87e7fff700 1 -- 172.21.15.47:0/3091773557 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f87e81cd080 con 0x7f87e81cc730 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e7fff700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 0x7f87e81ccb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87d8012010 tx=0x7f87d8004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 client.84235.objecter ms_handle_connect 0x7f87e81cc730 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 client.84235.objecter resend_mon_ops 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f87d8015cf0 con 0x7f87e81cc730 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 monclient: dump: 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.081 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f87d8015e50 con 0x7f87e81cc730 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 msgr2=0x7f87e81ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 0x7f87e81ccb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87d8012010 tx=0x7f87d8004750 comp rx=0 tx=0).stop 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 msgr2=0x7f87e81ccb40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.062+0000 7f87e4ff9700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f87e81cc730 0x7f87e81ccb40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:04.083 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 8, 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:04.084 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.085 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:04.086 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.087 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.088 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:04.089 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.090 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.091 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:04.092 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.093 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.094 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:04.095 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.096 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.098 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.099 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:04.100 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:04.102 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.103 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:04.104 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 10 librados: watch_flush enter 2024-04-20T02:29:04.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 10 librados: watch_flush exit 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 20 client.84235.objecter shutdown clearing up homeless session... 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 10 client.84235.objecter successfully canceled tick 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87d403a320 msgr2=0x7f87d403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f87d403a320 0x7f87d403c7e0 secure :-1 s=READY pgs=3305 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098f0 tx=0x7f87dc008030 comp rx=0 tx=0).stop 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 10 monclient: shutdown 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87e8135db0 msgr2=0x7f87e8135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 --2- 172.21.15.47:0/3091773557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f87e8135db0 0x7f87e8135220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87e000efc0 tx=0x7f87e000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ec827700 1 -- 172.21.15.47:0/3091773557 reap_dead start 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ec827700 1 -- 172.21.15.47:0/3091773557 reap_dead start 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 shutdown_connections 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 >> 172.21.15.47:0/3091773557 conn(0x7f87e80a4a50 msgr2=0x7f87e81bd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 shutdown_connections 2024-04-20T02:29:04.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 -- 172.21.15.47:0/3091773557 wait complete. 2024-04-20T02:29:04.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.063+0000 7f87ee28a700 1 librados: shutdown 2024-04-20T02:29:04.107 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:04.107 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:29:04.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.264+0000 7f8b25d62700 10 monclient: get_monmap_and_config 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.264+0000 7f8b25d62700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.264+0000 7f8b25d62700 10 monclient: monmap: 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:04.265932+0000 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:04.265932+0000 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.266+0000 7f8b25d62700 1 Processor -- start 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.266+0000 7f8b25d62700 1 -- start start 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.266+0000 7f8b25d62700 10 monclient: init 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.266+0000 7f8b25d62700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.266+0000 7f8b25d62700 10 monclient: _add_conns ranks=[7,0,9,6,2,3,5,4,10,8,1] 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.266+0000 7f8b25d62700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 0x7f8b200a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): picked mon.noname-i con 0x7f8b200a5c90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b200a0eb0 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8b200a6780 0x7f8b20143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): picked mon.noname-a con 0x7f8b200a6780 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200751e0 con 0x7f8b200a0eb0 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2009fd20 con 0x7f8b200a6780 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200a7bc0 con 0x7f8b200a5c90 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b25d62700 10 monclient(hunting): authenticate will time out at 2423.668091s 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 0x7f8b200a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 0x7f8b200a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:53484/0 (socket says 172.21.15.47:53484) 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 1 -- 172.21.15.47:0/732069094 learned_addr learned my addr 172.21.15.47:0/732069094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): get_auth_request con 0x7f8b200a5c90 auth_method 0 2024-04-20T02:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1ffff700 1 --2- 172.21.15.47:0/732069094 >> v2:172.21.15.47:3300/0 conn(0x7f8b200a6780 0x7f8b20143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1ffff700 10 monclient(hunting): get_auth_request con 0x7f8b200a6780 auth_method 0 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_done global_id 84220 payload 274 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 1 -- 172.21.15.47:0/732069094 >> v2:172.21.15.47:3300/0 conn(0x7f8b200a6780 msgr2=0x7f8b20143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/732069094 >> v2:172.21.15.47:3300/0 conn(0x7f8b200a6780 0x7f8b20143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.267+0000 7f8b1f7fe700 1 -- 172.21.15.47:0/732069094 >> v1:172.21.15.47:6789/0 conn(0x7f8b200a0eb0 legacy=0x7f8b200a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.268+0000 7f8b1f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.268+0000 7f8b1f7fe700 1 monclient: found mon.noname-i 2024-04-20T02:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.268+0000 7f8b1f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.268+0000 7f8b1f7fe700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.268+0000 7f8b1f7fe700 1 -- 172.21.15.47:0/732069094 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2009f760 con 0x7f8b200a5c90 2024-04-20T02:29:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.268+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/732069094 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 0x7f8b200a0970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b180098f0 tx=0x7f8b18009c00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=fc5662792560fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 1 -- 172.21.15.47:0/732069094 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b1800bb10 con 0x7f8b200a5c90 2024-04-20T02:29:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: dump: 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 1 -- 172.21.15.47:0/732069094 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1800bc70 con 0x7f8b200a5c90 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b25d62700 5 monclient: authenticate success, global_id 84220 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b25d62700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b25d62700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 1 -- 172.21.15.47:0/732069094 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b1800bde0 con 0x7f8b200a5c90 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b25d62700 4 set_mon_vals no callback set 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.269+0000 7f8b1e7fc700 10 monclient: dump: 2024-04-20T02:29:04.282 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 10 monclient: shutdown 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 -- 172.21.15.47:0/732069094 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 msgr2=0x7f8b200a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 --2- 172.21.15.47:0/732069094 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 0x7f8b200a0970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b180098f0 tx=0x7f8b18009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 -- 172.21.15.47:0/732069094 shutdown_connections 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 --2- 172.21.15.47:0/732069094 >> v2:172.21.15.47:3300/0 conn(0x7f8b200a6780 0x7f8b20143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 --2- 172.21.15.47:0/732069094 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a5c90 0x7f8b200a0970 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 -- 172.21.15.47:0/732069094 >> 172.21.15.47:0/732069094 conn(0x7f8b200aa8b0 msgr2=0x7f8b200a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 -- 172.21.15.47:0/732069094 shutdown_connections 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.270+0000 7f8b25d62700 1 -- 172.21.15.47:0/732069094 wait complete. 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.271+0000 7f8b25d62700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.271+0000 7f8b25d62700 10 monclient: monmap: 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:04.284 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.271+0000 7f8b25d62700 1 librados: starting msgr at 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.271+0000 7f8b25d62700 1 librados: starting objecter 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.271+0000 7f8b25d62700 1 Processor -- start 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 -- start start 2024-04-20T02:29:04.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 librados: setting wanted keys 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 librados: calling monclient init 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient: init 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient: _add_conns ranks=[7,0,3,1,6,4,5,2,8] 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a0eb0 0x7f8b2013e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): picked mon.noname-i con 0x7f8b200a0eb0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b200a5c90 0x7f8b2013e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): picked mon.noname-a con 0x7f8b200a5c90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b200a6780 0x7f8b2013eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): picked mon.noname-g con 0x7f8b200a6780 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20141690 con 0x7f8b200a5c90 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20140280 con 0x7f8b200a6780 2024-04-20T02:29:04.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2009d920 con 0x7f8b200a0eb0 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b25d62700 10 monclient(hunting): authenticate will time out at 2423.673755s 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b200a5c90 0x7f8b2013e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b200a5c90 0x7f8b2013e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45844/0 (socket says 172.21.15.47:45844) 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 1 -- 172.21.15.47:0/1756699353 learned_addr learned my addr 172.21.15.47:0/1756699353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 10 monclient(hunting): get_auth_request con 0x7f8b200a5c90 auth_method 0 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1ffff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b200a6780 0x7f8b2013eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a0eb0 0x7f8b2013e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.272+0000 7f8b1effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): get_auth_request con 0x7f8b200a0eb0 auth_method 0 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): get_auth_request con 0x7f8b200a6780 auth_method 0 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 10 monclient(hunting): handle_auth_done global_id 36468 payload 824 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a0eb0 msgr2=0x7f8b2013e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8b200a0eb0 0x7f8b2013e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b200a6780 msgr2=0x7f8b2013eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b200a6780 0x7f8b2013eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 monclient: found mon.noname-a 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b18009550 con 0x7f8b200a5c90 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1effd700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b200a5c90 0x7f8b2013e980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000cc80 tx=0x7f8b1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 client.?.objecter ms_handle_connect 0x7f8b200a5c90 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b10003e70 con 0x7f8b200a5c90 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: dump: 2024-04-20T02:29:04.289 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10001d00 con 0x7f8b200a5c90 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 5 monclient: authenticate success, global_id 36468 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 10 monclient: _renew_subs 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b1000ad30 con 0x7f8b200a5c90 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2009deb0 con 0x7f8b200a5c90 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b17fff700 10 monclient: dump: 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.291 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b1dffb700 4 set_mon_vals no callback set 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 10 client.36468.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:04.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 10 monclient: _renew_subs 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b201413b0 con 0x7f8b200a5c90 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.273+0000 7f8b25d62700 1 librados: init done 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b25d62700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b25d62700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 0x7f8b201d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20068180 con 0x7f8b2006aac0 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 0x7f8b201d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient: get_auth_request con 0x7f8b2006aac0 auth_method 0 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_done global_id 84173 payload 824 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8b1000a8c0 con 0x7f8b200a5c90 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.274+0000 7f8b1f7fe700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b200a7790 con 0x7f8b2006aac0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b0003a2f0 0x7f8b0003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(517..517 src has 266..517) v4 ==== 6176+0+0 (secure 0 0 0) 0x7f8b1004a670 con 0x7f8b200a5c90 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 client.36468.objecter ms_dispatch 0x7f8b20001550 osd_map(517..517 src has 266..517) v4 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 3 client.36468.objecter handle_osd_map got epochs [517,517] > 0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 3 client.36468.objecter handle_osd_map decoding full epoch 517 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 0x7f8b201d08b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b180098f0 tx=0x7f8b18004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 20 client.36468.objecter dump_active .. 0 homeless 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 client.36468.objecter ms_handle_connect 0x7f8b2006aac0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 client.36468.objecter resend_mon_ops 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b1ffff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b0003a2f0 0x7f8b0003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b1ffff700 10 monclient: get_auth_request con 0x7f8b0003a2f0 auth_method 0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b18020070 con 0x7f8b2006aac0 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 monclient: dump: 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.295 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8b18004250 con 0x7f8b2006aac0 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 msgr2=0x7f8b201d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 0x7f8b201d08b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b180098f0 tx=0x7f8b18004750 comp rx=0 tx=0).stop 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 msgr2=0x7f8b201d08b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b2006aac0 0x7f8b201d08b0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b180098f0 tx=0x7f8b18004750 comp rx=0 tx=0).stop 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b17fff700 10 client.36468.objecter ms_handle_connect 0x7f8b0003a2f0 2024-04-20T02:29:04.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.275+0000 7f8b1ffff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b0003a2f0 0x7f8b0003c7b0 secure :-1 s=READY pgs=3306 cs=0 l=1 rev1=1 crypto rx=0x7f8b2013f4e0 tx=0x7f8b08008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b25d62700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b25d62700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 0x7f8b201d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b25d62700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2006a400 con 0x7f8b20142070 2024-04-20T02:29:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 0x7f8b201d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient: get_auth_request con 0x7f8b20142070 auth_method 0 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.283+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b1f7fe700 10 monclient(hunting): handle_auth_done global_id 84182 payload 824 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b1f7fe700 1 -- 172.21.15.47:0/1756699353 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b201439a0 con 0x7f8b20142070 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b17fff700 10 client.36468.objecter ms_handle_connect 0x7f8b20142070 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b17fff700 10 client.36468.objecter resend_mon_ops 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b1f7fe700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 0x7f8b201d0df0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009130 tx=0x7f8b18011000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b1802c300 con 0x7f8b20142070 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b17fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b17fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:04.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.284+0000 7f8b17fff700 10 monclient: dump: 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.300 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f8b1802c8e0 con 0x7f8b20142070 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 msgr2=0x7f8b201d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 0x7f8b201d0df0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009130 tx=0x7f8b18011000 comp rx=0 tx=0).stop 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 msgr2=0x7f8b201d0df0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b17fff700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8b20142070 0x7f8b201d0df0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009130 tx=0x7f8b18011000 comp rx=0 tx=0).stop 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:04.301 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 9, 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:04.302 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.303 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:04.304 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.305 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:04.307 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.310 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.312 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.318 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.319 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.320 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:04.321 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:04.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 10 librados: watch_flush enter 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 10 librados: watch_flush exit 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 20 client.36468.objecter shutdown clearing up homeless session... 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 10 client.36468.objecter successfully canceled tick 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b0003a2f0 msgr2=0x7f8b0003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8b0003a2f0 0x7f8b0003c7b0 secure :-1 s=READY pgs=3306 cs=0 l=1 rev1=1 crypto rx=0x7f8b2013f4e0 tx=0x7f8b08008040 comp rx=0 tx=0).stop 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 10 monclient: shutdown 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b200a5c90 msgr2=0x7f8b2013e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 --2- 172.21.15.47:0/1756699353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b200a5c90 0x7f8b2013e980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000cc80 tx=0x7f8b1000cf90 comp rx=0 tx=0).stop 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b1ffff700 1 -- 172.21.15.47:0/1756699353 reap_dead start 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b1ffff700 1 -- 172.21.15.47:0/1756699353 reap_dead start 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 shutdown_connections 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 >> 172.21.15.47:0/1756699353 conn(0x7f8b200aa8b0 msgr2=0x7f8b200a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 shutdown_connections 2024-04-20T02:29:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 -- 172.21.15.47:0/1756699353 wait complete. 2024-04-20T02:29:04.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.285+0000 7f8b25d62700 1 librados: shutdown 2024-04-20T02:29:04.325 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:04.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:29:04.325 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.463+0000 7fc1fd87e700 10 monclient: get_monmap_and_config 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.463+0000 7fc1fd87e700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.463+0000 7fc1fd87e700 10 monclient: monmap: 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:04.464844+0000 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:04.464844+0000 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.464 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:04.465 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:04.466 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 1 Processor -- start 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 1 -- start start 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 10 monclient: init 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 10 monclient: _add_conns ranks=[0,9,7,5,4,3,10,8,1,2,6] 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc1f8137a20 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1f80a5120 0x7fc1f813cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.464+0000 7fc1fd87e700 10 monclient(hunting): picked mon.noname-a con 0x7fc1f80a5120 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f8138510 0x7fc1f813f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 10 monclient(hunting): picked mon.noname-i con 0x7fc1f8138510 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f80751e0 con 0x7fc1f8137a20 2024-04-20T02:29:04.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f80a2cc0 con 0x7fc1f80a5120 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80a2fa0 con 0x7fc1f8138510 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1fd87e700 10 monclient(hunting): authenticate will time out at 2423.866078s 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1f80a5120 0x7fc1f813cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f6ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc1f8137a20 0x7fc1f8137e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43718/0 (socket says 172.21.15.47:43718) 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f6ffd700 1 -- 172.21.15.47:0/2058179121 learned_addr learned my addr 172.21.15.47:0/2058179121 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): get_auth_request con 0x7fc1f80a5120 auth_method 0 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 1 --2- 172.21.15.47:0/2058179121 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f8138510 0x7fc1f813f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): get_auth_request con 0x7fc1f8138510 auth_method 0 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f5ffb700 1 -- 172.21.15.47:0/2058179121 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362409372 0 0) 0x7fc1f80751e0 con 0x7fc1f8137a20 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f5ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f5ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f5ffb700 10 monclient(hunting): my global_id is 36486 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f5ffb700 1 -- 172.21.15.47:0/2058179121 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e40035b0 con 0x7fc1f8137a20 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_done global_id 36477 payload 274 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 -- 172.21.15.47:0/2058179121 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f8138510 msgr2=0x7fc1f813f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/2058179121 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f8138510 0x7fc1f813f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 -- 172.21.15.47:0/2058179121 >> v1:172.21.15.47:6789/0 conn(0x7fc1f8137a20 legacy=0x7fc1f8137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 monclient: found mon.noname-a 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 -- 172.21.15.47:0/2058179121 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80a3d80 con 0x7fc1f80a5120 2024-04-20T02:29:04.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.465+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/2058179121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f80a5120 0x7fc1f813cf50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009840 tx=0x7fc1e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39b5ee10c79e92c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 1 -- 172.21.15.47:0/2058179121 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0011aa0 con 0x7fc1f80a5120 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: dump: 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.479 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 1 -- 172.21.15.47:0/2058179121 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e0011c00 con 0x7fc1f80a5120 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 5 monclient: authenticate success, global_id 36477 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 1 -- 172.21.15.47:0/2058179121 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0011d70 con 0x7fc1f80a5120 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 4 set_mon_vals no callback set 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1f5ffb700 10 monclient: dump: 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.481 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 10 monclient: shutdown 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 -- 172.21.15.47:0/2058179121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f80a5120 msgr2=0x7fc1f813cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/2058179121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f80a5120 0x7fc1f813cf50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009840 tx=0x7fc1e0009b50 comp rx=0 tx=0).stop 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 -- 172.21.15.47:0/2058179121 shutdown_connections 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/2058179121 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f8138510 0x7fc1f813f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/2058179121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1f80a5120 0x7fc1f813cf50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 -- 172.21.15.47:0/2058179121 >> 172.21.15.47:0/2058179121 conn(0x7fc1f80aa5a0 msgr2=0x7fc1f80aa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 -- 172.21.15.47:0/2058179121 shutdown_connections 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.466+0000 7fc1fd87e700 1 -- 172.21.15.47:0/2058179121 wait complete. 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.467+0000 7fc1fd87e700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.467+0000 7fc1fd87e700 10 monclient: monmap: 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.483 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.467+0000 7fc1fd87e700 1 librados: starting msgr at 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.467+0000 7fc1fd87e700 1 librados: starting objecter 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.467+0000 7fc1fd87e700 1 Processor -- start 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 -- start start 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 librados: setting wanted keys 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 librados: calling monclient init 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient: init 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient: _add_conns ranks=[5,6,1,3,7,8,4,2,0] 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f80a5120 0x7fc1f8120f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): picked mon.noname-h con 0x7fc1f80a5120 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1f8137a20 0x7fc1f81214c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): picked mon.noname-d con 0x7fc1f8137a20 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f8138510 0x7fc1f8121a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): picked mon.noname-f con 0x7fc1f8138510 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80a10a0 con 0x7fc1f8138510 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80a2180 con 0x7fc1f80a5120 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81228b0 con 0x7fc1f8137a20 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1fd87e700 10 monclient(hunting): authenticate will time out at 2423.869492s 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1f8137a20 0x7fc1f81214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1f8137a20 0x7fc1f81214c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59200/0 (socket says 172.21.15.47:59200) 2024-04-20T02:29:04.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 1 -- 172.21.15.47:0/3273227750 learned_addr learned my addr 172.21.15.47:0/3273227750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): get_auth_request con 0x7fc1f8137a20 auth_method 0 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f80a5120 0x7fc1f8120f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): get_auth_request con 0x7fc1f80a5120 auth_method 0 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f8138510 0x7fc1f8121a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): get_auth_request con 0x7fc1f8138510 auth_method 0 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient(hunting): handle_auth_done global_id 84191 payload 824 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1f8137a20 msgr2=0x7fc1f81214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1f8137a20 0x7fc1f81214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f8138510 msgr2=0x7fc1f8121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1f8138510 0x7fc1f8121a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 monclient: found mon.noname-h 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f6ffd700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00094a0 con 0x7fc1f80a5120 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.468+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc1f8137a20 0x7fc1f81214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1f6ffd700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f80a5120 0x7fc1f8120f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c420 tx=0x7fc1e800c730 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 client.?.objecter ms_handle_connect 0x7fc1f80a5120 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e800ce30 con 0x7fc1f80a5120 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: dump: 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.488 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e80070d0 con 0x7fc1f80a5120 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 5 monclient: authenticate success, global_id 84191 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 10 monclient: _renew_subs 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f8124370 con 0x7fc1f80a5120 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e80076b0 con 0x7fc1f80a5120 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1effff700 10 monclient: dump: 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1f57fa700 4 set_mon_vals no callback set 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 10 client.84191.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 10 monclient: _renew_subs 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f81231d0 con 0x7fc1f80a5120 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.469+0000 7fc1fd87e700 1 librados: init done 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc1e8007240 con 0x7fc1f80a5120 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1fd87e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 0x7fc1f809ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8068180 con 0x7fc1f809aa30 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d803a2f0 0x7fc1d803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 0x7fc1f809ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(517..517 src has 266..517) v4 ==== 6176+0+0 (secure 0 0 0) 0x7fc1e8049670 con 0x7fc1f80a5120 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 10 client.84191.objecter ms_dispatch 0x7fc1f8001550 osd_map(517..517 src has 266..517) v4 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 3 client.84191.objecter handle_osd_map got epochs [517,517] > 0 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 3 client.84191.objecter handle_osd_map decoding full epoch 517 2024-04-20T02:29:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient: get_auth_request con 0x7fc1f809aa30 auth_method 0 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1effff700 20 client.84191.objecter dump_active .. 0 homeless 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f77fe700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d803a2f0 0x7fc1d803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f77fe700 10 monclient: get_auth_request con 0x7fc1d803a2f0 auth_method 0 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_done global_id 84247 payload 824 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.470+0000 7fc1f67fc700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1f806d8a0 con 0x7fc1f809aa30 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 client.84191.objecter ms_handle_connect 0x7fc1f809aa30 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 client.84191.objecter resend_mon_ops 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 0x7fc1f809ae10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8120c80 tx=0x7fc1e0004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 client.84191.objecter ms_handle_connect 0x7fc1d803a2f0 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1f77fe700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d803a2f0 0x7fc1d803c7b0 secure :-1 s=READY pgs=3307 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0011340 tx=0x7fc1f000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e0020070 con 0x7fc1f809aa30 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 monclient: dump: 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc1e00038b0 con 0x7fc1f809aa30 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.471+0000 7fc1effff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.472+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 msgr2=0x7fc1f809ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.472+0000 7fc1effff700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 0x7fc1f809ae10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8120c80 tx=0x7fc1e0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.472+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 msgr2=0x7fc1f809ae10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.472+0000 7fc1effff700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809aa30 0x7fc1f809ae10 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8120c80 tx=0x7fc1e0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:04.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.479+0000 7fc1fd87e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:04.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.479+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 0x7fc1f81d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.479+0000 7fc1fd87e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.479+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8124500 con 0x7fc1f809b350 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 0x7fc1f81d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient: get_auth_request con 0x7fc1f809b350 auth_method 0 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 10 monclient(hunting): handle_auth_done global_id 84256 payload 824 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 1 -- 172.21.15.47:0/3273227750 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc1f8067620 con 0x7fc1f809b350 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1f67fc700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 0x7fc1f81d6d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0024000 tx=0x7fc1e0011000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1effff700 10 client.84191.objecter ms_handle_connect 0x7fc1f809b350 2024-04-20T02:29:04.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1effff700 10 client.84191.objecter resend_mon_ops 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1e001bab0 con 0x7fc1f809b350 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1effff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.480+0000 7fc1effff700 10 monclient: dump: 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.499 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fc1e001bc10 con 0x7fc1f809b350 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 msgr2=0x7fc1f81d6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 0x7fc1f81d6d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0024000 tx=0x7fc1e0011000 comp rx=0 tx=0).stop 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 msgr2=0x7fc1f81d6d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1effff700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc1f809b350 0x7fc1f81d6d50 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0024000 tx=0x7fc1e0011000 comp rx=0 tx=0).stop 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 88, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 9, 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:04.523 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 10 librados: watch_flush enter 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 10 librados: watch_flush exit 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 20 client.84191.objecter shutdown clearing up homeless session... 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 10 client.84191.objecter successfully canceled tick 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d803a2f0 msgr2=0x7fc1d803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc1d803a2f0 0x7fc1d803c7b0 secure :-1 s=READY pgs=3307 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0011340 tx=0x7fc1f000b040 comp rx=0 tx=0).stop 2024-04-20T02:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 10 monclient: shutdown 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f80a5120 msgr2=0x7fc1f8120f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 --2- 172.21.15.47:0/3273227750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc1f80a5120 0x7fc1f8120f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c420 tx=0x7fc1e800c730 comp rx=0 tx=0).stop 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1f77fe700 1 -- 172.21.15.47:0/3273227750 reap_dead start 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1f77fe700 1 -- 172.21.15.47:0/3273227750 reap_dead start 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 shutdown_connections 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 >> 172.21.15.47:0/3273227750 conn(0x7fc1f80aa5a0 msgr2=0x7fc1f81bf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 shutdown_connections 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 -- 172.21.15.47:0/3273227750 wait complete. 2024-04-20T02:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.481+0000 7fc1fd87e700 1 librados: shutdown 2024-04-20T02:29:04.527 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:29:04.527 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g'] 2024-04-20T02:29:04.527 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['e', 'h'] 2024-04-20T02:29:04.527 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-04-20T02:29:04.527 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g store 2024-04-20T02:29:04.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g sync_force --yes-i-really-mean-it 2024-04-20T02:29:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.667+0000 7fe68fdba700 10 monclient: get_monmap_and_config 2024-04-20T02:29:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.667+0000 7fe68fdba700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.667+0000 7fe68fdba700 10 monclient: monmap: 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:04.668221+0000 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:04.668221+0000 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.668 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:04.669 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:04.669 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.667+0000 7fe68fdba700 1 Processor -- start 2024-04-20T02:29:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.667+0000 7fe68fdba700 1 -- start start 2024-04-20T02:29:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.667+0000 7fe68fdba700 10 monclient: init 2024-04-20T02:29:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient: _add_conns ranks=[7,6,2,4,10,8,0,9,3,1,5] 2024-04-20T02:29:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6880a2870 0x7fe68809b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): picked mon.noname-i con 0x7fe6880a2870 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 0x7fe6880a3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): picked mon.noname-d con 0x7fe6880a2c50 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6881356a0 0x7fe688135b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): picked mon.noname-b con 0x7fe6881356a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880a0f50 con 0x7fe6881356a0 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe688072740 con 0x7fe6880a2c50 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880a0a60 con 0x7fe6880a2870 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68fdba700 10 monclient(hunting): authenticate will time out at 2424.069577s 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 0x7fe6880a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6881356a0 0x7fe688135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 0x7fe6880a3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59208/0 (socket says 172.21.15.47:59208) 2024-04-20T02:29:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 1 -- 172.21.15.47:0/203414241 learned_addr learned my addr 172.21.15.47:0/203414241 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): get_auth_request con 0x7fe6880a2c50 auth_method 0 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): get_auth_request con 0x7fe6881356a0 auth_method 0 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68e357700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68db56700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6880a2870 0x7fe68809b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68db56700 10 monclient(hunting): get_auth_request con 0x7fe6880a2870 auth_method 0 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68db56700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68db56700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.668+0000 7fe68db56700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68db56700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68db56700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68db56700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 10 monclient(hunting): handle_auth_done global_id 24617 payload 274 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 -- 172.21.15.47:0/203414241 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6880a2870 msgr2=0x7fe68809b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6880a2870 0x7fe68809b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 -- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 msgr2=0x7fe6880a3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 0x7fe6880a3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 monclient: found mon.noname-b 2024-04-20T02:29:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 -- 172.21.15.47:0/203414241 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880751e0 con 0x7fe6881356a0 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68d355700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 0x7fe6880a3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68e357700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6881356a0 0x7fe688135b20 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fe680009dc0 tx=0x7fe68000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e71b3135b831bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 1 -- 172.21.15.47:0/203414241 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe68000e070 con 0x7fe6881356a0 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: dump: 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.683 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 1 -- 172.21.15.47:0/203414241 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680012430 con 0x7fe6881356a0 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 5 monclient: authenticate success, global_id 24617 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 1 -- 172.21.15.47:0/203414241 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe680012a10 con 0x7fe6881356a0 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 4 set_mon_vals no callback set 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68cb54700 10 monclient: dump: 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.685 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 10 monclient: shutdown 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.669+0000 7fe68fdba700 1 -- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6881356a0 msgr2=0x7fe688135b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6881356a0 0x7fe688135b20 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fe680009dc0 tx=0x7fe68000b330 comp rx=0 tx=0).stop 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 -- 172.21.15.47:0/203414241 shutdown_connections 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe6880a2870 0x7fe68809b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6881356a0 0x7fe688135b20 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 --2- 172.21.15.47:0/203414241 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2c50 0x7fe6880a3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 -- 172.21.15.47:0/203414241 >> 172.21.15.47:0/203414241 conn(0x7fe6880a3f90 msgr2=0x7fe68806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 -- 172.21.15.47:0/203414241 shutdown_connections 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 1 -- 172.21.15.47:0/203414241 wait complete. 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 10 monclient: build_initial_monmap 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.670+0000 7fe68fdba700 10 monclient: monmap: 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:04.687 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 librados: starting msgr at 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 librados: starting objecter 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 Processor -- start 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 -- start start 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 librados: setting wanted keys 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 librados: calling monclient init 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient: init 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient: _add_conns ranks=[6,1,3,2,5,7,0,4,8] 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 0x7fe6881cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): picked mon.noname-d con 0x7fe6880a2870 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:04.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880a2c50 0x7fe6881cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): picked mon.noname-f con 0x7fe6880a2c50 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881356a0 0x7fe68809e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): picked mon.noname-g con 0x7fe6881356a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe688070140 con 0x7fe6880a2c50 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe688066f10 con 0x7fe6881356a0 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe688072410 con 0x7fe6880a2870 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68fdba700 10 monclient(hunting): authenticate will time out at 2424.072871s 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.671+0000 7fe68db56700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 0x7fe6881cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 0x7fe6881cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:59220/0 (socket says 172.21.15.47:59220) 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 -- 172.21.15.47:0/2214613523 learned_addr learned my addr 172.21.15.47:0/2214613523 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:04.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): get_auth_request con 0x7fe6880a2870 auth_method 0 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880a2c50 0x7fe6881cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881356a0 0x7fe68809e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): get_auth_request con 0x7fe6880a2c50 auth_method 0 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): get_auth_request con 0x7fe6881356a0 auth_method 0 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68e357700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient(hunting): handle_auth_done global_id 55032 payload 824 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881356a0 msgr2=0x7fe68809e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881356a0 0x7fe68809e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880a2c50 msgr2=0x7fe6881cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6880a2c50 0x7fe6881cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient: _finish_hunting 0 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 monclient: found mon.noname-d 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:04.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe678009550 con 0x7fe6880a2870 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68db56700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 0x7fe6881cbd90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe68400e9b0 tx=0x7fe68400ecc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 client.?.objecter ms_handle_connect 0x7fe6880a2870 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe684009710 con 0x7fe6880a2870 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: dump: 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.692 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 5 monclient: authenticate success, global_id 55032 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400ee50 con 0x7fe6880a2870 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 10 monclient: _renew_subs 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:29:04.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe680009a70 con 0x7fe6880a2870 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe68400ccc0 con 0x7fe6880a2870 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67ffff700 4 set_mon_vals no callback set 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 10 client.55032.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe67e7fc700 10 monclient: dump: 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.694 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 10 monclient: _renew_subs 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6880712e0 con 0x7fe6880a2870 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.672+0000 7fe68fdba700 1 librados: init done 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68fdba700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68fdba700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 0x7fe68809e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe688068280 con 0x7fe688071cc0 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe68400c850 con 0x7fe6880a2870 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68d355700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 0x7fe68809e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68d355700 10 monclient: get_auth_request con 0x7fe688071cc0 auth_method 0 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68d355700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68d355700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68d355700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.673+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66c03a2f0 0x7fe66c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(517..517 src has 266..517) v4 ==== 6176+0+0 (secure 0 0 0) 0x7fe68404a8b0 con 0x7fe6880a2870 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 client.55032.objecter ms_dispatch 0x7fe688001550 osd_map(517..517 src has 266..517) v4 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 3 client.55032.objecter handle_osd_map got epochs [517,517] > 0 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 3 client.55032.objecter handle_osd_map decoding full epoch 517 2024-04-20T02:29:04.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68d355700 10 monclient(hunting): handle_auth_done global_id 84261 payload 824 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68d355700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe68809bcb0 con 0x7fe688071cc0 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68e357700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66c03a2f0 0x7fe66c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 20 client.55032.objecter dump_active .. 0 homeless 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68e357700 10 monclient: get_auth_request con 0x7fe66c03a2f0 auth_method 0 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 client.55032.objecter ms_handle_connect 0x7fe688071cc0 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 client.55032.objecter resend_mon_ops 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68d355700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 0x7fe68809e8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe68809b400 tx=0x7fe678011f60 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe678014440 con 0x7fe688071cc0 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 monclient: dump: 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.697 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 client.55032.objecter ms_handle_connect 0x7fe66c03a2f0 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe68e357700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66c03a2f0 0x7fe66c03c7b0 secure :-1 s=READY pgs=3308 cs=0 l=1 rev1=1 crypto rx=0x7fe680009dc0 tx=0x7fe680011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe67800fab0 con 0x7fe688071cc0 2024-04-20T02:29:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 msgr2=0x7fe68809e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 0x7fe68809e8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe68809b400 tx=0x7fe678011f60 comp rx=0 tx=0).stop 2024-04-20T02:29:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 msgr2=0x7fe68809e8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.674+0000 7fe67e7fc700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe688071cc0 0x7fe68809e8a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68fdba700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68fdba700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 0x7fe68809ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68fdba700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68806a400 con 0x7fe6881cc970 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 0x7fe68809ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient: get_auth_request con 0x7fe6881cc970 auth_method 0 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:04.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 10 monclient(hunting): handle_auth_done global_id 84270 payload 824 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 1 -- 172.21.15.47:0/2214613523 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7fe68806d170 con 0x7fe6881cc970 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe68d355700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 0x7fe68809ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe67801c000 tx=0x7fe678004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe67e7fc700 10 client.55032.objecter ms_handle_connect 0x7fe6881cc970 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.684+0000 7fe67e7fc700 10 client.55032.objecter resend_mon_ops 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe6780229f0 con 0x7fe6881cc970 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 10 monclient: dump: 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:04.702 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7fe678022580 con 0x7fe6881cc970 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 msgr2=0x7fe68809ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 0x7fe68809ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe67801c000 tx=0x7fe678004750 comp rx=0 tx=0).stop 2024-04-20T02:29:04.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 msgr2=0x7fe68809ede0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:04.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe67e7fc700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe6881cc970 0x7fe68809ede0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.704 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:29:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:29:04.704 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 10 librados: watch_flush enter 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 10 librados: watch_flush exit 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 20 client.55032.objecter shutdown clearing up homeless session... 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 10 client.55032.objecter successfully canceled tick 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66c03a2f0 msgr2=0x7fe66c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe66c03a2f0 0x7fe66c03c7b0 secure :-1 s=READY pgs=3308 cs=0 l=1 rev1=1 crypto rx=0x7fe680009dc0 tx=0x7fe680011000 comp rx=0 tx=0).stop 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 10 monclient: shutdown 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 msgr2=0x7fe6881cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 0x7fe6881cbd90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe68400e9b0 tx=0x7fe68400ecc0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68e357700 1 -- 172.21.15.47:0/2214613523 reap_dead start 2024-04-20T02:29:04.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 shutdown_connections 2024-04-20T02:29:04.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 --2- 172.21.15.47:0/2214613523 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6880a2870 0x7fe6881cbd90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:04.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.685+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 >> 172.21.15.47:0/2214613523 conn(0x7fe6880a3f90 msgr2=0x7fe68809d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:04.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.686+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 shutdown_connections 2024-04-20T02:29:04.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.686+0000 7fe68fdba700 1 -- 172.21.15.47:0/2214613523 wait complete. 2024-04-20T02:29:04.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:04.686+0000 7fe68fdba700 1 librados: shutdown 2024-04-20T02:29:04.709 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-04-20T02:29:04.709 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-04-20T02:29:04.709 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:29:04.720 INFO:tasks.ceph.mon.g:Stopped 2024-04-20T02:29:04.721 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-04-20T02:29:04.721 INFO:tasks.ceph.mon.e:Sent signal 19 2024-04-20T02:29:04.721 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-04-20T02:29:04.721 INFO:tasks.ceph.mon.h:Sent signal 19 2024-04-20T02:29:04.721 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:29:19.722 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-04-20T02:29:19.723 INFO:tasks.ceph.mon.e:Sent signal 18 2024-04-20T02:29:19.723 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-04-20T02:29:19.723 INFO:tasks.ceph.mon.h:Sent signal 18 2024-04-20T02:29:19.723 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-04-20T02:29:19.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.867+0000 7ffb92e55700 10 monclient: get_monmap_and_config 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.867+0000 7ffb92e55700 10 monclient: build_initial_monmap 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.867+0000 7ffb92e55700 10 monclient: monmap: 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:19.869065+0000 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:19.869065+0000 2024-04-20T02:29:19.868 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:19.869 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:19.870 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.868+0000 7ffb92e55700 1 Processor -- start 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.868+0000 7ffb92e55700 1 -- start start 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.868+0000 7ffb92e55700 10 monclient: init 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient: _add_conns ranks=[4,0,5,7,6,3,8,1,2,10,9] 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): picked mon.noname-c con 0x7ffb8c0a37d0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:19.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb8c09fed0 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): picked mon.noname-h con 0x7ffb8c0a0330 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0751e0 con 0x7ffb8c09fed0 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c09c6b0 con 0x7ffb8c0a37d0 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c09d070 con 0x7ffb8c0a0330 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb92e55700 10 monclient(hunting): authenticate will time out at 2439.270437s 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb8bfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffb8c09fed0 0x7ffb8c0a1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:50066/0 (socket says 172.21.15.47:50066) 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 learned_addr learned my addr 172.21.15.47:0/3450682646 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a37d0 auth_method 0 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:19.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb913f2700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a0330 auth_method 0 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb913f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb913f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.869+0000 7ffb913f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:19.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.870+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:29:19.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.870+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:19.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.870+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:19.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:19.870+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:20.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a37d0 auth_method 0 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:20.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.070+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a0330 auth_method 0 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:20.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.071+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:29:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a37d0 auth_method 0 2024-04-20T02:29:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:20.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.471+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a0330 auth_method 0 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:20.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:20.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:20.472+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a37d0 auth_method 0 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.272+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 10 monclient(hunting): get_auth_request con 0x7ffb8c0a0330 auth_method 0 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:21.273+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): tick 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 monclient(hunting): continuing hunt 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 msgr2=0x7ffb8c0a22d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb8c0a0330 0x7ffb8c0a22d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 msgr2=0x7ffb8c131190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb8c0a37d0 0x7ffb8c131190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:22.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/3450682646 >> v1:172.21.15.47:6789/0 conn(0x7ffb8c09fed0 legacy=0x7ffb8c0a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient: _add_conns ranks=[8,10,1,7,9,6,3,2] 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): picked mon.noname-e con 0x7ffb6c003900 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): picked mon.noname-f con 0x7ffb6c006310 addr v2:172.21.15.179:3300/0 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ffb6c008e60 addr v1:172.21.15.179:6789/0 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/3450682646 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0a3bb0 con 0x7ffb6c008e60 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/3450682646 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c09c6b0 con 0x7ffb6c006310 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 1 -- 172.21.15.47:0/3450682646 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c09d070 con 0x7ffb6c003900 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8a7fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb6c003900 auth_method 0 2024-04-20T02:29:22.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request con 0x7ffb6c006310 auth_method 0 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:22.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:22.870+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:23.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb6c003900 auth_method 0 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request con 0x7ffb6c006310 auth_method 0 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:29:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.071+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb6c003900 auth_method 0 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:29:23.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request con 0x7ffb6c006310 auth_method 0 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:23.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:23.472+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:29:24.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb6c003900 auth_method 0 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request con 0x7ffb6c006310 auth_method 0 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:29:24.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:29:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:29:24.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.273+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:29:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.744+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702460022 0 0) 0x7ffb8c0a3bb0 con 0x7ffb6c008e60 2024-04-20T02:29:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.745+0000 7ffb8b7fe700 10 monclient(hunting): _negotiate 2024-04-20T02:29:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.745+0000 7ffb8b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.745+0000 7ffb8b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.745+0000 7ffb8b7fe700 10 monclient(hunting): my global_id is 55396 2024-04-20T02:29:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.745+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb780035b0 con 0x7ffb6c008e60 2024-04-20T02:29:24.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4162165170 0 0) 0x7ffb780035b0 con 0x7ffb6c008e60 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 msgr2=0x7ffb6c005d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb6c003900 0x7ffb6c005d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 msgr2=0x7ffb6c0087a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/3450682646 >> v2:172.21.15.179:3300/0 conn(0x7ffb6c006310 0x7ffb6c0087a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 reap_dead start 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c00dcb0 con 0x7ffb6c008e60 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:24.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7ffb84009c70 con 0x7ffb6c008e60 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: dump: 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:24.763 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 >> v1:172.21.15.179:6789/0 conn(0x7ffb6c008e60 legacy=0x7ffb6c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient: _add_conns ranks=[4,7,1,3,0,8,6,2,5] 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb78004250 0x7ffb78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): picked mon.c con 0x7ffb78004250 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ffb78004e60 0x7ffb780052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): picked mon.i con 0x7ffb78004e60 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78009c40 0x7ffb7800c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): picked mon.f con 0x7ffb78009c40 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c09c6b0 con 0x7ffb78009c40 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c09d070 con 0x7ffb78004250 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800c9e0 con 0x7ffb78004e60 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.758+0000 7ffb8b7fe700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78009c40 0x7ffb7800c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb78004250 0x7ffb78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): get_auth_request con 0x7ffb78009c40 auth_method 0 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ffb78004e60 0x7ffb780052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:24.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request con 0x7ffb78004250 auth_method 0 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request con 0x7ffb78004e60 auth_method 0 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:29:24.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8bfff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb90bf1700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:29:24.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient(hunting): handle_auth_done global_id 55396 payload 293 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ffb78004e60 msgr2=0x7ffb780052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ffb78004e60 0x7ffb780052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb78004250 msgr2=0x7ffb78004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb78004250 0x7ffb78004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient: _finish_hunting 0 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 monclient: found mon.f 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 -- 172.21.15.47:0/3450682646 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780035b0 con 0x7ffb78009c40 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient: _finish_auth 0 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 10 monclient: _check_auth_tickets 2024-04-20T02:29:24.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb92e55700 5 monclient: authenticate success, global_id 55396 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb92e55700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb913f2700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78009c40 0x7ffb7800c0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c06e030 tx=0x7ffb78005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b977b796df902215 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb84009ab0 con 0x7ffb78009c40 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 10 monclient: dump: 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:24.769 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb92e55700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8400f040 con 0x7ffb78009c40 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb92e55700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb92e55700 4 set_mon_vals no callback set 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 1 -- 172.21.15.47:0/3450682646 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb84016d40 con 0x7ffb78009c40 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.759+0000 7ffb8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:24.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb8b7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb8b7fe700 10 monclient: dump: 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:24.771 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 10 monclient: shutdown 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 -- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78009c40 msgr2=0x7ffb7800c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78009c40 0x7ffb7800c0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c06e030 tx=0x7ffb78005880 comp rx=0 tx=0).stop 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 -- 172.21.15.47:0/3450682646 shutdown_connections 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78009c40 0x7ffb7800c0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ffb78004e60 0x7ffb780052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 --2- 172.21.15.47:0/3450682646 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffb78004250 0x7ffb78004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 -- 172.21.15.47:0/3450682646 >> 172.21.15.47:0/3450682646 conn(0x7ffb8c0a2cb0 msgr2=0x7ffb8c0a30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 -- 172.21.15.47:0/3450682646 shutdown_connections 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.760+0000 7ffb92e55700 1 -- 172.21.15.47:0/3450682646 wait complete. 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.761+0000 7ffb92e55700 10 monclient: build_initial_monmap 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.761+0000 7ffb92e55700 10 monclient: monmap: 2024-04-20T02:29:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:24.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 librados: starting msgr at 2024-04-20T02:29:24.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 librados: starting objecter 2024-04-20T02:29:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 Processor -- start 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 -- start start 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 librados: setting wanted keys 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 librados: calling monclient init 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 10 monclient: init 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 10 monclient: _add_conns ranks=[3,1,8,5,0,4,7,6,2] 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6c008e60 0x7ffb8c1b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 10 monclient(hunting): picked mon.noname-g con 0x7ffb6c008e60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78004250 0x7ffb8c072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.762+0000 7ffb92e55700 10 monclient(hunting): picked mon.noname-f con 0x7ffb78004250 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 0x7ffb8c072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 10 monclient(hunting): picked mon.noname-e con 0x7ffb78004e60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:24.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1b9d00 con 0x7ffb78004250 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06f700 con 0x7ffb6c008e60 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c06f9e0 con 0x7ffb78004e60 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb92e55700 10 monclient(hunting): authenticate will time out at 2444.164035s 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb90bf1700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6c008e60 msgr2=0x7ffb8c1b91f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb90bf1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6c008e60 0x7ffb8c1b91f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 0x7ffb8c072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 0x7ffb8c072740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41826/0 (socket says 172.21.15.47:41826) 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 -- 172.21.15.47:0/619788113 learned_addr learned my addr 172.21.15.47:0/619788113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): get_auth_request con 0x7ffb78004e60 auth_method 0 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78004250 0x7ffb8c072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request con 0x7ffb78004250 auth_method 0 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient(hunting): handle_auth_done global_id 35765 payload 824 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 -- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6c008e60 msgr2=0x7ffb8c1b91f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:29:24.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6c008e60 0x7ffb8c1b91f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 -- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78004250 msgr2=0x7ffb8c072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78004250 0x7ffb8c072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient: _finish_hunting 0 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 monclient: found mon.noname-e 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 -- 172.21.15.47:0/619788113 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c06fcc0 con 0x7ffb78004e60 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb8bfff700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78004250 0x7ffb8c072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb913f2700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 0x7ffb8c072740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffb84000f80 tx=0x7ffb84004ad0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb897fa700 10 client.?.objecter ms_handle_connect 0x7ffb78004e60 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.763+0000 7ffb897fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb84009410 con 0x7ffb78004e60 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: dump: 2024-04-20T02:29:24.783 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: _finish_auth 0 2024-04-20T02:29:24.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: _check_auth_tickets 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84009c30 con 0x7ffb78004e60 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 5 monclient: authenticate success, global_id 35765 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 10 monclient: _renew_subs 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c12e5f0 con 0x7ffb78004e60 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb84009ab0 con 0x7ffb78004e60 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb8affd700 4 set_mon_vals no callback set 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb897fa700 10 monclient: dump: 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:24.785 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:24.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 10 client.35765.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 10 monclient: _renew_subs 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c1ca490 con 0x7ffb78004e60 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.764+0000 7ffb92e55700 1 librados: init done 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb92e55700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb92e55700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb92e55700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ffb8401f450 con 0x7ffb78004e60 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c067f90 con 0x7ffb78004e60 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb6c03b5a0 0x7ffb6c03da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(518..518 src has 266..518) v4 ==== 6549+0+0 (secure 0 0 0) 0x7ffb8404c4d0 con 0x7ffb78004e60 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 10 client.35765.objecter ms_dispatch 0x7ffb8c001550 osd_map(518..518 src has 266..518) v4 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 3 client.35765.objecter handle_osd_map got epochs [518,518] > 0 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 3 client.35765.objecter handle_osd_map decoding full epoch 518 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb897fa700 20 client.35765.objecter dump_active .. 0 homeless 2024-04-20T02:29:24.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb6c03b5a0 0x7ffb6c03da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:24.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.765+0000 7ffb90bf1700 10 monclient: get_auth_request con 0x7ffb6c03b5a0 auth_method 0 2024-04-20T02:29:24.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.766+0000 7ffb897fa700 10 client.35765.objecter ms_handle_connect 0x7ffb6c03b5a0 2024-04-20T02:29:24.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.766+0000 7ffb90bf1700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb6c03b5a0 0x7ffb6c03da60 secure :-1 s=READY pgs=3310 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c003770 tx=0x7ffb7c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:24.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.770+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ffb84027730 con 0x7ffb78004e60 2024-04-20T02:29:24.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.770+0000 7ffb897fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:24.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:24.770+0000 7ffb897fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.096+0000 7ffb92e55700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.096+0000 7ffb92e55700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.096+0000 7ffb92e55700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.096+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffb8c12ed10 con 0x7ffb78004e60 2024-04-20T02:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.097+0000 7ffb897fa700 1 -- 172.21.15.47:0/619788113 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3169 (secure 0 0 0) 0x7ffb8401ac80 con 0x7ffb78004e60 2024-04-20T02:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.097+0000 7ffb897fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.097+0000 7ffb897fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:25.098 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":94,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:29:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 10 librados: watch_flush enter 2024-04-20T02:29:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 10 librados: watch_flush exit 2024-04-20T02:29:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 20 client.35765.objecter shutdown clearing up homeless session... 2024-04-20T02:29:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 10 client.35765.objecter successfully canceled tick 2024-04-20T02:29:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb6c03b5a0 msgr2=0x7ffb6c03da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb6c03b5a0 0x7ffb6c03da60 secure :-1 s=READY pgs=3310 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c003770 tx=0x7ffb7c008040 comp rx=0 tx=0).stop 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 10 monclient: shutdown 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 msgr2=0x7ffb8c072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 0x7ffb8c072740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffb84000f80 tx=0x7ffb84004ad0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 shutdown_connections 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb78004e60 0x7ffb8c072740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb6c008e60 0x7ffb8c1b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb6c03b5a0 0x7ffb6c03da60 unknown :-1 s=CLOSED pgs=3310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 --2- 172.21.15.47:0/619788113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78004250 0x7ffb8c072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 >> 172.21.15.47:0/619788113 conn(0x7ffb8c0a2cb0 msgr2=0x7ffb8c1bc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 shutdown_connections 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.099+0000 7ffb92e55700 1 -- 172.21.15.47:0/619788113 wait complete. 2024-04-20T02:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.100+0000 7ffb92e55700 1 librados: shutdown 2024-04-20T02:29:25.114 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-04-20T02:29:25.114 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-04-20T02:29:25.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:29:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.251+0000 7fac897ff700 10 monclient: get_monmap_and_config 2024-04-20T02:29:25.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.251+0000 7fac897ff700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.251+0000 7fac897ff700 10 monclient: monmap: 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:25.252713+0000 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:25.252713+0000 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.252 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:25.253 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:25.253 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 Processor -- start 2024-04-20T02:29:25.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 -- start start 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient: init 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient: _add_conns ranks=[0,8,4,1,3,10,6,5,9,2,7] 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fac84137d90 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac8413cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): picked mon.noname-e con 0x7fac840a09e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac8413f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): picked mon.noname-c con 0x7fac8413d1c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac840751e0 con 0x7fac84137d90 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac8409fa40 con 0x7fac8413d1c0 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fac8409dfc0 con 0x7fac840a09e0 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac897ff700 10 monclient(hunting): authenticate will time out at 2444.653793s 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac827fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac8413cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac82ffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac84137d90 0x7fac84138170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59828/0 (socket says 172.21.15.47:59828) 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac82ffd700 1 -- 172.21.15.47:0/2948577019 learned_addr learned my addr 172.21.15.47:0/2948577019 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac827fc700 10 monclient(hunting): get_auth_request con 0x7fac840a09e0 auth_method 0 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.252+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 1 --2- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac8413f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): get_auth_request con 0x7fac8413d1c0 auth_method 0 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 10 monclient(hunting): handle_auth_done global_id 35774 payload 274 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 1 -- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 msgr2=0x7fac8413f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 1 --2- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac8413f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 1 -- 172.21.15.47:0/2948577019 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137480427 0 0) 0x7fac840751e0 con 0x7fac84137d90 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 1 -- 172.21.15.47:0/2948577019 >> v1:172.21.15.47:6789/0 conn(0x7fac84137d90 legacy=0x7fac84138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 1 monclient: found mon.noname-e 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 1 -- 172.21.15.47:0/2948577019 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84138880 con 0x7fac840a09e0 2024-04-20T02:29:25.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac827fc700 1 --2- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac8413cc80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fac780098a0 tx=0x7fac78009bb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ea0dc9e7365153fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 1 -- 172.21.15.47:0/2948577019 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac78011aa0 con 0x7fac840a09e0 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: dump: 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.268 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac897ff700 5 monclient: authenticate success, global_id 35774 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac897ff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 1 -- 172.21.15.47:0/2948577019 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac78011c00 con 0x7fac840a09e0 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac897ff700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac897ff700 4 set_mon_vals no callback set 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 1 -- 172.21.15.47:0/2948577019 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac78011d70 con 0x7fac840a09e0 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.253+0000 7fac81ffb700 10 monclient: dump: 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.270 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 10 monclient: shutdown 2024-04-20T02:29:25.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 -- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 msgr2=0x7fac8413cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 --2- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac8413cc80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fac780098a0 tx=0x7fac78009bb0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 -- 172.21.15.47:0/2948577019 shutdown_connections 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 --2- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac8413f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 --2- 172.21.15.47:0/2948577019 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac8413cc80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 -- 172.21.15.47:0/2948577019 >> 172.21.15.47:0/2948577019 conn(0x7fac840aa8b0 msgr2=0x7fac840a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 -- 172.21.15.47:0/2948577019 shutdown_connections 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.254+0000 7fac897ff700 1 -- 172.21.15.47:0/2948577019 wait complete. 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 10 monclient: monmap: 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 1 librados: starting msgr at 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 1 librados: starting objecter 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 1 Processor -- start 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 1 -- start start 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 1 librados: setting wanted keys 2024-04-20T02:29:25.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 1 librados: calling monclient init 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.255+0000 7fac897ff700 10 monclient: init 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient: _add_conns ranks=[8,7,4,3,6,2,1,5,0] 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac841310c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): picked mon.noname-e con 0x7fac840a09e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fac84137d90 0x7fac840a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): picked mon.noname-i con 0x7fac84137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac840a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): picked mon.noname-c con 0x7fac8413d1c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac8409dc20 con 0x7fac8413d1c0 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fac840a3930 con 0x7fac84137d90 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fac8409e9e0 con 0x7fac840a09e0 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient(hunting): authenticate will time out at 2444.657144s 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac841310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac840a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac841310c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41834/0 (socket says 172.21.15.47:41834) 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 -- 172.21.15.47:0/1985850458 learned_addr learned my addr 172.21.15.47:0/1985850458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): get_auth_request con 0x7fac840a09e0 auth_method 0 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): get_auth_request con 0x7fac8413d1c0 auth_method 0 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fac84137d90 0x7fac840a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): get_auth_request con 0x7fac84137d90 auth_method 0 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient(hunting): handle_auth_done global_id 35783 payload 824 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fac84137d90 msgr2=0x7fac840a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fac84137d90 0x7fac840a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 msgr2=0x7fac840a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac8413d1c0 0x7fac840a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 monclient: found mon.noname-e 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780094a0 con 0x7fac840a09e0 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac82ffd700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac841310c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c520 tx=0x7fac7400c830 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 client.?.objecter ms_handle_connect 0x7fac840a09e0 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac740070d0 con 0x7fac840a09e0 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 monclient: dump: 2024-04-20T02:29:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac74007270 con 0x7fac840a09e0 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 5 monclient: authenticate success, global_id 35783 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient: _renew_subs 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.256+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac84139070 con 0x7fac840a09e0 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac6b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac740133c0 con 0x7fac840a09e0 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac6b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac6b7fe700 10 monclient: dump: 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 10 client.35783.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 10 monclient: _renew_subs 2024-04-20T02:29:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac817fa700 4 set_mon_vals no callback set 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac840a2d60 con 0x7fac840a09e0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 1 librados: init done 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fac740073e0 con 0x7fac840a09e0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 0x7fac840a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.257+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84068180 con 0x7fac8406aac0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fac7003a2f0 0x7fac7003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(519..519 src has 266..519) v4 ==== 6562+0+0 (secure 0 0 0) 0x7fac7404a450 con 0x7fac840a09e0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 client.35783.objecter ms_dispatch 0x7fac84001550 osd_map(519..519 src has 266..519) v4 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 3 client.35783.objecter handle_osd_map got epochs [519,519] > 0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 3 client.35783.objecter handle_osd_map decoding full epoch 519 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 0x7fac840a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient: get_auth_request con 0x7fac8406aac0 auth_method 0 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 20 client.35783.objecter dump_active .. 0 homeless 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac837fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fac7003a2f0 0x7fac7003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac837fe700 10 monclient: get_auth_request con 0x7fac7003a2f0 auth_method 0 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 10 monclient(hunting): handle_auth_done global_id 36504 payload 824 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac8406af60 con 0x7fac8406aac0 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 client.35783.objecter ms_handle_connect 0x7fac8406aac0 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 client.35783.objecter resend_mon_ops 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac827fc700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 0x7fac840a2100 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac8409e9b0 tx=0x7fac78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac78020070 con 0x7fac8406aac0 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 monclient: dump: 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac6b7fe700 10 client.35783.objecter ms_handle_connect 0x7fac7003a2f0 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.258+0000 7fac837fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fac7003a2f0 0x7fac7003c7b0 secure :-1 s=READY pgs=3311 cs=0 l=1 rev1=1 crypto rx=0x7fac840a1280 tx=0x7fac7c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fac780036a0 con 0x7fac8406aac0 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 msgr2=0x7fac840a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 0x7fac840a2100 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac8409e9b0 tx=0x7fac78004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 msgr2=0x7fac840a2100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.259+0000 7fac6b7fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac8406aac0 0x7fac840a2100 secure :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fac8409e9b0 tx=0x7fac78004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac897ff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac897ff700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 0x7fac840a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac897ff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8406a400 con 0x7fac84131600 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 0x7fac840a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient: get_auth_request con 0x7fac84131600 auth_method 0 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 10 monclient(hunting): handle_auth_done global_id 36513 payload 824 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 1 -- 172.21.15.47:0/1985850458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fac84138ab0 con 0x7fac84131600 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac6b7fe700 10 client.35783.objecter ms_handle_connect 0x7fac84131600 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac827fc700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 0x7fac840a2640 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac78000c00 tx=0x7fac78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac6b7fe700 10 client.35783.objecter resend_mon_ops 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fac7802b2b0 con 0x7fac84131600 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac6b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac6b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.267+0000 7fac6b7fe700 10 monclient: dump: 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.288 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7fac7801ba50 con 0x7fac84131600 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 msgr2=0x7fac840a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 0x7fac840a2640 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac78000c00 tx=0x7fac78011000 comp rx=0 tx=0).stop 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 msgr2=0x7fac840a2640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac6b7fe700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fac84131600 0x7fac840a2640 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fac78000c00 tx=0x7fac78011000 comp rx=0 tx=0).stop 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:25.289 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.290 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:25.291 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:25.292 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.293 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:25.294 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.295 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:25.297 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.298 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.300 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.301 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:25.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.306 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.308 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.309 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:25.310 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:25.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 10 librados: watch_flush enter 2024-04-20T02:29:25.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 10 librados: watch_flush exit 2024-04-20T02:29:25.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 20 client.35783.objecter shutdown clearing up homeless session... 2024-04-20T02:29:25.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 10 client.35783.objecter successfully canceled tick 2024-04-20T02:29:25.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fac7003a2f0 msgr2=0x7fac7003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fac7003a2f0 0x7fac7003c7b0 secure :-1 s=READY pgs=3311 cs=0 l=1 rev1=1 crypto rx=0x7fac840a1280 tx=0x7fac7c00b040 comp rx=0 tx=0).stop 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 10 monclient: shutdown 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 msgr2=0x7fac841310c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 --2- 172.21.15.47:0/1985850458 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fac840a09e0 0x7fac841310c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c520 tx=0x7fac7400c830 comp rx=0 tx=0).stop 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac837fe700 1 -- 172.21.15.47:0/1985850458 reap_dead start 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac837fe700 1 -- 172.21.15.47:0/1985850458 reap_dead start 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 shutdown_connections 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 >> 172.21.15.47:0/1985850458 conn(0x7fac840aa8b0 msgr2=0x7fac84135060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 shutdown_connections 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 -- 172.21.15.47:0/1985850458 wait complete. 2024-04-20T02:29:25.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.268+0000 7fac897ff700 1 librados: shutdown 2024-04-20T02:29:25.313 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:25.313 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-04-20T02:29:25.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.449+0000 7fb85fe6b700 10 monclient: get_monmap_and_config 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.450+0000 7fb85fe6b700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.450+0000 7fb85fe6b700 10 monclient: monmap: 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:25.451183+0000 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:25.451183+0000 2024-04-20T02:29:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:25.451 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.450+0000 7fb85fe6b700 1 Processor -- start 2024-04-20T02:29:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.450+0000 7fb85fe6b700 1 -- start start 2024-04-20T02:29:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.450+0000 7fb85fe6b700 10 monclient: init 2024-04-20T02:29:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient: _add_conns ranks=[8,9,4,2,1,0,3,5,7,6,10] 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb858071630 0x7fb858071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): picked mon.noname-e con 0x7fb858071630 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb858071f50 0x7fb8580a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): picked mon.noname-a con 0x7fb858071f50 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb8580a2e90 0x7fb8580a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): picked mon.noname-c con 0x7fb8580a2e90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb858072720 con 0x7fb858071f50 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580a0820 con 0x7fb8580a2e90 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb85809e890 con 0x7fb858071630 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.451+0000 7fb85fe6b700 10 monclient(hunting): authenticate will time out at 2444.852880s 2024-04-20T02:29:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb858071f50 0x7fb8580a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb858071630 0x7fb858071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb8580a2e90 0x7fb8580a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb858071f50 0x7fb8580a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49358/0 (socket says 172.21.15.47:49358) 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 -- 172.21.15.47:0/4036407584 learned_addr learned my addr 172.21.15.47:0/4036407584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb858071630 0x7fb858071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41848/0 (socket says 172.21.15.47:41848) 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): get_auth_request con 0x7fb858071f50 auth_method 0 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): get_auth_request con 0x7fb858071630 auth_method 0 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): get_auth_request con 0x7fb8580a2e90 auth_method 0 2024-04-20T02:29:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85e408700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient(hunting): handle_auth_done global_id 36522 payload 274 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 -- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb858071630 msgr2=0x7fb858071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb858071630 0x7fb858071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 -- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb8580a2e90 msgr2=0x7fb8580a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb8580a2e90 0x7fb8580a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 monclient: found mon.noname-a 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 -- 172.21.15.47:0/4036407584 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580751e0 con 0x7fb858071f50 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85d406700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb858071f50 0x7fb8580a2950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0098f0 tx=0x7fb84c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d5767b6b57c706 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 1 -- 172.21.15.47:0/4036407584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb84c011aa0 con 0x7fb858071f50 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: dump: 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 1 -- 172.21.15.47:0/4036407584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c011c00 con 0x7fb858071f50 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85fe6b700 5 monclient: authenticate success, global_id 36522 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85fe6b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85fe6b700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 1 -- 172.21.15.47:0/4036407584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb84c011d70 con 0x7fb858071f50 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85fe6b700 4 set_mon_vals no callback set 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.452+0000 7fb85cc05700 10 monclient: dump: 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 10 monclient: shutdown 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 -- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb858071f50 msgr2=0x7fb8580a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb858071f50 0x7fb8580a2950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0098f0 tx=0x7fb84c009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 -- 172.21.15.47:0/4036407584 shutdown_connections 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb8580a2e90 0x7fb8580a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb858071630 0x7fb858071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/4036407584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb858071f50 0x7fb8580a2950 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 -- 172.21.15.47:0/4036407584 >> 172.21.15.47:0/4036407584 conn(0x7fb8580a7570 msgr2=0x7fb8580a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 -- 172.21.15.47:0/4036407584 shutdown_connections 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.453+0000 7fb85fe6b700 1 -- 172.21.15.47:0/4036407584 wait complete. 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 10 monclient: monmap: 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.471 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 1 librados: starting msgr at 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 1 librados: starting objecter 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 1 Processor -- start 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 1 -- start start 2024-04-20T02:29:25.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 1 librados: setting wanted keys 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 1 librados: calling monclient init 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.454+0000 7fb85fe6b700 10 monclient: init 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient: _add_conns ranks=[5,4,3,0,6,2,7,1,8] 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb858071630 0x7fb8581ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): picked mon.noname-h con 0x7fb858071630 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 0x7fb8581cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): picked mon.noname-c con 0x7fb858071f50 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8580a2e90 0x7fb858139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): picked mon.noname-g con 0x7fb8580a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85813d6f0 con 0x7fb8580a2e90 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8581357e0 con 0x7fb858071f50 2024-04-20T02:29:25.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb858121e60 con 0x7fb858071630 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient(hunting): authenticate will time out at 2444.856239s 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 0x7fb8581cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85e408700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8580a2e90 msgr2=0x7fb858139d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 0x7fb8581cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:56174/0 (socket says 172.21.15.47:56174) 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 -- 172.21.15.47:0/2867042026 learned_addr learned my addr 172.21.15.47:0/2867042026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): get_auth_request con 0x7fb858071f50 auth_method 0 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85e408700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8580a2e90 0x7fb858139d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85dc07700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb858071630 0x7fb8581ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85dc07700 10 monclient(hunting): get_auth_request con 0x7fb858071630 auth_method 0 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85dc07700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85dc07700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85dc07700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient(hunting): handle_auth_done global_id 84271 payload 824 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb858071630 msgr2=0x7fb8581ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb858071630 0x7fb8581ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8580a2e90 msgr2=0x7fb858139d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8580a2e90 0x7fb858139d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 monclient: found mon.noname-c 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c009550 con 0x7fb858071f50 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85dc07700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb858071630 0x7fb8581ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85d406700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 0x7fb8581cd120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb84c013000 tx=0x7fb84c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 client.?.objecter ms_handle_connect 0x7fb858071f50 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb84c004540 con 0x7fb858071f50 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 monclient: dump: 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.476 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c003b10 con 0x7fb858071f50 2024-04-20T02:29:25.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb84c003c80 con 0x7fb858071f50 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 5 monclient: authenticate success, global_id 84271 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient: _renew_subs 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb858072380 con 0x7fb858071f50 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.455+0000 7fb84bfff700 4 set_mon_vals no callback set 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb84a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb84a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb84a7fc700 10 monclient: dump: 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.478 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 10 client.84271.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 10 monclient: _renew_subs 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85813d310 con 0x7fb858071f50 2024-04-20T02:29:25.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 1 librados: init done 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb84c021070 con 0x7fb858071f50 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 0x7fb85813a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.456+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb858068310 con 0x7fb85806aac0 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb83c03a320 0x7fb83c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 0x7fb85813a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient: get_auth_request con 0x7fb85806aac0 auth_method 0 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(519..519 src has 266..519) v4 ==== 6562+0+0 (secure 0 0 0) 0x7fb84c04bd60 con 0x7fb858071f50 2024-04-20T02:29:25.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 client.84271.objecter ms_dispatch 0x7fb858001550 osd_map(519..519 src has 266..519) v4 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 3 client.84271.objecter handle_osd_map got epochs [519,519] > 0 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 3 client.84271.objecter handle_osd_map decoding full epoch 519 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 20 client.84271.objecter dump_active .. 0 homeless 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85e408700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb83c03a320 0x7fb83c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85e408700 10 monclient: get_auth_request con 0x7fb83c03a320 auth_method 0 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 10 monclient(hunting): handle_auth_done global_id 24626 payload 824 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8580a4480 con 0x7fb85806aac0 2024-04-20T02:29:25.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85dc07700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 0x7fb85813a2d0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fb854011830 tx=0x7fb854011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 client.84271.objecter ms_handle_connect 0x7fb85806aac0 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 client.84271.objecter resend_mon_ops 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb854015ae0 con 0x7fb85806aac0 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb85e408700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb83c03a320 0x7fb83c03c7e0 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 crypto rx=0x7fb85813cad0 tx=0x7fb85000d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 monclient: dump: 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.482 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.457+0000 7fb84a7fc700 10 client.84271.objecter ms_handle_connect 0x7fb83c03a320 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb854015c40 con 0x7fb85806aac0 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 msgr2=0x7fb85813a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 0x7fb85813a2d0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fb854011830 tx=0x7fb854011bf0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 msgr2=0x7fb85813a2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.458+0000 7fb84a7fc700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85806aac0 0x7fb85813a2d0 secure :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fb854011830 tx=0x7fb854011bf0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.465+0000 7fb85fe6b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.465+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 0x7fb85813ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.465+0000 7fb85fe6b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.465+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb858075710 con 0x7fb85813a810 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 0x7fb85813ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient: get_auth_request con 0x7fb85813a810 auth_method 0 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 10 monclient(hunting): handle_auth_done global_id 24635 payload 824 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 1 -- 172.21.15.47:0/2867042026 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb85806f090 con 0x7fb85813a810 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 client.84271.objecter ms_handle_connect 0x7fb85813a810 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 client.84271.objecter resend_mon_ops 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb85dc07700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 0x7fb85813ac20 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fb85401c000 tx=0x7fb854004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb854013870 con 0x7fb85813a810 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 monclient: dump: 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.487 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9174 (secure 0 0 0) 0x7fb854013400 con 0x7fb85813a810 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:25.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 msgr2=0x7fb85813ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 0x7fb85813ac20 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fb85401c000 tx=0x7fb854004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 msgr2=0x7fb85813ac20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.466+0000 7fb84a7fc700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb85813a810 0x7fb85813ac20 secure :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fb85401c000 tx=0x7fb854004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:25.489 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.490 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.491 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.492 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.493 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.494 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.495 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.497 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:25.498 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.499 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.501 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.503 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.505 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:25.506 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.508 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.510 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:25.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.510 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:25.510 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 10 librados: watch_flush enter 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 10 librados: watch_flush exit 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 20 client.84271.objecter shutdown clearing up homeless session... 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 10 client.84271.objecter successfully canceled tick 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb83c03a320 msgr2=0x7fb83c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb83c03a320 0x7fb83c03c7e0 secure :-1 s=READY pgs=3312 cs=0 l=1 rev1=1 crypto rx=0x7fb85813cad0 tx=0x7fb85000d040 comp rx=0 tx=0).stop 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 10 monclient: shutdown 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 msgr2=0x7fb8581cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 0x7fb8581cd120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb84c013000 tx=0x7fb84c004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85e408700 1 -- 172.21.15.47:0/2867042026 reap_dead start 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 shutdown_connections 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 --2- 172.21.15.47:0/2867042026 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb858071f50 0x7fb8581cd120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 >> 172.21.15.47:0/2867042026 conn(0x7fb8580a7570 msgr2=0x7fb85809c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 shutdown_connections 2024-04-20T02:29:25.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 -- 172.21.15.47:0/2867042026 wait complete. 2024-04-20T02:29:25.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.467+0000 7fb85fe6b700 1 librados: shutdown 2024-04-20T02:29:25.513 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:25.513 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-04-20T02:29:25.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:29:25.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.651+0000 7f2888976700 10 monclient: get_monmap_and_config 2024-04-20T02:29:25.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.651+0000 7f2888976700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.651+0000 7f2888976700 10 monclient: monmap: 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:25.652381+0000 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:25.652381+0000 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.652 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.651+0000 7f2888976700 1 Processor -- start 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.651+0000 7f2888976700 1 -- start start 2024-04-20T02:29:25.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.651+0000 7f2888976700 10 monclient: init 2024-04-20T02:29:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient: _add_conns ranks=[9,7,6,10,0,5,1,2,8,3,4] 2024-04-20T02:29:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 0x7f28840a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): picked mon.noname-a con 0x7f28840a07f0 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f28840a0bd0 0x7f28840a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): picked mon.noname-i con 0x7f28840a0bd0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 0x7f28840a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): picked mon.noname-d con 0x7f28840a1050 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2884067800 con 0x7f28840a07f0 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f28840672f0 con 0x7f28840a1050 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f288409ed90 con 0x7f28840a0bd0 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2888976700 10 monclient(hunting): authenticate will time out at 2445.053652s 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 0x7f28840a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 0x7f28840a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 0x7f28840a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49372/0 (socket says 172.21.15.47:49372) 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 0x7f28840a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:60176/0 (socket says 172.21.15.47:60176) 2024-04-20T02:29:25.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 1 -- 172.21.15.47:0/3741140789 learned_addr learned my addr 172.21.15.47:0/3741140789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): get_auth_request con 0x7f28840a07f0 auth_method 0 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): get_auth_request con 0x7f28840a1050 auth_method 0 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2881d9b700 1 --2- 172.21.15.47:0/3741140789 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f28840a0bd0 0x7f28840a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2882d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f288259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2881d9b700 10 monclient(hunting): get_auth_request con 0x7f28840a0bd0 auth_method 0 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2881d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.652+0000 7f2881d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2881d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 10 monclient(hunting): handle_auth_done global_id 55041 payload 274 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 -- 172.21.15.47:0/3741140789 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f28840a0bd0 msgr2=0x7f28840a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 --2- 172.21.15.47:0/3741140789 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f28840a0bd0 0x7f28840a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 -- 172.21.15.47:0/3741140789 >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 msgr2=0x7f28840a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 --2- 172.21.15.47:0/3741140789 >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 0x7f28840a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 monclient: found mon.noname-d 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 -- 172.21.15.47:0/3741140789 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840751e0 con 0x7f28840a1050 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288259c700 1 --2- 172.21.15.47:0/3741140789 >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 0x7f28840a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2882d9d700 1 --2- 172.21.15.47:0/3741140789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 0x7f28840a2db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f287c009d90 tx=0x7f287c00b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ec71f0bd2724124d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 1 -- 172.21.15.47:0/3741140789 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f287c00e070 con 0x7f28840a1050 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: dump: 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.666 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 1 -- 172.21.15.47:0/3741140789 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c012430 con 0x7f28840a1050 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2888976700 5 monclient: authenticate success, global_id 55041 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2888976700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2888976700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 1 -- 172.21.15.47:0/3741140789 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f287c012a10 con 0x7f28840a1050 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f2888976700 4 set_mon_vals no callback set 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.653+0000 7f288159a700 10 monclient: dump: 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.668 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 10 monclient: shutdown 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 -- 172.21.15.47:0/3741140789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 msgr2=0x7f28840a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 --2- 172.21.15.47:0/3741140789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 0x7f28840a2db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f287c009d90 tx=0x7f287c00b330 comp rx=0 tx=0).stop 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 -- 172.21.15.47:0/3741140789 shutdown_connections 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 --2- 172.21.15.47:0/3741140789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f28840a1050 0x7f28840a2db0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 --2- 172.21.15.47:0/3741140789 >> v2:172.21.15.47:3300/0 conn(0x7f28840a07f0 0x7f28840a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 --2- 172.21.15.47:0/3741140789 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f28840a0bd0 0x7f28840a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 -- 172.21.15.47:0/3741140789 >> 172.21.15.47:0/3741140789 conn(0x7f28840a3f90 msgr2=0x7f288406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 -- 172.21.15.47:0/3741140789 shutdown_connections 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.654+0000 7f2888976700 1 -- 172.21.15.47:0/3741140789 wait complete. 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 10 monclient: monmap: 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:25.670 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 1 librados: starting msgr at 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 1 librados: starting objecter 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 1 Processor -- start 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 1 -- start start 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 1 librados: setting wanted keys 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 1 librados: calling monclient init 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.655+0000 7f2888976700 10 monclient: init 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient: _add_conns ranks=[8,2,4,6,1,7,0,3,5] 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f28840a07f0 0x7f28841cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): picked mon.noname-e con 0x7f28840a07f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28840a0bd0 0x7f28841cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): picked mon.noname-b con 0x7f28840a0bd0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28840a1050 0x7f28841ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): picked mon.noname-c con 0x7f28840a1050 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:25.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288409c480 con 0x7f28840a0bd0 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f288409c6d0 con 0x7f28840a1050 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f288409e800 con 0x7f28840a07f0 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2888976700 10 monclient(hunting): authenticate will time out at 2445.057412s 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f28840a07f0 0x7f28841cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f28840a07f0 0x7f28841cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41858/0 (socket says 172.21.15.47:41858) 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 -- 172.21.15.47:0/2109923857 learned_addr learned my addr 172.21.15.47:0/2109923857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): get_auth_request con 0x7f28840a07f0 auth_method 0 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2881d9b700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28840a0bd0 0x7f28841cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2881d9b700 10 monclient(hunting): get_auth_request con 0x7f28840a0bd0 auth_method 0 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2881d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2881d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f2881d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient(hunting): handle_auth_done global_id 35801 payload 824 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28840a1050 msgr2=0x7f28841ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28840a1050 0x7f28841ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28840a0bd0 msgr2=0x7f28841cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f28840a0bd0 0x7f28841cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 monclient: found mon.noname-e 2024-04-20T02:29:25.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.656+0000 7f288259c700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874009550 con 0x7f28840a07f0 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f288259c700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f28840a07f0 0x7f28841cc080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f287400ec10 tx=0x7f2874009e70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 client.?.objecter ms_handle_connect 0x7f28840a07f0 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f287400cd50 con 0x7f28840a07f0 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: dump: 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.675 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400ced0 con 0x7f28840a07f0 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28740138e0 con 0x7f28840a07f0 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 5 monclient: authenticate success, global_id 35801 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 10 monclient: _renew_subs 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2880d99700 4 set_mon_vals no callback set 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c009a70 con 0x7f28840a07f0 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2872ffd700 10 monclient: dump: 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.677 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 10 client.35801.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 10 monclient: _renew_subs 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288409e3c0 con 0x7f28840a07f0 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.657+0000 7f2888976700 1 librados: init done 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2888976700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2874013470 con 0x7f28840a07f0 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2888976700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 0x7f28841b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2884068280 con 0x7f288406aac0 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 0x7f28841b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient: get_auth_request con 0x7f288406aac0 auth_method 0 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f286403a2f0 0x7f286403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(519..519 src has 266..519) v4 ==== 6562+0+0 (secure 0 0 0) 0x7f287404ab40 con 0x7f28840a07f0 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 10 client.35801.objecter ms_dispatch 0x7f2884001550 osd_map(519..519 src has 266..519) v4 2024-04-20T02:29:25.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 3 client.35801.objecter handle_osd_map got epochs [519,519] > 0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 3 client.35801.objecter handle_osd_map decoding full epoch 519 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 20 client.35801.objecter dump_active .. 0 homeless 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2882d9d700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f286403a2f0 0x7f286403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 10 monclient(hunting): handle_auth_done global_id 84280 payload 824 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f288409c840 con 0x7f288406aac0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2882d9d700 10 monclient: get_auth_request con 0x7f286403a2f0 auth_method 0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 10 client.35801.objecter ms_handle_connect 0x7f288406aac0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2872ffd700 10 client.35801.objecter resend_mon_ops 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.658+0000 7f2881d9b700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 0x7f28841b56c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2884075720 tx=0x7f2878011960 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2878015ae0 con 0x7f288406aac0 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 10 monclient: dump: 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.680 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 10 client.35801.objecter ms_handle_connect 0x7f286403a2f0 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2882d9d700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f286403a2f0 0x7f286403c7b0 secure :-1 s=READY pgs=3313 cs=0 l=1 rev1=1 crypto rx=0x7f287c009d90 tx=0x7f287c010000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2878015c40 con 0x7f288406aac0 2024-04-20T02:29:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 msgr2=0x7f28841b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 0x7f28841b56c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2884075720 tx=0x7f2878011960 comp rx=0 tx=0).stop 2024-04-20T02:29:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 msgr2=0x7f28841b56c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.659+0000 7f2872ffd700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f288406aac0 0x7f28841b56c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2888976700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2888976700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 0x7f28841b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2888976700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f288406a400 con 0x7f28841b5c00 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 0x7f28841b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient: get_auth_request con 0x7f28841b5c00 auth_method 0 2024-04-20T02:29:25.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 10 monclient(hunting): handle_auth_done global_id 84289 payload 824 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 1 -- 172.21.15.47:0/2109923857 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f288409f350 con 0x7f28841b5c00 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2881d9b700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 0x7f28841b6010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2878009700 tx=0x7f2878004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2872ffd700 10 client.35801.objecter ms_handle_connect 0x7f28841b5c00 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2872ffd700 10 client.35801.objecter resend_mon_ops 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2878025070 con 0x7f28841b5c00 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2872ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2872ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.667+0000 7f2872ffd700 10 monclient: dump: 2024-04-20T02:29:25.685 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7f28780066c0 con 0x7f28841b5c00 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 msgr2=0x7f28841b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 0x7f28841b6010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2878009700 tx=0x7f2878004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 msgr2=0x7f28841b6010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2872ffd700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f28841b5c00 0x7f28841b6010 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2878009700 tx=0x7f2878004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:25.687 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:25.688 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.689 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:25.690 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.691 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.693 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.694 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.696 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:25.697 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.699 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.702 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.705 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.706 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.707 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.707 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.707 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:25.707 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 10 librados: watch_flush enter 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 10 librados: watch_flush exit 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 20 client.35801.objecter shutdown clearing up homeless session... 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 10 client.35801.objecter successfully canceled tick 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f286403a2f0 msgr2=0x7f286403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f286403a2f0 0x7f286403c7b0 secure :-1 s=READY pgs=3313 cs=0 l=1 rev1=1 crypto rx=0x7f287c009d90 tx=0x7f287c010000 comp rx=0 tx=0).stop 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 10 monclient: shutdown 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f28840a07f0 msgr2=0x7f28841cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 --2- 172.21.15.47:0/2109923857 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f28840a07f0 0x7f28841cc080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f287400ec10 tx=0x7f2874009e70 comp rx=0 tx=0).stop 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2882d9d700 1 -- 172.21.15.47:0/2109923857 reap_dead start 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2882d9d700 1 -- 172.21.15.47:0/2109923857 reap_dead start 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 shutdown_connections 2024-04-20T02:29:25.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 >> 172.21.15.47:0/2109923857 conn(0x7f28840a3f90 msgr2=0x7f28840a26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 shutdown_connections 2024-04-20T02:29:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 -- 172.21.15.47:0/2109923857 wait complete. 2024-04-20T02:29:25.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.668+0000 7f2888976700 1 librados: shutdown 2024-04-20T02:29:25.710 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:25.710 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-04-20T02:29:25.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:29:25.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.853+0000 7f48c507b700 10 monclient: get_monmap_and_config 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.853+0000 7f48c507b700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.853+0000 7f48c507b700 10 monclient: monmap: 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:25.854361+0000 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:25.854361+0000 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:25.854 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.853+0000 7f48c507b700 1 Processor -- start 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.853+0000 7f48c507b700 1 -- start start 2024-04-20T02:29:25.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.853+0000 7f48c507b700 10 monclient: init 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient: _add_conns ranks=[6,2,8,4,10,7,0,1,3,9,5] 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 0x7f48c00a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): picked mon.noname-d con 0x7f48c01341a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c0134580 0x7f48c0130900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): picked mon.noname-b con 0x7f48c0134580 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 0x7f48c0130e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): picked mon.noname-e con 0x7f48c0134a00 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00a0000 con 0x7f48c0134580 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48c009f620 con 0x7f48c01341a0 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f48c009f900 con 0x7f48c0134a00 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48c507b700 10 monclient(hunting): authenticate will time out at 2445.255421s 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 0x7f48c00a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 0x7f48c0130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 0x7f48c0130e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41872/0 (socket says 172.21.15.47:41872) 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 0x7f48c00a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:60186/0 (socket says 172.21.15.47:60186) 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 -- 172.21.15.47:0/4071313845 learned_addr learned my addr 172.21.15.47:0/4071313845 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): get_auth_request con 0x7f48c0134a00 auth_method 0 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c0134580 0x7f48c0130900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): get_auth_request con 0x7f48c01341a0 auth_method 0 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): get_auth_request con 0x7f48c0134580 auth_method 0 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48be59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient(hunting): handle_auth_done global_id 35810 payload 274 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 -- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 msgr2=0x7f48c00a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 0x7f48c00a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 -- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c0134580 msgr2=0x7f48c0130900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c0134580 0x7f48c0130900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.854+0000 7f48bf59e700 1 monclient: found mon.noname-e 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bf59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bf59e700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:25.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bf59e700 1 -- 172.21.15.47:0/4071313845 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00751e0 con 0x7f48c0134a00 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bed9d700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 0x7f48c00a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48be59c700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c0134580 0x7f48c0130900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bf59e700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 0x7f48c0130e40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f48b800d180 tx=0x7f48b800d490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=162d5a4a9e50916f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 1 -- 172.21.15.47:0/4071313845 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b8010070 con 0x7f48c0134a00 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: dump: 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.872 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 1 -- 172.21.15.47:0/4071313845 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b800bbc0 con 0x7f48c0134a00 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 5 monclient: authenticate success, global_id 35810 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 1 -- 172.21.15.47:0/4071313845 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b800dd40 con 0x7f48c0134a00 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 4 set_mon_vals no callback set 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48bdd9b700 10 monclient: dump: 2024-04-20T02:29:25.873 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.874 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 10 monclient: shutdown 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 1 -- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 msgr2=0x7f48c0130e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.855+0000 7f48c507b700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 0x7f48c0130e40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f48b800d180 tx=0x7f48b800d490 comp rx=0 tx=0).stop 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 -- 172.21.15.47:0/4071313845 shutdown_connections 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f48c0134a00 0x7f48c0130e40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c01341a0 0x7f48c00a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 --2- 172.21.15.47:0/4071313845 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f48c0134580 0x7f48c0130900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 -- 172.21.15.47:0/4071313845 >> 172.21.15.47:0/4071313845 conn(0x7f48c00a2db0 msgr2=0x7f48c00a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 -- 172.21.15.47:0/4071313845 shutdown_connections 2024-04-20T02:29:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 1 -- 172.21.15.47:0/4071313845 wait complete. 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 10 monclient: build_initial_monmap 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.856+0000 7f48c507b700 10 monclient: monmap: 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:25.876 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 librados: starting msgr at 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 librados: starting objecter 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 Processor -- start 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 -- start start 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 librados: setting wanted keys 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 librados: calling monclient init 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 10 monclient: init 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 10 monclient: _add_conns ranks=[4,5,0,2,8,6,1,3,7] 2024-04-20T02:29:25.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48c01341a0 0x7f48c0131070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 10 monclient(hunting): picked mon.noname-c con 0x7f48c01341a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.857+0000 7f48c507b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48c0134580 0x7f48c012c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): picked mon.noname-h con 0x7f48c0134580 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c0134a00 0x7f48c012c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): picked mon.noname-a con 0x7f48c0134a00 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c009ee40 con 0x7f48c0134a00 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01315b0 con 0x7f48c01341a0 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48c012fb30 con 0x7f48c0134580 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient(hunting): authenticate will time out at 2445.259055s 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48c01341a0 0x7f48c0131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c0134a00 0x7f48c012c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48c01341a0 0x7f48c0131070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:56222/0 (socket says 172.21.15.47:56222) 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 1 -- 172.21.15.47:0/2880867790 learned_addr learned my addr 172.21.15.47:0/2880867790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): get_auth_request con 0x7f48c0134a00 auth_method 0 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): get_auth_request con 0x7f48c01341a0 auth_method 0 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48c0134580 0x7f48c012c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): get_auth_request con 0x7f48c0134580 auth_method 0 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48be59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient(hunting): handle_auth_done global_id 36540 payload 824 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48c0134580 msgr2=0x7f48c012c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f48c0134580 0x7f48c012c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48c01341a0 msgr2=0x7f48c0131070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48c01341a0 0x7f48c0131070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient: _finish_hunting 0 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 monclient: found mon.noname-a 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:25.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00094a0 con 0x7f48c0134a00 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bed9d700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f48c01341a0 0x7f48c0131070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48bf59e700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c0134a00 0x7f48c012c900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f48b8017000 tx=0x7f48b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 client.?.objecter ms_handle_connect 0x7f48c0134a00 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b8003cb0 con 0x7f48c0134a00 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 monclient: dump: 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.882 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.883 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 5 monclient: authenticate success, global_id 36540 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b8003e70 con 0x7f48c0134a00 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient: _renew_subs 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.858+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b40094a0 con 0x7f48c0134a00 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48a77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b8004040 con 0x7f48c0134a00 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48a77fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 10 client.36540.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48bd59a700 4 set_mon_vals no callback set 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48a77fe700 10 monclient: dump: 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.885 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 10 monclient: _renew_subs 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b800bd40 con 0x7f48c0134a00 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 1 librados: init done 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 0x7f48c012ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0067740 con 0x7f48c006aac0 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.859+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f48b80041c0 con 0x7f48c0134a00 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 0x7f48c012ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient: get_auth_request con 0x7f48c006aac0 auth_method 0 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f48ac03a320 0x7f48ac03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(520..520 src has 266..520) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f48b804bc00 con 0x7f48c0134a00 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 client.36540.objecter ms_dispatch 0x7f48c0001550 osd_map(520..520 src has 266..520) v4 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 3 client.36540.objecter handle_osd_map got epochs [520,520] > 0 2024-04-20T02:29:25.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 3 client.36540.objecter handle_osd_map decoding full epoch 520 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 20 client.36540.objecter dump_active .. 0 homeless 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 10 monclient(hunting): handle_auth_done global_id 55059 payload 824 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48c0067f90 con 0x7f48c006aac0 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 client.36540.objecter ms_handle_connect 0x7f48c006aac0 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 client.36540.objecter resend_mon_ops 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48bed9d700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 0x7f48c012ce40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48b000f4a0 tx=0x7f48b000f860 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48be59c700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f48ac03a320 0x7f48ac03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48be59c700 10 monclient: get_auth_request con 0x7f48ac03a320 auth_method 0 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b00044a0 con 0x7f48c006aac0 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 monclient: dump: 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.888 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f48b0009ec0 con 0x7f48c006aac0 2024-04-20T02:29:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:25.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.860+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 msgr2=0x7f48c012ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.861+0000 7f48a77fe700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 0x7f48c012ce40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48b000f4a0 tx=0x7f48b000f860 comp rx=0 tx=0).stop 2024-04-20T02:29:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.861+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 msgr2=0x7f48c012ce40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.861+0000 7f48a77fe700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c006aac0 0x7f48c012ce40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.861+0000 7f48be59c700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f48ac03a320 0x7f48ac03c7e0 secure :-1 s=READY pgs=3314 cs=0 l=1 rev1=1 crypto rx=0x7f48c0132c00 tx=0x7f48b4009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.861+0000 7f48a77fe700 10 client.36540.objecter ms_handle_connect 0x7f48ac03a320 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48c507b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48c507b700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 0x7f48c009a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48c507b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48c006a400 con 0x7f48c012e4f0 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48bed9d700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 0x7f48c009a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48bed9d700 10 monclient: get_auth_request con 0x7f48c012e4f0 auth_method 0 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48bed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48bed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:25.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.868+0000 7f48bed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48bed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48bed9d700 10 monclient(hunting): handle_auth_done global_id 55068 payload 824 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48bed9d700 1 -- 172.21.15.47:0/2880867790 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48c012dc90 con 0x7f48c012e4f0 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 client.36540.objecter ms_handle_connect 0x7f48c012e4f0 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 client.36540.objecter resend_mon_ops 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48bed9d700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 0x7f48c009a1b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48b000f4a0 tx=0x7f48b0004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b001ee10 con 0x7f48c012e4f0 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 monclient: dump: 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.895 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7f48b001e9a0 con 0x7f48c012e4f0 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:25.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 msgr2=0x7f48c009a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 0x7f48c009a1b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48b000f4a0 tx=0x7f48b0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 msgr2=0x7f48c009a1b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.869+0000 7f48a77fe700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f48c012e4f0 0x7f48c009a1b0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48b000f4a0 tx=0x7f48b0004750 comp rx=0 tx=0).stop 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:25.897 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.898 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.899 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:25.900 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:25.901 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.902 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.903 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.904 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.905 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.906 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.908 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:25.910 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.911 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.916 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:25.917 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:25.918 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:25.918 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 10 librados: watch_flush enter 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 10 librados: watch_flush exit 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 20 client.36540.objecter shutdown clearing up homeless session... 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 10 client.36540.objecter successfully canceled tick 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f48ac03a320 msgr2=0x7f48ac03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f48ac03a320 0x7f48ac03c7e0 secure :-1 s=READY pgs=3314 cs=0 l=1 rev1=1 crypto rx=0x7f48c0132c00 tx=0x7f48b4009210 comp rx=0 tx=0).stop 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 10 monclient: shutdown 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c0134a00 msgr2=0x7f48c012c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 --2- 172.21.15.47:0/2880867790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48c0134a00 0x7f48c012c900 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f48b8017000 tx=0x7f48b8004ad0 comp rx=0 tx=0).stop 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48bf59e700 1 -- 172.21.15.47:0/2880867790 reap_dead start 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48bf59e700 1 -- 172.21.15.47:0/2880867790 reap_dead start 2024-04-20T02:29:25.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 shutdown_connections 2024-04-20T02:29:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 >> 172.21.15.47:0/2880867790 conn(0x7f48c00a2db0 msgr2=0x7f48c01bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 shutdown_connections 2024-04-20T02:29:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 -- 172.21.15.47:0/2880867790 wait complete. 2024-04-20T02:29:25.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:25.870+0000 7f48c507b700 1 librados: shutdown 2024-04-20T02:29:25.921 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:25.921 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-04-20T02:29:25.922 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.059+0000 7f68f13ae700 10 monclient: get_monmap_and_config 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.059+0000 7f68f13ae700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.059+0000 7f68f13ae700 10 monclient: monmap: 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:26.060778+0000 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:26.060778+0000 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.060+0000 7f68f13ae700 1 Processor -- start 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.060+0000 7f68f13ae700 1 -- start start 2024-04-20T02:29:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.060+0000 7f68f13ae700 10 monclient: init 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.060+0000 7f68f13ae700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.060+0000 7f68f13ae700 10 monclient: _add_conns ranks=[0,3,8,7,1,9,6,4,5,10,2] 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f68ec137d90 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68ec0a09e0 0x7f68ec13cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): picked mon.noname-g con 0x7f68ec0a09e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec13d1c0 0x7f68ec13f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): picked mon.noname-e con 0x7f68ec13d1c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ec0751e0 con 0x7f68ec137d90 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec09fa40 con 0x7f68ec0a09e0 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec09dfc0 con 0x7f68ec13d1c0 2024-04-20T02:29:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient(hunting): authenticate will time out at 2445.462100s 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec13d1c0 0x7f68ec13f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68ea7fc700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68ec0a09e0 msgr2=0x7f68ec13cc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eaffd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f68ec137d90 0x7f68ec138170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59838/0 (socket says 172.21.15.47:59838) 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eaffd700 1 -- 172.21.15.47:0/972853401 learned_addr learned my addr 172.21.15.47:0/972853401 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68ea7fc700 1 --2- 172.21.15.47:0/972853401 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68ec0a09e0 0x7f68ec13cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): get_auth_request con 0x7f68ec13d1c0 auth_method 0 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_done global_id 35819 payload 274 2024-04-20T02:29:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 -- 172.21.15.47:0/972853401 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68ec0a09e0 msgr2=0x7f68ec13cc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 --2- 172.21.15.47:0/972853401 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68ec0a09e0 0x7f68ec13cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 1 -- 172.21.15.47:0/972853401 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299208260 0 0) 0x7f68ec0751e0 con 0x7f68ec137d90 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 -- 172.21.15.47:0/972853401 >> v1:172.21.15.47:6789/0 conn(0x7f68ec137d90 legacy=0x7f68ec138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 monclient: found mon.noname-e 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 -- 172.21.15.47:0/972853401 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec138880 con 0x7f68ec13d1c0 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68eb7fe700 1 --2- 172.21.15.47:0/972853401 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec13d1c0 0x7f68ec13f5b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009df0 tx=0x7f68e400b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d878f7ad79b8cf30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 1 -- 172.21.15.47:0/972853401 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e400e070 con 0x7f68ec13d1c0 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: dump: 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.074 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 5 monclient: authenticate success, global_id 35819 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 1 -- 172.21.15.47:0/972853401 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4012430 con 0x7f68ec13d1c0 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68f13ae700 4 set_mon_vals no callback set 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.061+0000 7f68e9ffb700 1 -- 172.21.15.47:0/972853401 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e4012a10 con 0x7f68ec13d1c0 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68e9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68e9ffb700 10 monclient: dump: 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 10 monclient: shutdown 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 -- 172.21.15.47:0/972853401 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec13d1c0 msgr2=0x7f68ec13f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 --2- 172.21.15.47:0/972853401 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec13d1c0 0x7f68ec13f5b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009df0 tx=0x7f68e400b330 comp rx=0 tx=0).stop 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 -- 172.21.15.47:0/972853401 shutdown_connections 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 --2- 172.21.15.47:0/972853401 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec13d1c0 0x7f68ec13f5b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 --2- 172.21.15.47:0/972853401 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f68ec0a09e0 0x7f68ec13cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 -- 172.21.15.47:0/972853401 >> 172.21.15.47:0/972853401 conn(0x7f68ec0aa8b0 msgr2=0x7f68ec0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 -- 172.21.15.47:0/972853401 shutdown_connections 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.062+0000 7f68f13ae700 1 -- 172.21.15.47:0/972853401 wait complete. 2024-04-20T02:29:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 10 monclient: monmap: 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 1 librados: starting msgr at 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 1 librados: starting objecter 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 1 Processor -- start 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 1 -- start start 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 1 librados: setting wanted keys 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 1 librados: calling monclient init 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.063+0000 7f68f13ae700 10 monclient: init 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient: _add_conns ranks=[0,2,1,6,4,7,5,8,3] 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec0a09e0 0x7f68ec130c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): picked mon.noname-a con 0x7f68ec0a09e0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f68ec137d90 0x7f68ec1311b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): picked mon.noname-b con 0x7f68ec137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68ec13d1c0 0x7f68ec0a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): picked mon.noname-f con 0x7f68ec13d1c0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec138f00 con 0x7f68ec0a09e0 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec09e9e0 con 0x7f68ec13d1c0 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec09dc20 con 0x7f68ec137d90 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68f13ae700 10 monclient(hunting): authenticate will time out at 2445.465450s 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec0a09e0 0x7f68ec130c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f68ec137d90 0x7f68ec1311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec0a09e0 0x7f68ec130c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49392/0 (socket says 172.21.15.47:49392) 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 -- 172.21.15.47:0/2051030805 learned_addr learned my addr 172.21.15.47:0/2051030805 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): get_auth_request con 0x7f68ec0a09e0 auth_method 0 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): get_auth_request con 0x7f68ec137d90 auth_method 0 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68ec13d1c0 0x7f68ec0a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): get_auth_request con 0x7f68ec13d1c0 auth_method 0 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient(hunting): handle_auth_done global_id 36558 payload 824 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f68ec137d90 msgr2=0x7f68ec1311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f68ec137d90 0x7f68ec1311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68ec13d1c0 msgr2=0x7f68ec0a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68ec13d1c0 0x7f68ec0a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 monclient: found mon.noname-a 2024-04-20T02:29:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.064+0000 7f68eaffd700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4009a70 con 0x7f68ec0a09e0 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68ea7fc700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f68ec137d90 0x7f68ec1311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68eaffd700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec0a09e0 0x7f68ec130c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c520 tx=0x7f68dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 client.?.objecter ms_handle_connect 0x7f68ec0a09e0 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68dc0070d0 con 0x7f68ec0a09e0 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: dump: 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc007270 con 0x7f68ec0a09e0 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 5 monclient: authenticate success, global_id 36558 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 10 monclient: _renew_subs 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ec0a3000 con 0x7f68ec0a09e0 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68dc0133c0 con 0x7f68ec0a09e0 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68e97fa700 4 set_mon_vals no callback set 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68d37fe700 10 monclient: dump: 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.085 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 10 client.36558.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 10 monclient: _renew_subs 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ec1316f0 con 0x7f68ec0a09e0 2024-04-20T02:29:26.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.065+0000 7f68f13ae700 1 librados: init done 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68f13ae700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68f13ae700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 0x7f68ec0a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec068180 con 0x7f68ec06aac0 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 0x7f68ec0a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient: get_auth_request con 0x7f68ec06aac0 auth_method 0 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_done global_id 35828 payload 824 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68ec0a5d00 con 0x7f68ec06aac0 2024-04-20T02:29:26.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68d37fe700 10 client.36558.objecter ms_handle_connect 0x7f68ec06aac0 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68d37fe700 10 client.36558.objecter resend_mon_ops 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68ea7fc700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 0x7f68ec0a1bc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0a36e0 tx=0x7f68e0005980 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.066+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f68dc0073e0 con 0x7f68ec0a09e0 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68d803a2f0 0x7f68d803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(520..520 src has 266..520) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f68dc04a450 con 0x7f68ec0a09e0 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 client.36558.objecter ms_dispatch 0x7f68ec001550 osd_map(520..520 src has 266..520) v4 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 3 client.36558.objecter handle_osd_map got epochs [520,520] > 0 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 3 client.36558.objecter handle_osd_map decoding full epoch 520 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 20 client.36558.objecter dump_active .. 0 homeless 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e000ec70 con 0x7f68ec06aac0 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 monclient: dump: 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.088 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f68e000edd0 con 0x7f68ec06aac0 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 msgr2=0x7f68ec0a1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 0x7f68ec0a1bc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0a36e0 tx=0x7f68e0005980 comp rx=0 tx=0).stop 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 msgr2=0x7f68ec0a1bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec06aac0 0x7f68ec0a1bc0 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0a36e0 tx=0x7f68e0005980 comp rx=0 tx=0).stop 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68eb7fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68d803a2f0 0x7f68d803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68eb7fe700 10 monclient: get_auth_request con 0x7f68d803a2f0 auth_method 0 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68eb7fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68d803a2f0 0x7f68d803c7b0 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 crypto rx=0x7f68ec09f3c0 tx=0x7f68e4011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.067+0000 7f68d37fe700 10 client.36558.objecter ms_handle_connect 0x7f68d803a2f0 2024-04-20T02:29:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68f13ae700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68f13ae700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 0x7f68ec0a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68f13ae700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec06a400 con 0x7f68ec0a2100 2024-04-20T02:29:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 0x7f68ec0a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient: get_auth_request con 0x7f68ec0a2100 auth_method 0 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 10 monclient(hunting): handle_auth_done global_id 35837 payload 824 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 1 -- 172.21.15.47:0/2051030805 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f68ec0a7bc0 con 0x7f68ec0a2100 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68d37fe700 10 client.36558.objecter ms_handle_connect 0x7f68ec0a2100 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68d37fe700 10 client.36558.objecter resend_mon_ops 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.075+0000 7f68ea7fc700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 0x7f68ec0a2510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009fd0 tx=0x7f68e0005a50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f68e0012860 con 0x7f68ec0a2100 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 10 monclient: dump: 2024-04-20T02:29:26.093 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.094 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7f68e00123f0 con 0x7f68ec0a2100 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 msgr2=0x7f68ec0a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 0x7f68ec0a2510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009fd0 tx=0x7f68e0005a50 comp rx=0 tx=0).stop 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 msgr2=0x7f68ec0a2510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68d37fe700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f68ec0a2100 0x7f68ec0a2510 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009fd0 tx=0x7f68e0005a50 comp rx=0 tx=0).stop 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:26.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.096 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.097 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.098 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:26.099 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:26.100 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.101 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.104 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.105 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.107 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.108 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.110 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.111 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.113 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:26.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.116 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:26.117 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 10 librados: watch_flush enter 2024-04-20T02:29:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 10 librados: watch_flush exit 2024-04-20T02:29:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 20 client.36558.objecter shutdown clearing up homeless session... 2024-04-20T02:29:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 10 client.36558.objecter successfully canceled tick 2024-04-20T02:29:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68d803a2f0 msgr2=0x7f68d803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f68d803a2f0 0x7f68d803c7b0 secure :-1 s=READY pgs=3315 cs=0 l=1 rev1=1 crypto rx=0x7f68ec09f3c0 tx=0x7f68e4011000 comp rx=0 tx=0).stop 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 10 monclient: shutdown 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec0a09e0 msgr2=0x7f68ec130c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 --2- 172.21.15.47:0/2051030805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68ec0a09e0 0x7f68ec130c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c520 tx=0x7f68dc00c830 comp rx=0 tx=0).stop 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68eb7fe700 1 -- 172.21.15.47:0/2051030805 reap_dead start 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68eb7fe700 1 -- 172.21.15.47:0/2051030805 reap_dead start 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 shutdown_connections 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 >> 172.21.15.47:0/2051030805 conn(0x7f68ec0aa8b0 msgr2=0x7f68ec135060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 shutdown_connections 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 -- 172.21.15.47:0/2051030805 wait complete. 2024-04-20T02:29:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.076+0000 7f68f13ae700 1 librados: shutdown 2024-04-20T02:29:26.121 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:26.121 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-04-20T02:29:26.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.259+0000 7fecb5f00700 10 monclient: get_monmap_and_config 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.259+0000 7fecb5f00700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.259+0000 7fecb5f00700 10 monclient: monmap: 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:26.261098+0000 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:26.261098+0000 2024-04-20T02:29:26.260 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:26.261 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.260+0000 7fecb5f00700 1 Processor -- start 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.260+0000 7fecb5f00700 1 -- start start 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.260+0000 7fecb5f00700 10 monclient: init 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient: _add_conns ranks=[5,9,0,2,3,10,4,1,6,7,8] 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fecb009f9e0 0x7fecb0135e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): picked mon.noname-h con 0x7fecb009f9e0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fecb009fdc0 0x7fecb00a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): picked mon.noname-a con 0x7fecb009fdc0 addr v2:172.21.15.47:3300/0 2024-04-20T02:29:26.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fecb00a1f10 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00751e0 con 0x7fecb00a1f10 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb0136350 con 0x7fecb009fdc0 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00a0740 con 0x7fecb009f9e0 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecb5f00700 10 monclient(hunting): authenticate will time out at 2445.662391s 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fecb009fdc0 0x7fecb00a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fecb009fdc0 0x7fecb00a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49408/0 (socket says 172.21.15.47:49408) 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 -- 172.21.15.47:0/634385708 learned_addr learned my addr 172.21.15.47:0/634385708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): get_auth_request con 0x7fecb009fdc0 auth_method 0 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaf7fe700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fecb009f9e0 0x7fecb0135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaf7fe700 10 monclient(hunting): get_auth_request con 0x7fecb009f9e0 auth_method 0 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaf7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaf7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient(hunting): handle_auth_done global_id 36567 payload 274 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 -- 172.21.15.47:0/634385708 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fecb009f9e0 msgr2=0x7fecb0135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fecb009f9e0 0x7fecb0135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 -- 172.21.15.47:0/634385708 >> v1:172.21.15.47:6789/0 conn(0x7fecb00a1f10 legacy=0x7fecb00a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 monclient: found mon.noname-a 2024-04-20T02:29:26.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaeffd700 1 -- 172.21.15.47:0/634385708 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb009e520 con 0x7fecb009fdc0 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.261+0000 7fecaf7fe700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fecb009f9e0 0x7fecb0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecaeffd700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb00a0200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feca40098f0 tx=0x7feca4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184820a56d8e7ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 1 -- 172.21.15.47:0/634385708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feca400bb10 con 0x7fecb009fdc0 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: dump: 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.276 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 1 -- 172.21.15.47:0/634385708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca400bc70 con 0x7fecb009fdc0 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 5 monclient: authenticate success, global_id 36567 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 1 -- 172.21.15.47:0/634385708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feca400bde0 con 0x7fecb009fdc0 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 4 set_mon_vals no callback set 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecae7fc700 10 monclient: dump: 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.278 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 10 monclient: shutdown 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 -- 172.21.15.47:0/634385708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 msgr2=0x7fecb00a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb00a0200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feca40098f0 tx=0x7feca4009c00 comp rx=0 tx=0).stop 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 -- 172.21.15.47:0/634385708 shutdown_connections 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fecb009f9e0 0x7fecb0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 --2- 172.21.15.47:0/634385708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb00a0200 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 -- 172.21.15.47:0/634385708 >> 172.21.15.47:0/634385708 conn(0x7fecb00a3000 msgr2=0x7fecb00a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.262+0000 7fecb5f00700 1 -- 172.21.15.47:0/634385708 shutdown_connections 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.263+0000 7fecb5f00700 1 -- 172.21.15.47:0/634385708 wait complete. 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.263+0000 7fecb5f00700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.263+0000 7fecb5f00700 10 monclient: monmap: 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.263+0000 7fecb5f00700 1 librados: starting msgr at 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.263+0000 7fecb5f00700 1 librados: starting objecter 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.263+0000 7fecb5f00700 1 Processor -- start 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 -- start start 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 librados: setting wanted keys 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 librados: calling monclient init 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient: init 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient: _add_conns ranks=[3,0,4,1,5,7,2,6,8] 2024-04-20T02:29:26.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecb009f9e0 0x7fecb00725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): picked mon.noname-g con 0x7fecb009f9e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb0132f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): picked mon.noname-a con 0x7fecb009fdc0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fecb00a1f10 0x7fecb012b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): picked mon.noname-c con 0x7fecb00a1f10 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb012e6d0 con 0x7fecb009fdc0 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01cb4e0 con 0x7fecb009f9e0 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fecb012db70 con 0x7fecb00a1f10 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecb5f00700 10 monclient(hunting): authenticate will time out at 2445.665718s 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb0132f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaf7fe700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecb009f9e0 msgr2=0x7fecb00725c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:29:26.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaf7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecb009f9e0 0x7fecb00725c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb0132f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49412/0 (socket says 172.21.15.47:49412) 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 1 -- 172.21.15.47:0/1670857438 learned_addr learned my addr 172.21.15.47:0/1670857438 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): get_auth_request con 0x7fecb009fdc0 auth_method 0 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaffff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fecb00a1f10 0x7fecb012b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaeffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaffff700 10 monclient(hunting): get_auth_request con 0x7fecb00a1f10 auth_method 0 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.264+0000 7fecaffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 10 monclient(hunting): handle_auth_done global_id 36576 payload 824 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fecb00a1f10 msgr2=0x7fecb012b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fecb00a1f10 0x7fecb012b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecb009f9e0 msgr2=0x7fecb00725c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fecb009f9e0 0x7fecb00725c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 monclient: found mon.noname-a 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca4009550 con 0x7fecb009fdc0 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecaeffd700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb0132f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feca4009130 tx=0x7feca4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 client.?.objecter ms_handle_connect 0x7fecb009fdc0 2024-04-20T02:29:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feca40036a0 con 0x7fecb009fdc0 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: dump: 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.285 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 5 monclient: authenticate success, global_id 36576 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca4003860 con 0x7fecb009fdc0 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feca4003e40 con 0x7fecb009fdc0 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecadffb700 4 set_mon_vals no callback set 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fec97fff700 10 monclient: dump: 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.287 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 10 monclient: _renew_subs 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb012e3f0 con 0x7fecb009fdc0 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 10 client.36576.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 10 monclient: _renew_subs 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb01caa70 con 0x7fecb009fdc0 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.265+0000 7fecb5f00700 1 librados: init done 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7feca4021070 con 0x7fecb009fdc0 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecb5f00700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecb5f00700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 0x7fecb012bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0068210 con 0x7fecb012c710 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec9c03a2f0 0x7fec9c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(520..520 src has 266..520) v4 ==== 6557+0+0 (secure 0 0 0) 0x7feca404bc20 con 0x7fecb009fdc0 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 10 client.36576.objecter ms_dispatch 0x7fecb0001550 osd_map(520..520 src has 266..520) v4 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 3 client.36576.objecter handle_osd_map got epochs [520,520] > 0 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 3 client.36576.objecter handle_osd_map decoding full epoch 520 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fec97fff700 20 client.36576.objecter dump_active .. 0 homeless 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaf7fe700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 0x7fecb012bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaf7fe700 10 monclient: get_auth_request con 0x7fecb012c710 auth_method 0 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaf7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaf7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaffff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec9c03a2f0 0x7fec9c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.266+0000 7fecaffff700 10 monclient: get_auth_request con 0x7fec9c03a2f0 auth_method 0 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_done global_id 55423 payload 824 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaf7fe700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fecb0135490 con 0x7fecb012c710 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 client.36576.objecter ms_handle_connect 0x7fec9c03a2f0 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaffff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec9c03a2f0 0x7fec9c03c7b0 secure :-1 s=READY pgs=3316 cs=0 l=1 rev1=1 crypto rx=0x7feca80012e0 tx=0x7feca8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 client.36576.objecter ms_handle_connect 0x7fecb012c710 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 client.36576.objecter resend_mon_ops 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fecaf7fe700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 0x7fecb012bb00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feca000d700 tx=0x7feca000dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feca00044a0 con 0x7fecb012c710 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 monclient: dump: 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.291 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7feca0004030 con 0x7fecb012c710 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 msgr2=0x7fecb012bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 0x7fecb012bb00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feca000d700 tx=0x7feca000dac0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 msgr2=0x7fecb012bb00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.267+0000 7fec97fff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb012c710 0x7fecb012bb00 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feca000d700 tx=0x7feca000dac0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.275+0000 7fecb5f00700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.275+0000 7fecb5f00700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 0x7fecb012c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.275+0000 7fecb5f00700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.275+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb006a400 con 0x7fecb006aac0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 0x7fecb012c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient: get_auth_request con 0x7fecb006aac0 auth_method 0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 10 monclient(hunting): handle_auth_done global_id 55432 payload 824 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 1 -- 172.21.15.47:0/1670857438 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fecb012e840 con 0x7fecb006aac0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fecaf7fe700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 0x7fecb012c040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feca0006eb0 tx=0x7feca00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fec97fff700 10 client.36576.objecter ms_handle_connect 0x7fecb006aac0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fec97fff700 10 client.36576.objecter resend_mon_ops 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feca0003de0 con 0x7fecb006aac0 2024-04-20T02:29:26.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fec97fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fec97fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.276+0000 7fec97fff700 10 monclient: dump: 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.296 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7feca0003970 con 0x7fecb006aac0 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 msgr2=0x7fecb012c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 0x7fecb012c040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feca0006eb0 tx=0x7feca00047c0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 msgr2=0x7fecb012c040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fec97fff700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb006aac0 0x7fecb012c040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:26.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.298 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.299 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:26.300 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.301 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:26.302 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:26.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.305 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.305 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.307 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:26.308 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:26.311 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.315 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.316 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.317 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.319 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:26.320 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 10 librados: watch_flush enter 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 10 librados: watch_flush exit 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 20 client.36576.objecter shutdown clearing up homeless session... 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 10 client.36576.objecter successfully canceled tick 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec9c03a2f0 msgr2=0x7fec9c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fec9c03a2f0 0x7fec9c03c7b0 secure :-1 s=READY pgs=3316 cs=0 l=1 rev1=1 crypto rx=0x7feca80012e0 tx=0x7feca8009040 comp rx=0 tx=0).stop 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 10 monclient: shutdown 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 msgr2=0x7fecb0132f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 --2- 172.21.15.47:0/1670857438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fecb009fdc0 0x7fecb0132f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feca4009130 tx=0x7feca4004750 comp rx=0 tx=0).stop 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecaffff700 1 -- 172.21.15.47:0/1670857438 reap_dead start 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecaffff700 1 -- 172.21.15.47:0/1670857438 reap_dead start 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 shutdown_connections 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 >> 172.21.15.47:0/1670857438 conn(0x7fecb00a3000 msgr2=0x7fecb009e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 shutdown_connections 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 -- 172.21.15.47:0/1670857438 wait complete. 2024-04-20T02:29:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.277+0000 7fecb5f00700 1 librados: shutdown 2024-04-20T02:29:26.323 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:26.323 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-04-20T02:29:26.324 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.462+0000 7f64ad3d3700 10 monclient: get_monmap_and_config 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.462+0000 7f64ad3d3700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.462+0000 7f64ad3d3700 10 monclient: monmap: 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:26.463946+0000 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:26.463946+0000 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:26.463 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:26.464 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.463+0000 7f64ad3d3700 1 Processor -- start 2024-04-20T02:29:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.463+0000 7f64ad3d3700 1 -- start start 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.463+0000 7f64ad3d3700 10 monclient: init 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.463+0000 7f64ad3d3700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.463+0000 7f64ad3d3700 10 monclient: _add_conns ranks=[8,6,1,2,10,3,9,4,7,0,5] 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): picked mon.noname-e con 0x7f64a8137d90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 0x7f64a80a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): picked mon.noname-d con 0x7f64a8138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f64a81385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a80751e0 con 0x7f64a81385b0 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f64a809fa40 con 0x7f64a8138170 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f64a809dfc0 con 0x7f64a8137d90 2024-04-20T02:29:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64ad3d3700 10 monclient(hunting): authenticate will time out at 2445.865110s 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a80a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 0x7f64a80a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a80a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41902/0 (socket says 172.21.15.47:41902) 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 1 -- 172.21.15.47:0/3746832040 learned_addr learned my addr 172.21.15.47:0/3746832040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 0x7f64a80a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:60210/0 (socket says 172.21.15.47:60210) 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): get_auth_request con 0x7f64a8137d90 auth_method 0 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): get_auth_request con 0x7f64a8138170 auth_method 0 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient(hunting): handle_auth_done global_id 55077 payload 274 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 -- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 msgr2=0x7f64a80a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a80a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 -- 172.21.15.47:0/3746832040 >> v1:172.21.15.179:6789/0 conn(0x7f64a81385b0 legacy=0x7f64a80a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 monclient: found mon.noname-d 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 -- 172.21.15.47:0/3746832040 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a80a7bc0 con 0x7f64a8138170 2024-04-20T02:29:26.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a5ffb700 1 -- 172.21.15.47:0/3746832040 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859569759 0 0) 0x7f64a80751e0 con 0x7f64a81385b0 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a5ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a6ffd700 1 --2- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a80a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 0x7f64a80a5c90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64900098a0 tx=0x7f6490009bb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=bc9c5dd7093af3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a5ffb700 1 -- 172.21.15.47:0/3746832040 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6490011aa0 con 0x7f64a8138170 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.464+0000 7f64a5ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: dump: 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.479 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 1 -- 172.21.15.47:0/3746832040 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490011c00 con 0x7f64a8138170 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 5 monclient: authenticate success, global_id 55077 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:26.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 1 -- 172.21.15.47:0/3746832040 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6490011d70 con 0x7f64a8138170 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 4 set_mon_vals no callback set 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64a5ffb700 10 monclient: dump: 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 10 monclient: shutdown 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 msgr2=0x7f64a80a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 0x7f64a80a5c90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f64900098a0 tx=0x7f6490009bb0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3746832040 shutdown_connections 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a80a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3746832040 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f64a8138170 0x7f64a80a5c90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3746832040 >> 172.21.15.47:0/3746832040 conn(0x7f64a80aa8b0 msgr2=0x7f64a80a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3746832040 shutdown_connections 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.465+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3746832040 wait complete. 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.466+0000 7f64ad3d3700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.466+0000 7f64ad3d3700 10 monclient: monmap: 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.466+0000 7f64ad3d3700 1 librados: starting msgr at 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.466+0000 7f64ad3d3700 1 librados: starting objecter 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.466+0000 7f64ad3d3700 1 Processor -- start 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 -- start start 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 librados: setting wanted keys 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 librados: calling monclient init 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient: init 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient: _add_conns ranks=[8,1,0,2,6,4,5,7,3] 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a81cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): picked mon.noname-e con 0x7f64a8137d90 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a8138170 0x7f64a8136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): picked mon.noname-f con 0x7f64a8138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64a81385b0 0x7f64a8136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): picked mon.noname-a con 0x7f64a81385b0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8124220 con 0x7f64a81385b0 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8124470 con 0x7f64a8138170 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f64a809e9e0 con 0x7f64a8137d90 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64ad3d3700 10 monclient(hunting): authenticate will time out at 2445.868613s 2024-04-20T02:29:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a81cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a81cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41918/0 (socket says 172.21.15.47:41918) 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 1 -- 172.21.15.47:0/3137030781 learned_addr learned my addr 172.21.15.47:0/3137030781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): get_auth_request con 0x7f64a8137d90 auth_method 0 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64a81385b0 0x7f64a8136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): get_auth_request con 0x7f64a81385b0 auth_method 0 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a8138170 0x7f64a8136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a67fc700 10 monclient(hunting): get_auth_request con 0x7f64a8138170 auth_method 0 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.467+0000 7f64a67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 10 monclient(hunting): handle_auth_done global_id 35855 payload 824 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a8138170 msgr2=0x7f64a8136ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64a8138170 0x7f64a8136ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64a81385b0 msgr2=0x7f64a8136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64a81385b0 0x7f64a8136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 monclient: found mon.noname-e 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6498009550 con 0x7f64a8137d90 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a77fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64a81385b0 0x7f64a8136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a6ffd700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a81cc200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f649800ec10 tx=0x7f649800ef20 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 client.?.objecter ms_handle_connect 0x7f64a8137d90 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f649800ccc0 con 0x7f64a8137d90 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: dump: 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.489 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649800ce60 con 0x7f64a8137d90 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f64980138e0 con 0x7f64a8137d90 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 5 monclient: authenticate success, global_id 35855 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 10 monclient: _renew_subs 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64900094a0 con 0x7f64a8137d90 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64a57fa700 4 set_mon_vals no callback set 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f649f7fe700 10 monclient: dump: 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.491 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 10 client.35855.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 10 monclient: _renew_subs 2024-04-20T02:29:26.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a8122c70 con 0x7f64a8137d90 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.468+0000 7f64ad3d3700 1 librados: init done 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64ad3d3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 0x7f64a81208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8068180 con 0x7f64a806aac0 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6498013470 con 0x7f64a8137d90 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 0x7f64a81208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient: get_auth_request con 0x7f64a806aac0 auth_method 0 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f649f7fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f648803a2f0 0x7f648803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(520..520 src has 266..520) v4 ==== 6557+0+0 (secure 0 0 0) 0x7f649804aae0 con 0x7f64a8137d90 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f649f7fe700 10 client.35855.objecter ms_dispatch 0x7f64a8001550 osd_map(520..520 src has 266..520) v4 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f649f7fe700 3 client.35855.objecter handle_osd_map got epochs [520,520] > 0 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f649f7fe700 3 client.35855.objecter handle_osd_map decoding full epoch 520 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 10 monclient(hunting): handle_auth_done global_id 84227 payload 824 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.469+0000 7f64a67fc700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64a809f760 con 0x7f64a806aac0 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 20 client.35855.objecter dump_active .. 0 homeless 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f64a77fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f648803a2f0 0x7f648803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f64a77fe700 10 monclient: get_auth_request con 0x7f648803a2f0 auth_method 0 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 client.35855.objecter ms_handle_connect 0x7f64a806aac0 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 client.35855.objecter resend_mon_ops 2024-04-20T02:29:26.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 0x7f64a81208d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f64a809e9b0 tx=0x7f6490004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6490020070 con 0x7f64a806aac0 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 monclient: dump: 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.495 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 client.35855.objecter ms_handle_connect 0x7f648803a2f0 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f64a77fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f648803a2f0 0x7f648803c7b0 secure :-1 s=READY pgs=3317 cs=0 l=1 rev1=1 crypto rx=0x7f64a80a9880 tx=0x7f64a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f64900045e0 con 0x7f64a806aac0 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 msgr2=0x7f64a81208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 0x7f64a81208d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f64a809e9b0 tx=0x7f6490004750 comp rx=0 tx=0).stop 2024-04-20T02:29:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 msgr2=0x7f64a81208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.470+0000 7f649f7fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a806aac0 0x7f64a81208d0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f64a809e9b0 tx=0x7f6490004750 comp rx=0 tx=0).stop 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64ad3d3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 0x7f64a81211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64ad3d3700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64a806a400 con 0x7f64a8120e10 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 0x7f64a81211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64a67fc700 10 monclient: get_auth_request con 0x7f64a8120e10 auth_method 0 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64a67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64a67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.478+0000 7f64a67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f64a67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f64a67fc700 10 monclient(hunting): handle_auth_done global_id 84236 payload 824 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f64a67fc700 1 -- 172.21.15.47:0/3137030781 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f64a8121730 con 0x7f64a8120e10 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 client.35855.objecter ms_handle_connect 0x7f64a8120e10 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 client.35855.objecter resend_mon_ops 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f64a67fc700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 0x7f64a81211f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6490009870 tx=0x7f6490011000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6490003ec0 con 0x7f64a8120e10 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 monclient: dump: 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.500 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9172 (secure 0 0 0) 0x7f649002b2b0 con 0x7f64a8120e10 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 msgr2=0x7f64a81211f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 0x7f64a81211f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6490009870 tx=0x7f6490011000 comp rx=0 tx=0).stop 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 msgr2=0x7f64a81211f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.479+0000 7f649f7fe700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f64a8120e10 0x7f64a81211f0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6490009870 tx=0x7f6490011000 comp rx=0 tx=0).stop 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:26.502 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.503 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:26.504 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:26.505 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.506 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:26.508 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.509 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.510 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:26.512 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.513 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.515 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.516 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.518 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:26.520 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.522 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.523 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.523 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:26.523 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 10 librados: watch_flush enter 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 10 librados: watch_flush exit 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 20 client.35855.objecter shutdown clearing up homeless session... 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 10 client.35855.objecter successfully canceled tick 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f648803a2f0 msgr2=0x7f648803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f648803a2f0 0x7f648803c7b0 secure :-1 s=READY pgs=3317 cs=0 l=1 rev1=1 crypto rx=0x7f64a80a9880 tx=0x7f64a0009040 comp rx=0 tx=0).stop 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 10 monclient: shutdown 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 msgr2=0x7f64a81cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 --2- 172.21.15.47:0/3137030781 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f64a8137d90 0x7f64a81cc200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f649800ec10 tx=0x7f649800ef20 comp rx=0 tx=0).stop 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64a77fe700 1 -- 172.21.15.47:0/3137030781 reap_dead start 2024-04-20T02:29:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64a77fe700 1 -- 172.21.15.47:0/3137030781 reap_dead start 2024-04-20T02:29:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 shutdown_connections 2024-04-20T02:29:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 >> 172.21.15.47:0/3137030781 conn(0x7f64a80aa8b0 msgr2=0x7f64a81bfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 shutdown_connections 2024-04-20T02:29:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 -- 172.21.15.47:0/3137030781 wait complete. 2024-04-20T02:29:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.480+0000 7f64ad3d3700 1 librados: shutdown 2024-04-20T02:29:26.526 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:26.526 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-04-20T02:29:26.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.664+0000 7fa121537700 10 monclient: get_monmap_and_config 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.664+0000 7fa121537700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.664+0000 7fa121537700 10 monclient: monmap: 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:29:26.665257+0000 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:29:26.665257+0000 2024-04-20T02:29:26.664 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:29:26.665 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.664+0000 7fa121537700 1 Processor -- start 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.664+0000 7fa121537700 1 -- start start 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.664+0000 7fa121537700 10 monclient: init 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient: _add_conns ranks=[3,10,0,9,8,4,2,6,7,5,1] 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c0a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): picked mon.noname-g con 0x7fa11c0a0780 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa11c0a0b60 0x7fa11c0a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): picked mon.noname-f con 0x7fa11c0a0b60 addr v2:172.21.15.179:3300/0 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa11c0a2950 addr v1:172.21.15.47:6789/0 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0751e0 con 0x7fa11c0a2950 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c067770 con 0x7fa11c0a0b60 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c09eee0 con 0x7fa11c0a0780 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa121537700 10 monclient(hunting): authenticate will time out at 2446.066434s 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11affd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 msgr2=0x7fa11c0a0180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c0a0180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11b7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa11c0a2950 0x7fa11c0a2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59858/0 (socket says 172.21.15.47:59858) 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11b7fe700 1 -- 172.21.15.47:0/3230882643 learned_addr learned my addr 172.21.15.47:0/3230882643 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 1 --2- 172.21.15.47:0/3230882643 >> v2:172.21.15.179:3300/0 conn(0x7fa11c0a0b60 0x7fa11c0a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): get_auth_request con 0x7fa11c0a0b60 auth_method 0 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa119ffb700 1 -- 172.21.15.47:0/3230882643 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568578053 0 0) 0x7fa11c0751e0 con 0x7fa11c0a2950 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa119ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa119ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa119ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa119ffb700 10 monclient(hunting): my global_id is 36594 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa119ffb700 1 -- 172.21.15.47:0/3230882643 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1080035b0 con 0x7fa11c0a2950 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_done global_id 55459 payload 274 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 1 -- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 msgr2=0x7fa11c0a0180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 1 --2- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c0a0180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 1 -- 172.21.15.47:0/3230882643 >> v1:172.21.15.47:6789/0 conn(0x7fa11c0a2950 legacy=0x7fa11c0a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 1 monclient: found mon.noname-f 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.665+0000 7fa11a7fc700 1 -- 172.21.15.47:0/3230882643 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c09f1c0 con 0x7fa11c0a0b60 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa11a7fc700 1 --2- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11c0a0b60 0x7fa11c0a0fa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa104009840 tx=0x7fa104009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744dc51dea5b1437 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 1 -- 172.21.15.47:0/3230882643 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa104011aa0 con 0x7fa11c0a0b60 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: dump: 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.680 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 1 -- 172.21.15.47:0/3230882643 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa104011c00 con 0x7fa11c0a0b60 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 5 monclient: authenticate success, global_id 55459 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:29:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 1 -- 172.21.15.47:0/3230882643 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa104011d70 con 0x7fa11c0a0b60 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 10 monclient: get_monmap_and_config success 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 4 set_mon_vals no callback set 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa119ffb700 10 monclient: dump: 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 10 monclient: shutdown 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 1 -- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11c0a0b60 msgr2=0x7fa11c0a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.666+0000 7fa121537700 1 --2- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11c0a0b60 0x7fa11c0a0fa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa104009840 tx=0x7fa104009b50 comp rx=0 tx=0).stop 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 1 -- 172.21.15.47:0/3230882643 shutdown_connections 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 1 --2- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c0a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 1 --2- 172.21.15.47:0/3230882643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa11c0a0b60 0x7fa11c0a0fa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 1 -- 172.21.15.47:0/3230882643 >> 172.21.15.47:0/3230882643 conn(0x7fa11c0a7570 msgr2=0x7fa11c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 1 -- 172.21.15.47:0/3230882643 shutdown_connections 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 1 -- 172.21.15.47:0/3230882643 wait complete. 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 10 monclient: build_initial_monmap 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.667+0000 7fa121537700 10 monclient: monmap: 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:29:26.684 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 1 librados: starting msgr at 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 1 librados: starting objecter 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 1 Processor -- start 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 1 -- start start 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 1 librados: setting wanted keys 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 1 librados: calling monclient init 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.668+0000 7fa121537700 10 monclient: init 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient: _reopen_session rank -1 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient: _add_conns ranks=[3,8,0,7,2,6,4,5,1] 2024-04-20T02:29:26.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c1cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): picked mon.noname-g con 0x7fa11c0a0780 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa11c0a0b60 0x7fa11c1bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): picked mon.noname-e con 0x7fa11c0a0b60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c0a2950 0x7fa11c1bc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): picked mon.noname-a con 0x7fa11c0a2950 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c09e900 con 0x7fa11c0a2950 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c09e440 con 0x7fa11c0a0780 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1cc800 con 0x7fa11c0a0b60 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): _renew_subs 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa121537700 10 monclient(hunting): authenticate will time out at 2446.070542s 2024-04-20T02:29:26.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11affd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 msgr2=0x7fa11c1cc150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c1cc150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c0a2950 0x7fa11c1bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa11c0a0b60 0x7fa11c1bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c0a2950 0x7fa11c1bc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49430/0 (socket says 172.21.15.47:49430) 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa11c0a0b60 0x7fa11c1bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:41932/0 (socket says 172.21.15.47:41932) 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 1 -- 172.21.15.47:0/3040695605 learned_addr learned my addr 172.21.15.47:0/3040695605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): get_auth_request con 0x7fa11c0a2950 auth_method 0 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): get_auth_request con 0x7fa11c0a0b60 auth_method 0 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.669+0000 7fa11a7fc700 10 monclient(hunting): handle_auth_done global_id 35864 payload 824 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 msgr2=0x7fa11c1cc150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa11c0a0780 0x7fa11c1cc150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c0a2950 msgr2=0x7fa11c1bc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c0a2950 0x7fa11c1bc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 monclient: found mon.noname-e 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:29:26.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040094a0 con 0x7fa11c0a0b60 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11b7fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa11c0a2950 0x7fa11c1bc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa11a7fc700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa11c0a0b60 0x7fa11c1bc0e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa104016000 tx=0x7fa104004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 client.?.objecter ms_handle_connect 0x7fa11c0a0b60 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa1040043d0 con 0x7fa11c0a0b60 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: dump: 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.689 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.690 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: _finish_auth 0 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: _check_auth_tickets 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10401f070 con 0x7fa11c0a0b60 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 5 monclient: authenticate success, global_id 35864 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 10 monclient: _renew_subs 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c1ab890 con 0x7fa11c0a0b60 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa104004590 con 0x7fa11c0a0b60 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1137fe700 10 monclient: dump: 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.691 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa1197fa700 4 set_mon_vals no callback set 2024-04-20T02:29:26.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 10 client.35864.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 10 monclient: _renew_subs 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1aaff0 con 0x7fa11c0a0b60 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.670+0000 7fa121537700 1 librados: init done 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa121537700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa121537700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 0x7fa11c1a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c068310 con 0x7fa11c06aac0 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa104021030 con 0x7fa11c0a0b60 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0fc03a320 0x7fa0fc03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 0x7fa11c1a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient: get_auth_request con 0x7fa11c06aac0 auth_method 0 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(520..520 src has 266..520) v4 ==== 6557+0+0 (secure 0 0 0) 0x7fa10404b9d0 con 0x7fa11c0a0b60 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 10 client.35864.objecter ms_dispatch 0x7fa11c001550 osd_map(520..520 src has 266..520) v4 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 3 client.35864.objecter handle_osd_map got epochs [520,520] > 0 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 3 client.35864.objecter handle_osd_map decoding full epoch 520 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa1137fe700 20 client.35864.objecter dump_active .. 0 homeless 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11b7fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0fc03a320 0x7fa0fc03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.671+0000 7fa11b7fe700 10 monclient: get_auth_request con 0x7fa0fc03a320 auth_method 0 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa11affd700 10 monclient(hunting): handle_auth_done global_id 84292 payload 824 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa11affd700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa11c0a4450 con 0x7fa11c06aac0 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa11affd700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 0x7fa11c1a79d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b450 tx=0x7fa10c00b810 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 client.35864.objecter ms_handle_connect 0x7fa11c06aac0 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 client.35864.objecter resend_mon_ops 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 client.35864.objecter ms_handle_connect 0x7fa0fc03a320 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa11b7fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0fc03a320 0x7fa0fc03c7e0 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 crypto rx=0x7fa114001560 tx=0x7fa114009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa10c003c50 con 0x7fa11c06aac0 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 monclient: dump: 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.695 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa10c003db0 con 0x7fa11c06aac0 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 msgr2=0x7fa11c1a79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 0x7fa11c1a79d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b450 tx=0x7fa10c00b810 comp rx=0 tx=0).stop 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 msgr2=0x7fa11c1a79d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.672+0000 7fa1137fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c06aac0 0x7fa11c1a79d0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00b450 tx=0x7fa10c00b810 comp rx=0 tx=0).stop 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa121537700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa121537700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 0x7fa11c1a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa121537700 10 monclient(hunting): start opening mon connection 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0a04d0 con 0x7fa11c1ab180 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa11affd700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 0x7fa11c1a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa11affd700 10 monclient: get_auth_request con 0x7fa11c1ab180 auth_method 0 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa11affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa11affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.680+0000 7fa11affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa11affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa11affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa11affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:29:26.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa11affd700 10 monclient(hunting): handle_auth_done global_id 84301 payload 824 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa11affd700 1 -- 172.21.15.47:0/3040695605 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa11c09f810 con 0x7fa11c1ab180 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa11affd700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 0x7fa11c1a7f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa10c000c00 tx=0x7fa10c005a60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 client.35864.objecter ms_handle_connect 0x7fa11c1ab180 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 client.35864.objecter resend_mon_ops 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa10c01e070 con 0x7fa11c1ab180 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 monclient: dump: 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:29:26.700 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9342 (secure 0 0 0) 0x7fa10c019e50 con 0x7fa11c1ab180 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 msgr2=0x7fa11c1a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.681+0000 7fa1137fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 0x7fa11c1a7f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa10c000c00 tx=0x7fa10c005a60 comp rx=0 tx=0).stop 2024-04-20T02:29:26.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa1137fe700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 msgr2=0x7fa11c1a7f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa1137fe700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa11c1ab180 0x7fa11c1a7f10 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa10c000c00 tx=0x7fa10c005a60 comp rx=0 tx=0).stop 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 94, 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:29:26.702 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.703 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:29:26.704 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:29:26.705 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:29:26.706 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:29:26.707 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.708 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.711 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.712 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:29:26.713 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:29:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.719 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:29:26.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.721 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.722 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:29:26.723 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 10 librados: watch_flush enter 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 10 librados: watch_flush exit 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 20 client.35864.objecter shutdown clearing up homeless session... 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 10 client.35864.objecter successfully canceled tick 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0fc03a320 msgr2=0x7fa0fc03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0fc03a320 0x7fa0fc03c7e0 secure :-1 s=READY pgs=3318 cs=0 l=1 rev1=1 crypto rx=0x7fa114001560 tx=0x7fa114009040 comp rx=0 tx=0).stop 2024-04-20T02:29:26.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 10 monclient: shutdown 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa11c0a0b60 msgr2=0x7fa11c1bc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 --2- 172.21.15.47:0/3040695605 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa11c0a0b60 0x7fa11c1bc0e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa104016000 tx=0x7fa104004750 comp rx=0 tx=0).stop 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa11b7fe700 1 -- 172.21.15.47:0/3040695605 reap_dead start 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa11b7fe700 1 -- 172.21.15.47:0/3040695605 reap_dead start 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 shutdown_connections 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 >> 172.21.15.47:0/3040695605 conn(0x7fa11c0a7570 msgr2=0x7fa11c0a5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 shutdown_connections 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 -- 172.21.15.47:0/3040695605 wait complete. 2024-04-20T02:29:26.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:29:26.682+0000 7fa121537700 1 librados: shutdown 2024-04-20T02:29:26.727 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-04-20T02:29:26.727 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (8961 ms) 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (5987 ms) 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (19176 ms) 2024-04-20T02:30:38.243 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (40834 ms) 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14043 ms) 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (11968 ms) 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 00000010 69 73 |is| 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 00000012 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12041 ms) 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: require_osd_release = reef 2024-04-20T02:30:38.244 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6043 ms) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (672166 ms total) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1025 ms) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10983 ms) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (22173 ms) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (1 ms) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (34182 ms total) 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 2024-04-20T02:30:38.245 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (6991 ms) 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (26723 ms) 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: waiting for scrub... 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: done waiting 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (23285 ms) 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10006 ms) 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8053 ms) 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-04-20T02:30:38.246 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8008 ms) 2024-04-20T02:30:38.247 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-04-20T02:30:38.247 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (9980 ms) 2024-04-20T02:30:38.247 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-04-20T02:30:56.727 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-04-20T02:30:56.727 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-04-20T02:30:56.728 INFO:tasks.ceph.mon.g:Restarting daemon 2024-04-20T02:30:56.728 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-04-20T02:30:56.731 INFO:tasks.ceph.mon.g:Started 2024-04-20T02:30:56.732 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-04-20T02:30:56.732 INFO:tasks.ceph.mon.e:Sent signal 19 2024-04-20T02:30:56.732 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-04-20T02:30:56.732 INFO:tasks.ceph.mon.h:Sent signal 19 2024-04-20T02:30:56.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:30:56.847 INFO:tasks.ceph.mon.g.smithi179.stderr:2024-04-20T02:30:56.848+0000 7f9a60ecacc0 -1 mon.g@-1(???).log v0 log_external_backlog rewinding external_log_to from 541 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:31:11.733 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-04-20T02:31:11.734 INFO:tasks.ceph.mon.e:Sent signal 18 2024-04-20T02:31:11.734 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-04-20T02:31:11.734 INFO:tasks.ceph.mon.h:Sent signal 18 2024-04-20T02:31:11.734 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:31:11.735 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:31:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.910+0000 7fb23f6ce700 10 monclient: get_monmap_and_config 2024-04-20T02:31:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.910+0000 7fb23f6ce700 10 monclient: build_initial_monmap 2024-04-20T02:31:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.910+0000 7fb23f6ce700 10 monclient: monmap: 2024-04-20T02:31:11.910 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:11.911511+0000 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:11.911511+0000 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:11.911 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:11.912 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:11.912 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:11.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.910+0000 7fb23f6ce700 1 Processor -- start 2024-04-20T02:31:11.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 1 -- start start 2024-04-20T02:31:11.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient: init 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient: _add_conns ranks=[5,1,6,3,10,8,4,2,0,9,7] 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb238134b50 0x7fb23806db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): picked mon.noname-h con 0x7fb238134b50 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb238134f30 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 0x7fb23813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): picked mon.noname-d con 0x7fb238138020 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2380751e0 con 0x7fb238134f30 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380a4260 con 0x7fb238134b50 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:11.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb238070170 con 0x7fb238138020 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23f6ce700 10 monclient(hunting): authenticate will time out at 2551.312851s 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.911+0000 7fb23dc6b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 0x7fb23813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 0x7fb23813fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52866/0 (socket says 172.21.15.47:52866) 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 -- 172.21.15.47:0/3344046733 learned_addr learned my addr 172.21.15.47:0/3344046733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): get_auth_request con 0x7fb238138020 auth_method 0 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:11.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 1 --2- 172.21.15.47:0/3344046733 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb238134b50 0x7fb23806db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): get_auth_request con 0x7fb238134b50 auth_method 0 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23d46a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient(hunting): handle_auth_done global_id 55086 payload 274 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 -- 172.21.15.47:0/3344046733 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb238134b50 msgr2=0x7fb23806db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 --2- 172.21.15.47:0/3344046733 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb238134b50 0x7fb23806db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 -- 172.21.15.47:0/3344046733 >> v1:172.21.15.179:6789/0 conn(0x7fb238134f30 legacy=0x7fb238137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient: _finish_hunting 0 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 monclient: found mon.noname-d 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:11.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 -- 172.21.15.47:0/3344046733 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23806e090 con 0x7fb238138020 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23dc6b700 1 --2- 172.21.15.47:0/3344046733 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 0x7fb23813fc90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb230009eb0 tx=0x7fb23000b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c3677c9df0b1f6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 1 -- 172.21.15.47:0/3344046733 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb23000e070 con 0x7fb238138020 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: dump: 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:11.924 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: _finish_auth 0 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 1 -- 172.21.15.47:0/3344046733 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2300099b0 con 0x7fb238138020 2024-04-20T02:31:11.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23f6ce700 5 monclient: authenticate success, global_id 55086 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23f6ce700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23f6ce700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 1 -- 172.21.15.47:0/3344046733 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb23000bd40 con 0x7fb238138020 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb23f6ce700 4 set_mon_vals no callback set 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.912+0000 7fb22ffff700 10 monclient: dump: 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:11.926 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:11.927 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 10 monclient: shutdown 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 -- 172.21.15.47:0/3344046733 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 msgr2=0x7fb23813fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/3344046733 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 0x7fb23813fc90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb230009eb0 tx=0x7fb23000b330 comp rx=0 tx=0).stop 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 -- 172.21.15.47:0/3344046733 shutdown_connections 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/3344046733 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb238138020 0x7fb23813fc90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/3344046733 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb238134b50 0x7fb23806db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 -- 172.21.15.47:0/3344046733 >> 172.21.15.47:0/3344046733 conn(0x7fb2380a5680 msgr2=0x7fb238066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 -- 172.21.15.47:0/3344046733 shutdown_connections 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.913+0000 7fb23f6ce700 1 -- 172.21.15.47:0/3344046733 wait complete. 2024-04-20T02:31:11.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 10 monclient: build_initial_monmap 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 10 monclient: monmap: 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:11.929 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 1 librados: starting msgr at 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 1 librados: starting objecter 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 1 Processor -- start 2024-04-20T02:31:11.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 1 -- start start 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 1 librados: setting wanted keys 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 1 librados: calling monclient init 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.914+0000 7fb23f6ce700 10 monclient: init 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient: _add_conns ranks=[4,8,7,6,2,3,5,1,0] 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 0x7fb2380a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): picked mon.noname-c con 0x7fb238134b50 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb238134f30 0x7fb238133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): picked mon.noname-e con 0x7fb238134f30 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb238138020 0x7fb23811e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): picked mon.noname-i con 0x7fb238138020 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb238120ec0 con 0x7fb238134b50 2024-04-20T02:31:11.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb2381c9e10 con 0x7fb238138020 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb238072920 con 0x7fb238134f30 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23f6ce700 10 monclient(hunting): authenticate will time out at 2551.316226s 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 0x7fb2380a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 0x7fb2380a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43632/0 (socket says 172.21.15.47:43632) 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 -- 172.21.15.47:0/1792023563 learned_addr learned my addr 172.21.15.47:0/1792023563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): get_auth_request con 0x7fb238134b50 auth_method 0 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb238134f30 0x7fb238133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:11.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): get_auth_request con 0x7fb238134f30 auth_method 0 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23dc6b700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb238138020 0x7fb23811e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23dc6b700 10 monclient(hunting): get_auth_request con 0x7fb238138020 auth_method 0 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23dc6b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:11.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23dc6b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23dc6b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient(hunting): handle_auth_done global_id 84316 payload 824 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 -- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb238134f30 msgr2=0x7fb238133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb238134f30 0x7fb238133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 -- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb238138020 msgr2=0x7fb23811e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb238138020 0x7fb23811e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient: _finish_hunting 0 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 monclient: found mon.noname-c 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 -- 172.21.15.47:0/1792023563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230009b30 con 0x7fb238134b50 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23dc6b700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb238138020 0x7fb23811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:11.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23cc69700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb238134f30 0x7fb238133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb23d46a700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 0x7fb2380a17e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb23400e9b0 tx=0x7fb23400ecc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb22dffb700 10 client.?.objecter ms_handle_connect 0x7fb238134b50 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.915+0000 7fb22dffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb234009710 con 0x7fb238134b50 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: dump: 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:11.935 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23400ccc0 con 0x7fb238134b50 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 5 monclient: authenticate success, global_id 84316 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 10 monclient: _renew_subs 2024-04-20T02:31:11.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb238072540 con 0x7fb238134b50 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb23400ce30 con 0x7fb238134b50 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22dffb700 10 monclient: dump: 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:11.937 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb22f7fe700 4 set_mon_vals no callback set 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 10 client.84316.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 10 monclient: _renew_subs 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23811fd20 con 0x7fb238134b50 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.916+0000 7fb23f6ce700 1 librados: init done 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb23f6ce700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:11.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb23f6ce700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb23f6ce700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb238067f90 con 0x7fb238134b50 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb23400c850 con 0x7fb238134b50 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb22dffb700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb21c03a2f0 0x7fb21c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(598..598 src has 266..598) v4 ==== 6646+0+0 (secure 0 0 0) 0x7fb23404aae0 con 0x7fb238134b50 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb22dffb700 10 client.84316.objecter ms_dispatch 0x7fb238001550 osd_map(598..598 src has 266..598) v4 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb22dffb700 3 client.84316.objecter handle_osd_map got epochs [598,598] > 0 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.917+0000 7fb22dffb700 3 client.84316.objecter handle_osd_map decoding full epoch 598 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.918+0000 7fb22dffb700 20 client.84316.objecter dump_active .. 0 homeless 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.918+0000 7fb23cc69700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb21c03a2f0 0x7fb21c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.918+0000 7fb23cc69700 10 monclient: get_auth_request con 0x7fb21c03a2f0 auth_method 0 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.918+0000 7fb22dffb700 10 client.84316.objecter ms_handle_connect 0x7fb21c03a2f0 2024-04-20T02:31:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.918+0000 7fb23cc69700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb21c03a2f0 0x7fb21c03c7b0 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 crypto rx=0x7fb2380a1eb0 tx=0x7fb228009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:11.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.923+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fb23401b3b0 con 0x7fb238134b50 2024-04-20T02:31:11.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.923+0000 7fb22dffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:11.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:11.923+0000 7fb22dffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:12.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.231+0000 7fb23f6ce700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:31:12.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.232+0000 7fb23f6ce700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:12.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.232+0000 7fb23f6ce700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:12.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.232+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb23806c150 con 0x7fb238134b50 2024-04-20T02:31:12.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.232+0000 7fb22dffb700 1 -- 172.21.15.47:0/1792023563 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fb234017080 con 0x7fb238134b50 2024-04-20T02:31:12.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.232+0000 7fb22dffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:12.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.232+0000 7fb22dffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:12.236 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:31:12.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.234+0000 7fb23f6ce700 10 librados: watch_flush enter 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.234+0000 7fb23f6ce700 10 librados: watch_flush exit 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.234+0000 7fb23f6ce700 20 client.84316.objecter shutdown clearing up homeless session... 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.234+0000 7fb23f6ce700 10 client.84316.objecter successfully canceled tick 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb21c03a2f0 msgr2=0x7fb21c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb21c03a2f0 0x7fb21c03c7b0 secure :-1 s=READY pgs=3324 cs=0 l=1 rev1=1 crypto rx=0x7fb2380a1eb0 tx=0x7fb228009040 comp rx=0 tx=0).stop 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 10 monclient: shutdown 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 msgr2=0x7fb2380a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 0x7fb2380a17e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb23400e9b0 tx=0x7fb23400ecc0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 shutdown_connections 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb21c03a2f0 0x7fb21c03c7b0 unknown :-1 s=CLOSED pgs=3324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb238138020 0x7fb23811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb238134b50 0x7fb2380a17e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 --2- 172.21.15.47:0/1792023563 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb238134f30 0x7fb238133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 >> 172.21.15.47:0/1792023563 conn(0x7fb2380a5680 msgr2=0x7fb2381bf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 shutdown_connections 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 -- 172.21.15.47:0/1792023563 wait complete. 2024-04-20T02:31:12.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.235+0000 7fb23f6ce700 1 librados: shutdown 2024-04-20T02:31:12.249 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:31:12.249 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:31:12.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.384+0000 7f950401b700 10 monclient: get_monmap_and_config 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.384+0000 7f950401b700 10 monclient: build_initial_monmap 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.384+0000 7f950401b700 10 monclient: monmap: 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:12.386023+0000 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:12.386023+0000 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:12.385 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:12.386 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.385+0000 7f950401b700 1 Processor -- start 2024-04-20T02:31:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.385+0000 7f950401b700 1 -- start start 2024-04-20T02:31:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.385+0000 7f950401b700 10 monclient: init 2024-04-20T02:31:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:12.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient: _add_conns ranks=[6,7,9,1,0,5,8,2,3,10,4] 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 0x7f94fc0a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): picked mon.noname-d con 0x7f94fc0a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f94fc0a3270 0x7f94fc137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): picked mon.noname-i con 0x7f94fc0a3270 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94fc138040 0x7f94fc1384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): picked mon.noname-a con 0x7f94fc138040 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc072830 con 0x7f94fc138040 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc09f150 con 0x7f94fc0a2950 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc09f710 con 0x7f94fc0a3270 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f950401b700 10 monclient(hunting): authenticate will time out at 2551.787732s 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 0x7f94fc0a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f95025b8700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94fc138040 0x7f94fc1384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 0x7f94fc0a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52874/0 (socket says 172.21.15.47:52874) 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 1 -- 172.21.15.47:0/2595348491 learned_addr learned my addr 172.21.15.47:0/2595348491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 10 monclient(hunting): get_auth_request con 0x7f94fc0a2950 auth_method 0 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f95025b8700 10 monclient(hunting): get_auth_request con 0x7f94fc138040 auth_method 0 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f95025b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f95025b8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f95025b8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.386+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95015b6700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f94fc0a3270 0x7f94fc137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 10 monclient(hunting): handle_auth_done global_id 36612 payload 274 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 -- 172.21.15.47:0/2595348491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f94fc0a3270 msgr2=0x7f94fc137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f94fc0a3270 0x7f94fc137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 -- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 msgr2=0x7f94fc0a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 0x7f94fc0a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 10 monclient: _finish_hunting 0 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 monclient: found mon.noname-a 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 -- 172.21.15.47:0/2595348491 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0751e0 con 0x7f94fc138040 2024-04-20T02:31:12.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95015b6700 10 monclient: get_auth_request con 0x7f94fc0a3270 auth_method 0 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9501db7700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 0x7f94fc0a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95015b6700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f94fc0a3270 0x7f94fc137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f95025b8700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc138040 0x7f94fc1384f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009d90 tx=0x7f94f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82641c7c6b860122 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 1 -- 172.21.15.47:0/2595348491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f400e070 con 0x7f94fc138040 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: dump: 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.401 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: _finish_auth 0 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: _check_auth_tickets 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 1 -- 172.21.15.47:0/2595348491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4012430 con 0x7f94fc138040 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f950401b700 5 monclient: authenticate success, global_id 36612 2024-04-20T02:31:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 1 -- 172.21.15.47:0/2595348491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f4012a10 con 0x7f94fc138040 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f9500db5700 10 monclient: dump: 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.403 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f950401b700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.387+0000 7f950401b700 4 set_mon_vals no callback set 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 10 monclient: shutdown 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 -- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc138040 msgr2=0x7f94fc1384f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc138040 0x7f94fc1384f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009d90 tx=0x7f94f400b330 comp rx=0 tx=0).stop 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 -- 172.21.15.47:0/2595348491 shutdown_connections 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc138040 0x7f94fc1384f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f94fc0a2950 0x7f94fc0a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 --2- 172.21.15.47:0/2595348491 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f94fc0a3270 0x7f94fc137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 -- 172.21.15.47:0/2595348491 >> 172.21.15.47:0/2595348491 conn(0x7f94fc0a7570 msgr2=0x7f94fc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 -- 172.21.15.47:0/2595348491 shutdown_connections 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 1 -- 172.21.15.47:0/2595348491 wait complete. 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 10 monclient: build_initial_monmap 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.388+0000 7f950401b700 10 monclient: monmap: 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:12.405 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 librados: starting msgr at 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 librados: starting objecter 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 Processor -- start 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 -- start start 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 librados: setting wanted keys 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 librados: calling monclient init 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient: init 2024-04-20T02:31:12.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient: _add_conns ranks=[3,4,5,6,2,7,1,8,0] 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f94fc0a2950 0x7f94fc0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): picked mon.noname-g con 0x7f94fc0a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94fc0a3270 0x7f94fc1370c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): picked mon.noname-c con 0x7f94fc0a3270 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f94fc138040 0x7f94fc137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): picked mon.noname-h con 0x7f94fc138040 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1248a0 con 0x7f94fc0a2950 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc09d960 con 0x7f94fc0a3270 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1241e0 con 0x7f94fc138040 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.389+0000 7f950401b700 10 monclient(hunting): authenticate will time out at 2551.790875s 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94fc0a3270 0x7f94fc1370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94fc0a3270 0x7f94fc1370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:43638/0 (socket says 172.21.15.47:43638) 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 -- 172.21.15.47:0/3089777463 learned_addr learned my addr 172.21.15.47:0/3089777463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): get_auth_request con 0x7f94fc0a3270 auth_method 0 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f94fc138040 0x7f94fc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): get_auth_request con 0x7f94fc138040 auth_method 0 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f94fc0a2950 0x7f94fc0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): get_auth_request con 0x7f94fc0a2950 auth_method 0 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95025b8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient(hunting): handle_auth_done global_id 84325 payload 824 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f94fc138040 msgr2=0x7f94fc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f94fc138040 0x7f94fc137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f94fc0a2950 msgr2=0x7f94fc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f94fc0a2950 0x7f94fc0715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient: _finish_hunting 0 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 monclient: found mon.noname-c 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:12.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8009550 con 0x7f94fc0a3270 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f95015b6700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94fc0a3270 0x7f94fc1370c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00c770 tx=0x7f94ec00ca80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 client.?.objecter ms_handle_connect 0x7f94fc0a3270 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94ec011a70 con 0x7f94fc0a3270 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: dump: 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.411 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec011c30 con 0x7f94fc0a3270 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 5 monclient: authenticate success, global_id 84325 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 10 monclient: _renew_subs 2024-04-20T02:31:12.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f4009a70 con 0x7f94fc0a3270 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94ec011da0 con 0x7f94fc0a3270 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f3fff700 4 set_mon_vals no callback set 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f94f27fc700 10 monclient: dump: 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.413 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 10 client.84325.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 10 monclient: _renew_subs 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:12.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc0721d0 con 0x7f94fc0a3270 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.390+0000 7f950401b700 1 librados: init done 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f94ec011600 con 0x7f94fc0a3270 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f950401b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f950401b700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 0x7f94fc1219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc068310 con 0x7f94fc06aac0 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 0x7f94fc1219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient: get_auth_request con 0x7f94fc06aac0 auth_method 0 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f94f27fc700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f94e003a2f0 0x7f94e003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.391+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(598..598 src has 266..598) v4 ==== 6646+0+0 (secure 0 0 0) 0x7f94ec04a4d0 con 0x7f94fc0a3270 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 client.84325.objecter ms_dispatch 0x7f94fc001550 osd_map(598..598 src has 266..598) v4 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 3 client.84325.objecter handle_osd_map got epochs [598,598] > 0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 3 client.84325.objecter handle_osd_map decoding full epoch 598 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f9501db7700 10 monclient(hunting): handle_auth_done global_id 36621 payload 824 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f95025b8700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f94e003a2f0 0x7f94e003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 20 client.84325.objecter dump_active .. 0 homeless 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f9501db7700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94fc09ee70 con 0x7f94fc06aac0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f95025b8700 10 monclient: get_auth_request con 0x7f94e003a2f0 auth_method 0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 client.84325.objecter ms_handle_connect 0x7f94fc06aac0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f9501db7700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 0x7f94fc1219a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94f8011bd0 tx=0x7f94f8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 client.84325.objecter resend_mon_ops 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f800fdf0 con 0x7f94fc06aac0 2024-04-20T02:31:12.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 monclient: dump: 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.417 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 client.84325.objecter ms_handle_connect 0x7f94e003a2f0 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f95025b8700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f94e003a2f0 0x7f94e003c7b0 secure :-1 s=READY pgs=3325 cs=0 l=1 rev1=1 crypto rx=0x7f94f4014040 tx=0x7f94f4011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f94f800f980 con 0x7f94fc06aac0 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 msgr2=0x7f94fc1219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 0x7f94fc1219a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94f8011bd0 tx=0x7f94f8011f90 comp rx=0 tx=0).stop 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 msgr2=0x7f94fc1219a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:12.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.392+0000 7f94f27fc700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc06aac0 0x7f94fc1219a0 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f94f8011bd0 tx=0x7f94f8011f90 comp rx=0 tx=0).stop 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f950401b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f950401b700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 0x7f94fc1222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f950401b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc06a400 con 0x7f94fc121ee0 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 0x7f94fc1222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient: get_auth_request con 0x7f94fc121ee0 auth_method 0 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.400+0000 7f9501db7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f9501db7700 10 monclient(hunting): handle_auth_done global_id 36630 payload 824 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f9501db7700 1 -- 172.21.15.47:0/3089777463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f94fc067730 con 0x7f94fc121ee0 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 client.84325.objecter ms_handle_connect 0x7f94fc121ee0 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f9501db7700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 0x7f94fc1222f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94f8011bd0 tx=0x7f94f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 client.84325.objecter resend_mon_ops 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94f80148b0 con 0x7f94fc121ee0 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 monclient: dump: 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.423 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f94f8014440 con 0x7f94fc121ee0 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 msgr2=0x7f94fc1222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 0x7f94fc1222f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94f8011bd0 tx=0x7f94f8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 msgr2=0x7f94fc1222f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f94f27fc700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94fc121ee0 0x7f94fc1222f0 secure :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f94f8011bd0 tx=0x7f94f8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:12.424 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:12.425 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.427 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:12.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.429 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:12.431 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.433 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.435 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:12.438 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:12.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:12.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:12.445 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.446 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.447 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:12.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:12.451 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.401+0000 7f950401b700 10 librados: watch_flush enter 2024-04-20T02:31:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 10 librados: watch_flush exit 2024-04-20T02:31:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 20 client.84325.objecter shutdown clearing up homeless session... 2024-04-20T02:31:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 10 client.84325.objecter successfully canceled tick 2024-04-20T02:31:12.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f94e003a2f0 msgr2=0x7f94e003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f94e003a2f0 0x7f94e003c7b0 secure :-1 s=READY pgs=3325 cs=0 l=1 rev1=1 crypto rx=0x7f94f4014040 tx=0x7f94f4011000 comp rx=0 tx=0).stop 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 10 monclient: shutdown 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94fc0a3270 msgr2=0x7f94fc1370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 --2- 172.21.15.47:0/3089777463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94fc0a3270 0x7f94fc1370c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00c770 tx=0x7f94ec00ca80 comp rx=0 tx=0).stop 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f95025b8700 1 -- 172.21.15.47:0/3089777463 reap_dead start 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f95025b8700 1 -- 172.21.15.47:0/3089777463 reap_dead start 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 shutdown_connections 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 >> 172.21.15.47:0/3089777463 conn(0x7f94fc0a7570 msgr2=0x7f94fc0a4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 shutdown_connections 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 -- 172.21.15.47:0/3089777463 wait complete. 2024-04-20T02:31:12.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.402+0000 7f950401b700 1 librados: shutdown 2024-04-20T02:31:12.455 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:12.455 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:31:12.455 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.594+0000 7ff8898fe700 10 monclient: get_monmap_and_config 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.594+0000 7ff8898fe700 10 monclient: build_initial_monmap 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.594+0000 7ff8898fe700 10 monclient: monmap: 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:12.595279+0000 2024-04-20T02:31:12.594 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:12.595279+0000 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:12.595 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:12.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.594+0000 7ff8898fe700 1 Processor -- start 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.594+0000 7ff8898fe700 1 -- start start 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.594+0000 7ff8898fe700 10 monclient: init 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient: _add_conns ranks=[2,0,3,4,5,8,7,10,1,6,9] 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8840a07f0 0x7ff8840a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): picked mon.noname-b con 0x7ff8840a07f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff8840a0bd0 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a1050 0x7ff8840a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): picked mon.noname-g con 0x7ff8840a1050 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840751e0 con 0x7ff8840a0bd0 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff884067800 con 0x7ff8840a07f0 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840672f0 con 0x7ff8840a1050 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8898fe700 10 monclient(hunting): authenticate will time out at 2551.996425s 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8840a07f0 0x7ff8840a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8827fc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff8840a0bd0 0x7ff8840a2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39128/0 (socket says 172.21.15.47:39128) 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8827fc700 1 -- 172.21.15.47:0/40078619 learned_addr learned my addr 172.21.15.47:0/40078619 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): get_auth_request con 0x7ff8840a07f0 auth_method 0 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 1 --2- 172.21.15.47:0/40078619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a1050 0x7ff8840a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): get_auth_request con 0x7ff8840a1050 auth_method 0 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff8837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff881ffb700 1 -- 172.21.15.47:0/40078619 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899102778 0 0) 0x7ff8840751e0 con 0x7ff8840a0bd0 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient(hunting): handle_auth_done global_id 24671 payload 274 2024-04-20T02:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 1 -- 172.21.15.47:0/40078619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a1050 msgr2=0x7ff8840a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 1 --2- 172.21.15.47:0/40078619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a1050 0x7ff8840a2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 1 -- 172.21.15.47:0/40078619 >> v1:172.21.15.47:6789/0 conn(0x7ff8840a0bd0 legacy=0x7ff8840a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 1 monclient: found mon.noname-b 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff882ffd700 1 -- 172.21.15.47:0/40078619 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88409ed90 con 0x7ff8840a07f0 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.595+0000 7ff881ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff882ffd700 1 --2- 172.21.15.47:0/40078619 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8840a07f0 0x7ff8840a0010 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff8740098a0 tx=0x7ff874009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3600519bc5db25c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 1 -- 172.21.15.47:0/40078619 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff874011aa0 con 0x7ff8840a07f0 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: dump: 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.611 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 1 -- 172.21.15.47:0/40078619 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874011c00 con 0x7ff8840a07f0 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 5 monclient: authenticate success, global_id 24671 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 1 -- 172.21.15.47:0/40078619 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff874011d70 con 0x7ff8840a07f0 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 4 set_mon_vals no callback set 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff881ffb700 10 monclient: dump: 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.613 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 10 monclient: shutdown 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 1 -- 172.21.15.47:0/40078619 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8840a07f0 msgr2=0x7ff8840a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 1 --2- 172.21.15.47:0/40078619 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8840a07f0 0x7ff8840a0010 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff8740098a0 tx=0x7ff874009bb0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 1 -- 172.21.15.47:0/40078619 shutdown_connections 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 1 --2- 172.21.15.47:0/40078619 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a1050 0x7ff8840a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.596+0000 7ff8898fe700 1 --2- 172.21.15.47:0/40078619 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8840a07f0 0x7ff8840a0010 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 1 -- 172.21.15.47:0/40078619 >> 172.21.15.47:0/40078619 conn(0x7ff8840a3f90 msgr2=0x7ff88406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 1 -- 172.21.15.47:0/40078619 shutdown_connections 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 1 -- 172.21.15.47:0/40078619 wait complete. 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 10 monclient: build_initial_monmap 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 10 monclient: monmap: 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:12.615 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 1 librados: starting msgr at 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.597+0000 7ff8898fe700 1 librados: starting objecter 2024-04-20T02:31:12.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 Processor -- start 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 -- start start 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 librados: setting wanted keys 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 librados: calling monclient init 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient: init 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient: _add_conns ranks=[3,7,1,0,2,8,4,6,5] 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a07f0 0x7ff8841c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): picked mon.noname-g con 0x7ff8840a07f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff8840a0bd0 0x7ff8841bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): picked mon.noname-i con 0x7ff8840a0bd0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8840a1050 0x7ff8841cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): picked mon.noname-f con 0x7ff8840a1050 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:12.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841aa530 con 0x7ff8840a1050 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841aa250 con 0x7ff8840a07f0 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff8841a9c90 con 0x7ff8840a0bd0 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.598+0000 7ff8898fe700 10 monclient(hunting): authenticate will time out at 2551.999957s 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8840a1050 0x7ff8841cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a07f0 0x7ff8841c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff8840a0bd0 0x7ff8841bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a07f0 0x7ff8841c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:55218/0 (socket says 172.21.15.47:55218) 2024-04-20T02:31:12.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8840a1050 0x7ff8841cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:60394/0 (socket says 172.21.15.47:60394) 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 -- 172.21.15.47:0/4294447159 learned_addr learned my addr 172.21.15.47:0/4294447159 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): get_auth_request con 0x7ff8840a07f0 auth_method 0 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): get_auth_request con 0x7ff8840a0bd0 auth_method 0 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): get_auth_request con 0x7ff8840a1050 auth_method 0 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff8837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient(hunting): handle_auth_done global_id 94125 payload 824 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff8840a0bd0 msgr2=0x7ff8841bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff8840a0bd0 0x7ff8841bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8840a1050 msgr2=0x7ff8841cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8840a1050 0x7ff8841cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 monclient: found mon.noname-g 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740094a0 con 0x7ff8840a07f0 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff882ffd700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a07f0 0x7ff8841c8cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff874009790 tx=0x7ff874004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff86bfff700 10 client.?.objecter ms_handle_connect 0x7ff8840a07f0 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.599+0000 7ff86bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff874011a20 con 0x7ff8840a07f0 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:31:12.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: dump: 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.622 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: _finish_auth 0 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87401f070 con 0x7ff8840a07f0 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 5 monclient: authenticate success, global_id 94125 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 monclient: _renew_subs 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8841a9f70 con 0x7ff8840a07f0 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff874004470 con 0x7ff8840a07f0 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff86bfff700 10 monclient: dump: 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.623 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8817fa700 4 set_mon_vals no callback set 2024-04-20T02:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 client.94125.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 monclient: _renew_subs 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8841bd4e0 con 0x7ff8840a07f0 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 1 librados: init done 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 0x7ff8841cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.600+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff884068280 con 0x7ff8841bd0b0 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 0x7ff8841cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff874021030 con 0x7ff8840a07f0 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient: get_auth_request con 0x7ff8841bd0b0 auth_method 0 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff87003a2f0 0x7ff87003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 10 monclient(hunting): handle_auth_done global_id 24680 payload 824 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8841bd6d0 con 0x7ff8841bd0b0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(598..598 src has 266..598) v4 ==== 6646+0+0 (secure 0 0 0) 0x7ff87404bb40 con 0x7ff8840a07f0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 10 client.94125.objecter ms_dispatch 0x7ff884001550 osd_map(598..598 src has 266..598) v4 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 3 client.94125.objecter handle_osd_map got epochs [598,598] > 0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 3 client.94125.objecter handle_osd_map decoding full epoch 598 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 20 client.94125.objecter dump_active .. 0 homeless 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8827fc700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 0x7ff8841cc700 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7ff8841a7d60 tx=0x7ff8780018f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 10 client.94125.objecter ms_handle_connect 0x7ff8841bd0b0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 10 client.94125.objecter resend_mon_ops 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8837fe700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff87003a2f0 0x7ff87003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff8837fe700 10 monclient: get_auth_request con 0x7ff87003a2f0 auth_method 0 2024-04-20T02:31:12.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff878017070 con 0x7ff8841bd0b0 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.601+0000 7ff86bfff700 10 monclient: dump: 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff878012560 con 0x7ff8841bd0b0 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 msgr2=0x7ff8841cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 0x7ff8841cc700 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7ff8841a7d60 tx=0x7ff8780018f0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 msgr2=0x7ff8841cc700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff8841bd0b0 0x7ff8841cc700 secure :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7ff8841a7d60 tx=0x7ff8780018f0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff86bfff700 10 client.94125.objecter ms_handle_connect 0x7ff87003a2f0 2024-04-20T02:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.602+0000 7ff8837fe700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff87003a2f0 0x7ff87003c7b0 secure :-1 s=READY pgs=3326 cs=0 l=1 rev1=1 crypto rx=0x7ff8841cd280 tx=0x7ff87c00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.609+0000 7ff8898fe700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:12.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.609+0000 7ff8898fe700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 0x7ff8841ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.609+0000 7ff8898fe700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.609+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88406a400 con 0x7ff88406aac0 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 0x7ff8841ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient: get_auth_request con 0x7ff88406aac0 auth_method 0 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 10 monclient(hunting): handle_auth_done global_id 24689 payload 824 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 1 -- 172.21.15.47:0/4294447159 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff88409f350 con 0x7ff88406aac0 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 client.94125.objecter ms_handle_connect 0x7ff88406aac0 2024-04-20T02:31:12.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff8827fc700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 0x7ff8841ccc40 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff8841c9490 tx=0x7ff878000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 client.94125.objecter resend_mon_ops 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff878023070 con 0x7ff88406aac0 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 monclient: dump: 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.632 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7ff878012e90 con 0x7ff88406aac0 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 msgr2=0x7ff8841ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 0x7ff8841ccc40 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff8841c9490 tx=0x7ff878000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 msgr2=0x7ff8841ccc40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:12.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.610+0000 7ff86bfff700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff88406aac0 0x7ff8841ccc40 secure :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff8841c9490 tx=0x7ff878000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:12.634 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.635 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:12.637 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.638 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.639 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:12.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:12.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:12.649 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.651 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.652 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:12.654 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.655 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.658 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.659 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.660 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:12.660 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 10 librados: watch_flush enter 2024-04-20T02:31:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 10 librados: watch_flush exit 2024-04-20T02:31:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 20 client.94125.objecter shutdown clearing up homeless session... 2024-04-20T02:31:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 10 client.94125.objecter successfully canceled tick 2024-04-20T02:31:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff87003a2f0 msgr2=0x7ff87003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff87003a2f0 0x7ff87003c7b0 secure :-1 s=READY pgs=3326 cs=0 l=1 rev1=1 crypto rx=0x7ff8841cd280 tx=0x7ff87c00b040 comp rx=0 tx=0).stop 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 10 monclient: shutdown 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a07f0 msgr2=0x7ff8841c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 --2- 172.21.15.47:0/4294447159 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff8840a07f0 0x7ff8841c8cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff874009790 tx=0x7ff874004750 comp rx=0 tx=0).stop 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8837fe700 1 -- 172.21.15.47:0/4294447159 reap_dead start 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8837fe700 1 -- 172.21.15.47:0/4294447159 reap_dead start 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 shutdown_connections 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 >> 172.21.15.47:0/4294447159 conn(0x7ff8840a3f90 msgr2=0x7ff8840a26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 shutdown_connections 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 -- 172.21.15.47:0/4294447159 wait complete. 2024-04-20T02:31:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.611+0000 7ff8898fe700 1 librados: shutdown 2024-04-20T02:31:12.664 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:12.664 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:31:12.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:31:12.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.803+0000 7f341bfff700 10 monclient: get_monmap_and_config 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.803+0000 7f341bfff700 10 monclient: build_initial_monmap 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.803+0000 7f341bfff700 10 monclient: monmap: 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:12.804719+0000 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:12.804719+0000 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:12.804 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 Processor -- start 2024-04-20T02:31:12.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 -- start start 2024-04-20T02:31:12.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient: init 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient: _add_conns ranks=[8,5,4,6,10,2,0,1,7,3,9] 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 0x7f341c09f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): picked mon.noname-e con 0x7f341c0a1070 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f341c0707a0 0x7f341c0a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): picked mon.noname-h con 0x7f341c0707a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a1990 0x7f341c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f341c0a1990 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:12.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341c071c50 con 0x7f341c0a1990 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341c071f30 con 0x7f341c0707a0 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0a52f0 con 0x7f341c0a1070 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341bfff700 10 monclient(hunting): authenticate will time out at 2552.205764s 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 0x7f341c09f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a1990 0x7f341c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 0x7f341c09f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:38388/0 (socket says 172.21.15.47:38388) 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341b7fe700 10 monclient(hunting): get_auth_request con 0x7f341c0a1990 auth_method 0 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 1 -- 172.21.15.47:0/2380821679 learned_addr learned my addr 172.21.15.47:0/2380821679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:12.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 10 monclient(hunting): get_auth_request con 0x7f341c0a1070 auth_method 0 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.804+0000 7f341affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341a7fc700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f341c0707a0 0x7f341c0a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341a7fc700 10 monclient(hunting): get_auth_request con 0x7f341c0707a0 auth_method 0 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 10 monclient(hunting): handle_auth_done global_id 35882 payload 274 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 -- 172.21.15.47:0/2380821679 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f341c0707a0 msgr2=0x7f341c0a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f341c0707a0 0x7f341c0a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 -- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a1990 msgr2=0x7f341c0a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a1990 0x7f341c0a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 monclient: found mon.noname-e 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 -- 172.21.15.47:0/2380821679 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0751e0 con 0x7f341c0a1070 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341b7fe700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a1990 0x7f341c0a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341a7fc700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f341c0707a0 0x7f341c0a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341affd700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 0x7f341c09f780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f340c009920 tx=0x7f340c009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=611c65dd1a5734bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 1 -- 172.21.15.47:0/2380821679 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340c011aa0 con 0x7f341c0a1070 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: dump: 2024-04-20T02:31:12.820 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 1 -- 172.21.15.47:0/2380821679 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c011c00 con 0x7f341c0a1070 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341bfff700 5 monclient: authenticate success, global_id 35882 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 1 -- 172.21.15.47:0/2380821679 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340c011d70 con 0x7f341c0a1070 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341bfff700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f341bfff700 4 set_mon_vals no callback set 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.805+0000 7f3419ffb700 10 monclient: dump: 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.822 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 10 monclient: shutdown 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 -- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 msgr2=0x7f341c09f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 0x7f341c09f780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f340c009920 tx=0x7f340c009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 -- 172.21.15.47:0/2380821679 shutdown_connections 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a1990 0x7f341c0a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f341c0a1070 0x7f341c09f780 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 --2- 172.21.15.47:0/2380821679 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f341c0707a0 0x7f341c0a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 -- 172.21.15.47:0/2380821679 >> 172.21.15.47:0/2380821679 conn(0x7f341c0a8450 msgr2=0x7f341c0a4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 -- 172.21.15.47:0/2380821679 shutdown_connections 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 1 -- 172.21.15.47:0/2380821679 wait complete. 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.806+0000 7f341bfff700 10 monclient: build_initial_monmap 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 10 monclient: monmap: 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:12.824 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 1 librados: starting msgr at 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 1 librados: starting objecter 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 1 Processor -- start 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 1 -- start start 2024-04-20T02:31:12.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 1 librados: setting wanted keys 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 1 librados: calling monclient init 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.807+0000 7f341bfff700 10 monclient: init 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient: _add_conns ranks=[1,6,0,3,5,8,2,7,4] 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c0707a0 0x7f341c1358d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): picked mon.noname-f con 0x7f341c0707a0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f341c0a1070 0x7f341c135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): picked mon.noname-d con 0x7f341c0a1070 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341c0a1990 0x7f341c1c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): picked mon.noname-a con 0x7f341c0a1990 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c123290 con 0x7f341c0a1990 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c120f30 con 0x7f341c0707a0 2024-04-20T02:31:12.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f341c070ff0 con 0x7f341c0a1070 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341bfff700 10 monclient(hunting): authenticate will time out at 2552.209263s 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f341c0a1070 0x7f341c135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341c0a1990 0x7f341c1c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341c0a1990 0x7f341c1c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39708/0 (socket says 172.21.15.47:39708) 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f341c0a1070 0x7f341c135e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:52886/0 (socket says 172.21.15.47:52886) 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 -- 172.21.15.47:0/2296936168 learned_addr learned my addr 172.21.15.47:0/2296936168 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): get_auth_request con 0x7f341c0a1990 auth_method 0 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): get_auth_request con 0x7f341c0a1070 auth_method 0 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c0707a0 0x7f341c1358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): get_auth_request con 0x7f341c0707a0 auth_method 0 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient(hunting): handle_auth_done global_id 36648 payload 824 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f341c0a1070 msgr2=0x7f341c135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f341c0a1070 0x7f341c135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c0707a0 msgr2=0x7f341c1358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f341c0707a0 0x7f341c1358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 monclient: found mon.noname-a 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414009a70 con 0x7f341c0a1990 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341a7fc700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f341c0a1070 0x7f341c135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:12.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f341b7fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341c0a1990 0x7f341c1c9e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f341400ef90 tx=0x7f341400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f34037fe700 10 client.?.objecter ms_handle_connect 0x7f341c0a1990 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.808+0000 7f34037fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f341400cea0 con 0x7f341c0a1990 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: dump: 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.830 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341400f040 con 0x7f341c0a1990 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:12.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f34140148f0 con 0x7f341c0a1990 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 5 monclient: authenticate success, global_id 36648 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 10 monclient: _renew_subs 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c009550 con 0x7f341c0a1990 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34197fa700 4 set_mon_vals no callback set 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f34037fe700 10 monclient: dump: 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.832 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 10 client.36648.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:12.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 10 monclient: _renew_subs 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c121d20 con 0x7f341c0a1990 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 1 librados: init done 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.809+0000 7f341bfff700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 0x7f341c1ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341c06a900 con 0x7f341c1229f0 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 0x7f341c1ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient: get_auth_request con 0x7f341c1229f0 auth_method 0 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3414014480 con 0x7f341c0a1990 2024-04-20T02:31:12.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 10 monclient(hunting): handle_auth_done global_id 84343 payload 824 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f341c06acc0 con 0x7f341c1229f0 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f340803a2f0 0x7f340803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(599..599 src has 266..599) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f341404b490 con 0x7f341c0a1990 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 10 client.36648.objecter ms_dispatch 0x7f341c001550 osd_map(599..599 src has 266..599) v4 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 3 client.36648.objecter handle_osd_map got epochs [599,599] > 0 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 3 client.36648.objecter handle_osd_map decoding full epoch 599 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341affd700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 0x7f341c1ca340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f340c009920 tx=0x7f340c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 20 client.36648.objecter dump_active .. 0 homeless 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 10 client.36648.objecter ms_handle_connect 0x7f341c1229f0 2024-04-20T02:31:12.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 10 client.36648.objecter resend_mon_ops 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341a7fc700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f340803a2f0 0x7f340803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f341a7fc700 10 monclient: get_auth_request con 0x7f340803a2f0 auth_method 0 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340c020070 con 0x7f341c1229f0 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.810+0000 7f34037fe700 10 monclient: dump: 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.836 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f340c0044f0 con 0x7f341c1229f0 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:12.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 msgr2=0x7f341c1ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 0x7f341c1ca340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f340c009920 tx=0x7f340c004750 comp rx=0 tx=0).stop 2024-04-20T02:31:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 msgr2=0x7f341c1ca340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c1229f0 0x7f341c1ca340 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f340c009920 tx=0x7f340c004750 comp rx=0 tx=0).stop 2024-04-20T02:31:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f341a7fc700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f340803a2f0 0x7f340803c7b0 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 crypto rx=0x7f341c11f340 tx=0x7f3410008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.811+0000 7f34037fe700 10 client.36648.objecter ms_handle_connect 0x7f340803a2f0 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341bfff700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 0x7f341c1ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341bfff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341c06a400 con 0x7f341c0a3be0 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 0x7f341c1ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient: get_auth_request con 0x7f341c0a3be0 auth_method 0 2024-04-20T02:31:12.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 10 monclient(hunting): handle_auth_done global_id 84352 payload 824 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 1 -- 172.21.15.47:0/2296936168 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f341c121680 con 0x7f341c0a3be0 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f34037fe700 10 client.36648.objecter ms_handle_connect 0x7f341c0a3be0 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f34037fe700 10 client.36648.objecter resend_mon_ops 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f341affd700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 0x7f341c1ca880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f340c009c70 tx=0x7f340c011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f340c01b8a0 con 0x7f341c0a3be0 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f34037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f34037fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.819+0000 7f34037fe700 10 monclient: dump: 2024-04-20T02:31:12.841 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:12.842 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f340c01b430 con 0x7f341c0a3be0 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 msgr2=0x7f341c1ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 0x7f341c1ca880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f340c009c70 tx=0x7f340c011000 comp rx=0 tx=0).stop 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 msgr2=0x7f341c1ca880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:12.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f34037fe700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f341c0a3be0 0x7f341c1ca880 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f340c009c70 tx=0x7f340c011000 comp rx=0 tx=0).stop 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:12.844 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.845 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.846 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:12.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:12.848 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:12.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.852 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.854 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:12.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.857 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:12.858 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.859 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.860 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.861 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:12.862 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:12.863 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:12.864 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.865 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:12.866 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 10 librados: watch_flush enter 2024-04-20T02:31:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 10 librados: watch_flush exit 2024-04-20T02:31:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 20 client.36648.objecter shutdown clearing up homeless session... 2024-04-20T02:31:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 10 client.36648.objecter successfully canceled tick 2024-04-20T02:31:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f340803a2f0 msgr2=0x7f340803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f340803a2f0 0x7f340803c7b0 secure :-1 s=READY pgs=3327 cs=0 l=1 rev1=1 crypto rx=0x7f341c11f340 tx=0x7f3410008040 comp rx=0 tx=0).stop 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 10 monclient: shutdown 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341c0a1990 msgr2=0x7f341c1c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 --2- 172.21.15.47:0/2296936168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f341c0a1990 0x7f341c1c9e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f341400ef90 tx=0x7f341400c3a0 comp rx=0 tx=0).stop 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341b7fe700 1 -- 172.21.15.47:0/2296936168 reap_dead start 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341b7fe700 1 -- 172.21.15.47:0/2296936168 reap_dead start 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 shutdown_connections 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 >> 172.21.15.47:0/2296936168 conn(0x7f341c0a8450 msgr2=0x7f341c1bd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 shutdown_connections 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 -- 172.21.15.47:0/2296936168 wait complete. 2024-04-20T02:31:12.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:12.820+0000 7f341bfff700 1 librados: shutdown 2024-04-20T02:31:12.869 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:12.870 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:31:12.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:31:13.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.008+0000 7fe637de4700 10 monclient: get_monmap_and_config 2024-04-20T02:31:13.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.008+0000 7fe637de4700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.008+0000 7fe637de4700 10 monclient: monmap: 2024-04-20T02:31:13.008 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.008 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:13.009636+0000 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:13.009636+0000 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:13.009 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 Processor -- start 2024-04-20T02:31:13.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 -- start start 2024-04-20T02:31:13.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient: init 2024-04-20T02:31:13.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient: _add_conns ranks=[10,7,2,3,4,0,5,1,9,8,6] 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe630071630 0x7fe630071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): picked mon.noname-f con 0x7fe630071630 addr v2:172.21.15.179:3300/0 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe630071f50 0x7fe6300a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): picked mon.noname-i con 0x7fe630071f50 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 0x7fe6300a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): picked mon.noname-b con 0x7fe6300a2e90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe630072720 con 0x7fe630071630 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300a0820 con 0x7fe6300a2e90 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe63009e890 con 0x7fe630071f50 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.009+0000 7fe637de4700 10 monclient(hunting): authenticate will time out at 2552.410990s 2024-04-20T02:31:13.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 0x7fe6300a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 0x7fe6300a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:46896/0 (socket says 172.21.15.47:46896) 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 -- 172.21.15.47:0/3370116042 learned_addr learned my addr 172.21.15.47:0/3370116042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): get_auth_request con 0x7fe6300a2e90 auth_method 0 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient(hunting): handle_auth_done global_id 24698 payload 274 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 -- 172.21.15.47:0/3370116042 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe630071f50 msgr2=0x7fe6300a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 --2- 172.21.15.47:0/3370116042 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe630071f50 0x7fe6300a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 -- 172.21.15.47:0/3370116042 >> v2:172.21.15.179:3300/0 conn(0x7fe630071630 msgr2=0x7fe630071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 --2- 172.21.15.47:0/3370116042 >> v2:172.21.15.179:3300/0 conn(0x7fe630071630 0x7fe630071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 monclient: found mon.noname-b 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 -- 172.21.15.47:0/3370116042 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6300751e0 con 0x7fe6300a2e90 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe636381700 1 --2- 172.21.15.47:0/3370116042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 0x7fe6300a3340 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe628009e50 tx=0x7fe62800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57727c22dc9f2b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 1 -- 172.21.15.47:0/3370116042 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe62800e070 con 0x7fe6300a2e90 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: dump: 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.023 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 1 -- 172.21.15.47:0/3370116042 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6280099b0 con 0x7fe6300a2e90 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe637de4700 5 monclient: authenticate success, global_id 24698 2024-04-20T02:31:13.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe637de4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 1 -- 172.21.15.47:0/3370116042 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe62800be30 con 0x7fe6300a2e90 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe637de4700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe637de4700 4 set_mon_vals no callback set 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.010+0000 7fe634b7e700 10 monclient: dump: 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.025 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 10 monclient: shutdown 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 -- 172.21.15.47:0/3370116042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 msgr2=0x7fe6300a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 --2- 172.21.15.47:0/3370116042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 0x7fe6300a3340 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe628009e50 tx=0x7fe62800b330 comp rx=0 tx=0).stop 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 -- 172.21.15.47:0/3370116042 shutdown_connections 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 --2- 172.21.15.47:0/3370116042 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe6300a2e90 0x7fe6300a3340 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 --2- 172.21.15.47:0/3370116042 >> v2:172.21.15.179:3300/0 conn(0x7fe630071630 0x7fe630071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 --2- 172.21.15.47:0/3370116042 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fe630071f50 0x7fe6300a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 -- 172.21.15.47:0/3370116042 >> 172.21.15.47:0/3370116042 conn(0x7fe6300a7570 msgr2=0x7fe6300a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 -- 172.21.15.47:0/3370116042 shutdown_connections 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.011+0000 7fe637de4700 1 -- 172.21.15.47:0/3370116042 wait complete. 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 10 monclient: monmap: 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.027 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 1 librados: starting msgr at 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 1 librados: starting objecter 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 1 Processor -- start 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 1 -- start start 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 1 librados: setting wanted keys 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 1 librados: calling monclient init 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.012+0000 7fe637de4700 10 monclient: init 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient: _add_conns ranks=[8,0,1,2,4,7,6,3,5] 2024-04-20T02:31:13.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 0x7fe6301c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): picked mon.noname-e con 0x7fe630071630 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630071f50 0x7fe6301bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): picked mon.noname-a con 0x7fe630071f50 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6300a2e90 0x7fe6301bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): picked mon.noname-f con 0x7fe6300a2e90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6301aa6c0 con 0x7fe630071f50 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63009f200 con 0x7fe6300a2e90 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe630072380 con 0x7fe630071630 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe637de4700 10 monclient(hunting): authenticate will time out at 2552.414359s 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 0x7fe6301c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630071f50 0x7fe6301bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 0x7fe6301c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:38396/0 (socket says 172.21.15.47:38396) 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 -- 172.21.15.47:0/1037758957 learned_addr learned my addr 172.21.15.47:0/1037758957 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): get_auth_request con 0x7fe630071630 auth_method 0 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): get_auth_request con 0x7fe630071f50 auth_method 0 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe636381700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6300a2e90 0x7fe6301bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe636381700 10 monclient(hunting): get_auth_request con 0x7fe6300a2e90 auth_method 0 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe636381700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe636381700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe636381700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient(hunting): handle_auth_done global_id 35891 payload 824 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6300a2e90 msgr2=0x7fe6301bd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6300a2e90 0x7fe6301bd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630071f50 msgr2=0x7fe6301bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630071f50 0x7fe6301bcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 monclient: found mon.noname-e 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628009b30 con 0x7fe630071630 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe63537f700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe630071f50 0x7fe6301bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe635b80700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 0x7fe6301c9020 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b530 tx=0x7fe62c00b840 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe6267fc700 10 client.?.objecter ms_handle_connect 0x7fe630071630 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.013+0000 7fe6267fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe62c011aa0 con 0x7fe630071630 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: dump: 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.032 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.033 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c011c00 con 0x7fe630071630 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 5 monclient: authenticate success, global_id 35891 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 10 monclient: _renew_subs 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6301a82d0 con 0x7fe630071630 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe62c011db0 con 0x7fe630071630 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:13.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe6267fc700 10 monclient: dump: 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.035 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe627fff700 4 set_mon_vals no callback set 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 10 client.35891.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 10 monclient: _renew_subs 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6301a90c0 con 0x7fe630071630 2024-04-20T02:31:13.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.014+0000 7fe637de4700 1 librados: init done 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe637de4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe637de4700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 0x7fe6301cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe630068310 con 0x7fe63006aac0 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe62c011630 con 0x7fe630071630 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 0x7fe6301cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient: get_auth_request con 0x7fe63006aac0 auth_method 0 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe61403a320 0x7fe61403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(599..599 src has 266..599) v4 ==== 6641+0+0 (secure 0 0 0) 0x7fe62c04a9c0 con 0x7fe630071630 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 10 client.35891.objecter ms_dispatch 0x7fe630001550 osd_map(599..599 src has 266..599) v4 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 3 client.35891.objecter handle_osd_map got epochs [599,599] > 0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 3 client.35891.objecter handle_osd_map decoding full epoch 599 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 10 monclient(hunting): handle_auth_done global_id 55113 payload 824 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe630075770 con 0x7fe63006aac0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 20 client.35891.objecter dump_active .. 0 homeless 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe636381700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe61403a320 0x7fe61403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe636381700 10 monclient: get_auth_request con 0x7fe61403a320 auth_method 0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 10 client.35891.objecter ms_handle_connect 0x7fe63006aac0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe63537f700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 0x7fe6301cc520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe62000eae0 tx=0x7fe62000eea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.015+0000 7fe6267fc700 10 client.35891.objecter resend_mon_ops 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe62000cd00 con 0x7fe63006aac0 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:13.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 10 monclient: dump: 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.039 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 10 client.35891.objecter ms_handle_connect 0x7fe61403a320 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe636381700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe61403a320 0x7fe61403c7e0 secure :-1 s=READY pgs=3328 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b370 tx=0x7fe628009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe62000ce60 con 0x7fe63006aac0 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 msgr2=0x7fe6301cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 0x7fe6301cc520 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe62000eae0 tx=0x7fe62000eea0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 msgr2=0x7fe6301cc520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.016+0000 7fe6267fc700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe63006aac0 0x7fe6301cc520 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe62000eae0 tx=0x7fe62000eea0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe637de4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe637de4700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 0x7fe6301cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe637de4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe63006a400 con 0x7fe6301cca60 2024-04-20T02:31:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 0x7fe6301cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient: get_auth_request con 0x7fe6301cca60 auth_method 0 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 10 monclient(hunting): handle_auth_done global_id 55122 payload 824 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 1 -- 172.21.15.47:0/1037758957 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe6300a4480 con 0x7fe6301cca60 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe63537f700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 0x7fe6301cce70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe62000eae0 tx=0x7fe620004860 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe6267fc700 10 client.35891.objecter ms_handle_connect 0x7fe6301cca60 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe6267fc700 10 client.35891.objecter resend_mon_ops 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.024+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe620023070 con 0x7fe6301cca60 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:13.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 10 monclient: dump: 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.044 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fe6200067d0 con 0x7fe6301cca60 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 msgr2=0x7fe6301cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 0x7fe6301cce70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe62000eae0 tx=0x7fe620004860 comp rx=0 tx=0).stop 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 msgr2=0x7fe6301cce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe6267fc700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe6301cca60 0x7fe6301cce70 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe62000eae0 tx=0x7fe620004860 comp rx=0 tx=0).stop 2024-04-20T02:31:13.045 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:13.046 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.047 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.048 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:13.049 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.050 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.053 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.056 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:13.057 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.059 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:13.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.061 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.063 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:13.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.066 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.068 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:13.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:13.070 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 10 librados: watch_flush enter 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 10 librados: watch_flush exit 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 20 client.35891.objecter shutdown clearing up homeless session... 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 10 client.35891.objecter successfully canceled tick 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe61403a320 msgr2=0x7fe61403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe61403a320 0x7fe61403c7e0 secure :-1 s=READY pgs=3328 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b370 tx=0x7fe628009000 comp rx=0 tx=0).stop 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 10 monclient: shutdown 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 msgr2=0x7fe6301c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 0x7fe6301c9020 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b530 tx=0x7fe62c00b840 comp rx=0 tx=0).stop 2024-04-20T02:31:13.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe636381700 1 -- 172.21.15.47:0/1037758957 reap_dead start 2024-04-20T02:31:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 shutdown_connections 2024-04-20T02:31:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 --2- 172.21.15.47:0/1037758957 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe630071630 0x7fe6301c9020 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 >> 172.21.15.47:0/1037758957 conn(0x7fe6300a7570 msgr2=0x7fe630070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 shutdown_connections 2024-04-20T02:31:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 -- 172.21.15.47:0/1037758957 wait complete. 2024-04-20T02:31:13.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.025+0000 7fe637de4700 1 librados: shutdown 2024-04-20T02:31:13.074 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:13.074 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:31:13.075 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:31:13.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.214+0000 7f6ec279b700 10 monclient: get_monmap_and_config 2024-04-20T02:31:13.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.214+0000 7f6ec279b700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.214+0000 7f6ec279b700 10 monclient: monmap: 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:13.215845+0000 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:13.215845+0000 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:13.273 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:13.274 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.215+0000 7f6ec279b700 1 Processor -- start 2024-04-20T02:31:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.215+0000 7f6ec279b700 1 -- start start 2024-04-20T02:31:13.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.215+0000 7f6ec279b700 10 monclient: init 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient: _add_conns ranks=[0,2,8,4,5,10,7,6,1,3,9] 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ebc0a3960 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ebc0a4450 0x7f6ebc070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): picked mon.noname-b con 0x7f6ebc0a4450 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc13cf20 0x7f6ebc13f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): picked mon.noname-e con 0x7f6ebc13cf20 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0751e0 con 0x7f6ebc0a3960 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc06ca20 con 0x7f6ebc0a4450 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc06e110 con 0x7f6ebc13cf20 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec279b700 10 monclient(hunting): authenticate will time out at 2552.617234s 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ebc0a4450 0x7f6ebc070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebbfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ebc0a3960 0x7f6ebc0a3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:39136/0 (socket says 172.21.15.47:39136) 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebbfff700 1 -- 172.21.15.47:0/4240495023 learned_addr learned my addr 172.21.15.47:0/4240495023 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): get_auth_request con 0x7f6ebc0a4450 auth_method 0 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 1 --2- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc13cf20 0x7f6ebc13f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): get_auth_request con 0x7f6ebc13cf20 auth_method 0 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ec0d38700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_done global_id 24707 payload 274 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc13cf20 msgr2=0x7f6ebc13f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 --2- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc13cf20 0x7f6ebc13f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/4240495023 >> v1:172.21.15.47:6789/0 conn(0x7f6ebc0a3960 legacy=0x7f6ebc0a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebaffd700 1 -- 172.21.15.47:0/4240495023 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130591323 0 0) 0x7f6ebc0751e0 con 0x7f6ebc0a3960 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 monclient: found mon.noname-b 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/4240495023 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc06e3f0 con 0x7f6ebc0a4450 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebaffd700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:31:13.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.216+0000 7f6ebb7fe700 1 --2- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ebc0a4450 0x7f6ebc070590 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009870 tx=0x7f6ea4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab9bf8ec6b8efff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 1 -- 172.21.15.47:0/4240495023 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ea4011aa0 con 0x7f6ebc0a4450 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: dump: 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.290 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 1 -- 172.21.15.47:0/4240495023 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011c00 con 0x7f6ebc0a4450 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 5 monclient: authenticate success, global_id 24707 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 1 -- 172.21.15.47:0/4240495023 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ea4011d70 con 0x7f6ebc0a4450 2024-04-20T02:31:13.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 4 set_mon_vals no callback set 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ebaffd700 10 monclient: dump: 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.292 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 10 monclient: shutdown 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 -- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ebc0a4450 msgr2=0x7f6ebc070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 --2- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ebc0a4450 0x7f6ebc070590 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009870 tx=0x7f6ea4009b80 comp rx=0 tx=0).stop 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 -- 172.21.15.47:0/4240495023 shutdown_connections 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 --2- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc13cf20 0x7f6ebc13f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 --2- 172.21.15.47:0/4240495023 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6ebc0a4450 0x7f6ebc070590 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 -- 172.21.15.47:0/4240495023 >> 172.21.15.47:0/4240495023 conn(0x7f6ebc0a8610 msgr2=0x7f6ebc0a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 -- 172.21.15.47:0/4240495023 shutdown_connections 2024-04-20T02:31:13.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.217+0000 7f6ec279b700 1 -- 172.21.15.47:0/4240495023 wait complete. 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.218+0000 7f6ec279b700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.218+0000 7f6ec279b700 10 monclient: monmap: 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.294 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.218+0000 7f6ec279b700 1 librados: starting msgr at 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.218+0000 7f6ec279b700 1 librados: starting objecter 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.218+0000 7f6ec279b700 1 Processor -- start 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 -- start start 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 librados: setting wanted keys 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 librados: calling monclient init 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient: init 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient: _add_conns ranks=[5,0,1,8,4,2,7,6,3] 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6ebc0a3960 0x7f6ebc13bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): picked mon.noname-h con 0x7f6ebc0a3960 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc0a4450 0x7f6ebc134ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): picked mon.noname-a con 0x7f6ebc0a4450 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ebc13cf20 0x7f6ebc135530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): picked mon.noname-f con 0x7f6ebc13cf20 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1213e0 con 0x7f6ebc0a4450 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1cc750 con 0x7f6ebc13cf20 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc071f80 con 0x7f6ebc0a3960 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec279b700 10 monclient(hunting): authenticate will time out at 2552.620544s 2024-04-20T02:31:13.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc0a4450 0x7f6ebc134ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc0a4450 0x7f6ebc134ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:39728/0 (socket says 172.21.15.47:39728) 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/2357783613 learned_addr learned my addr 172.21.15.47:0/2357783613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): get_auth_request con 0x7f6ebc0a4450 auth_method 0 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ebc13cf20 0x7f6ebc135530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): get_auth_request con 0x7f6ebc13cf20 auth_method 0 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebbfff700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6ebc0a3960 0x7f6ebc13bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebbfff700 10 monclient(hunting): get_auth_request con 0x7f6ebc0a3960 auth_method 0 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ec0d38700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient(hunting): handle_auth_done global_id 36675 payload 824 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6ebc0a3960 msgr2=0x7f6ebc13bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6ebc0a3960 0x7f6ebc13bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ebc13cf20 msgr2=0x7f6ebc135530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ebc13cf20 0x7f6ebc135530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.219+0000 7f6ebb7fe700 1 monclient: found mon.noname-a 2024-04-20T02:31:13.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ebb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ebb7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ebb7fe700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40094a0 con 0x7f6ebc0a4450 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ebbfff700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f6ebc0a3960 0x7f6ebc13bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ebb7fe700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc0a4450 0x7f6ebc134ff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40090b0 tx=0x7f6ea4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 client.?.objecter ms_handle_connect 0x7f6ebc0a4450 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ea4004330 con 0x7f6ebc0a4450 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: dump: 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.299 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 5 monclient: authenticate success, global_id 36675 2024-04-20T02:31:13.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 10 monclient: _renew_subs 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc072340 con 0x7f6ebc0a4450 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea401f070 con 0x7f6ebc0a4450 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ea4004490 con 0x7f6ebc0a4450 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 10 client.36675.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eb8ff9700 10 monclient: dump: 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.301 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.302 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 10 monclient: _renew_subs 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc11f640 con 0x7f6ebc0a4450 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6ec279b700 1 librados: init done 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.220+0000 7f6eba7fc700 4 set_mon_vals no callback set 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ec279b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ec279b700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 0x7f6ebc121570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc068380 con 0x7f6ebc06aac0 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6ea4021030 con 0x7f6ebc0a4450 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 0x7f6ebc121570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient: get_auth_request con 0x7f6ebc06aac0 auth_method 0 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e9c03a2f0 0x7f6e9c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(599..599 src has 266..599) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f6ea404bb40 con 0x7f6ebc0a4450 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 10 client.36675.objecter ms_dispatch 0x7f6ebc001550 osd_map(599..599 src has 266..599) v4 2024-04-20T02:31:13.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 3 client.36675.objecter handle_osd_map got epochs [599,599] > 0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 3 client.36675.objecter handle_osd_map decoding full epoch 599 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_done global_id 35909 payload 824 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ebc0a5490 con 0x7f6ebc06aac0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 20 client.36675.objecter dump_active .. 0 homeless 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ec0d38700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e9c03a2f0 0x7f6e9c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ec0d38700 10 monclient: get_auth_request con 0x7f6e9c03a2f0 auth_method 0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 10 client.36675.objecter ms_handle_connect 0x7f6ebc06aac0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6eb8ff9700 10 client.36675.objecter resend_mon_ops 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.221+0000 7f6ebbfff700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 0x7f6ebc121570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac001870 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eac011960 con 0x7f6ebc06aac0 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 10 monclient: dump: 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.305 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 10 client.36675.objecter ms_handle_connect 0x7f6e9c03a2f0 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6ec0d38700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e9c03a2f0 0x7f6e9c03c7b0 secure :-1 s=READY pgs=3329 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc070120 tx=0x7f6eb400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6eac0114f0 con 0x7f6ebc06aac0 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 msgr2=0x7f6ebc121570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 0x7f6ebc121570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac001870 comp rx=0 tx=0).stop 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 msgr2=0x7f6ebc121570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.222+0000 7f6eb8ff9700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc06aac0 0x7f6ebc121570 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac001870 comp rx=0 tx=0).stop 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ec279b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ec279b700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 0x7f6ebc11e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ec279b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc06a400 con 0x7f6ebc11e5a0 2024-04-20T02:31:13.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 0x7f6ebc11e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient: get_auth_request con 0x7f6ebc11e5a0 auth_method 0 2024-04-20T02:31:13.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 10 monclient(hunting): handle_auth_done global_id 35918 payload 824 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 1 -- 172.21.15.47:0/2357783613 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ebc070200 con 0x7f6ebc11e5a0 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6eb8ff9700 10 client.36675.objecter ms_handle_connect 0x7f6ebc11e5a0 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6ebbfff700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 0x7f6ebc11e9b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6eac016000 tx=0x7f6eac000ed0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.230+0000 7f6eb8ff9700 10 client.36675.objecter resend_mon_ops 2024-04-20T02:31:13.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eac021070 con 0x7f6ebc11e5a0 2024-04-20T02:31:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 10 monclient: dump: 2024-04-20T02:31:13.310 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.311 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f6eac01c8e0 con 0x7f6ebc11e5a0 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 msgr2=0x7f6ebc11e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 0x7f6ebc11e9b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6eac016000 tx=0x7f6eac000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 msgr2=0x7f6ebc11e9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6eb8ff9700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6ebc11e5a0 0x7f6ebc11e9b0 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6eac016000 tx=0x7f6eac000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:13.312 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:31:13.313 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:13.314 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:13.315 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.316 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:13.317 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.318 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.319 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:13.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.321 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:13.322 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.323 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:13.326 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:13.329 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.330 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:13.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:13.332 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 10 librados: watch_flush enter 2024-04-20T02:31:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 10 librados: watch_flush exit 2024-04-20T02:31:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 20 client.36675.objecter shutdown clearing up homeless session... 2024-04-20T02:31:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 10 client.36675.objecter successfully canceled tick 2024-04-20T02:31:13.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e9c03a2f0 msgr2=0x7f6e9c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e9c03a2f0 0x7f6e9c03c7b0 secure :-1 s=READY pgs=3329 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc070120 tx=0x7f6eb400b040 comp rx=0 tx=0).stop 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 10 monclient: shutdown 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc0a4450 msgr2=0x7f6ebc134ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 --2- 172.21.15.47:0/2357783613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ebc0a4450 0x7f6ebc134ff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40090b0 tx=0x7f6ea4004750 comp rx=0 tx=0).stop 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec0d38700 1 -- 172.21.15.47:0/2357783613 reap_dead start 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec0d38700 1 -- 172.21.15.47:0/2357783613 reap_dead start 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 shutdown_connections 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 >> 172.21.15.47:0/2357783613 conn(0x7f6ebc0a8610 msgr2=0x7f6ebc0a1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 shutdown_connections 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 -- 172.21.15.47:0/2357783613 wait complete. 2024-04-20T02:31:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.231+0000 7f6ec279b700 1 librados: shutdown 2024-04-20T02:31:13.336 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:13.336 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:31:13.336 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.477+0000 7f171e45d700 10 monclient: get_monmap_and_config 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.477+0000 7f171e45d700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.477+0000 7f171e45d700 10 monclient: monmap: 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:13.478286+0000 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:13.478286+0000 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.477 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:13.478 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.477+0000 7f171e45d700 1 Processor -- start 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.477+0000 7f171e45d700 1 -- start start 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient: init 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient: _add_conns ranks=[1,9,5,0,10,2,3,6,4,8,7] 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1718137d90 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17180a09e0 0x7f171813cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): picked mon.noname-a con 0x7f17180a09e0 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f171813d1c0 0x7f171813f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): picked mon.noname-h con 0x7f171813d1c0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17180751e0 con 0x7f1718137d90 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171809fa40 con 0x7f17180a09e0 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f171809dfc0 con 0x7f171813d1c0 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171e45d700 10 monclient(hunting): authenticate will time out at 2552.879545s 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17180a09e0 0x7f171813cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f17180a09e0 0x7f171813cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49760/0 (socket says 172.21.15.47:49760) 2024-04-20T02:31:13.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 1 -- 172.21.15.47:0/563824992 learned_addr learned my addr 172.21.15.47:0/563824992 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): get_auth_request con 0x7f17180a09e0 auth_method 0 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171c9fa700 1 --2- 172.21.15.47:0/563824992 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f171813d1c0 0x7f171813f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171c9fa700 10 monclient(hunting): get_auth_request con 0x7f171813d1c0 auth_method 0 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171c9fa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171c9fa700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f171c9fa700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.478+0000 7f17177fe700 10 monclient(hunting): handle_auth_done global_id 36684 payload 274 2024-04-20T02:31:13.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 1 -- 172.21.15.47:0/563824992 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f171813d1c0 msgr2=0x7f171813f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 1 --2- 172.21.15.47:0/563824992 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f171813d1c0 0x7f171813f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 1 -- 172.21.15.47:0/563824992 >> v1:172.21.15.179:6789/0 conn(0x7f1718137d90 legacy=0x7f1718138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 1 monclient: found mon.noname-a 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 1 -- 172.21.15.47:0/563824992 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718138880 con 0x7f17180a09e0 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f17177fe700 1 --2- 172.21.15.47:0/563824992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17180a09e0 0x7f171813cc80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f170c0098f0 tx=0x7f170c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66666650fff6b7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 1 -- 172.21.15.47:0/563824992 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f170c00bb10 con 0x7f17180a09e0 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: dump: 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.491 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 1 -- 172.21.15.47:0/563824992 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00bc70 con 0x7f17180a09e0 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f171e45d700 5 monclient: authenticate success, global_id 36684 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f171e45d700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 1 -- 172.21.15.47:0/563824992 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f170c00bde0 con 0x7f17180a09e0 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f171e45d700 4 set_mon_vals no callback set 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.479+0000 7f1716ffd700 10 monclient: dump: 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.493 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 10 monclient: shutdown 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 -- 172.21.15.47:0/563824992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17180a09e0 msgr2=0x7f171813cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 --2- 172.21.15.47:0/563824992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17180a09e0 0x7f171813cc80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f170c0098f0 tx=0x7f170c009c00 comp rx=0 tx=0).stop 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 -- 172.21.15.47:0/563824992 shutdown_connections 2024-04-20T02:31:13.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 --2- 172.21.15.47:0/563824992 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f171813d1c0 0x7f171813f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 --2- 172.21.15.47:0/563824992 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17180a09e0 0x7f171813cc80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 -- 172.21.15.47:0/563824992 >> 172.21.15.47:0/563824992 conn(0x7f17180aa8b0 msgr2=0x7f17180a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 -- 172.21.15.47:0/563824992 shutdown_connections 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 -- 172.21.15.47:0/563824992 wait complete. 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 10 monclient: monmap: 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:13.495 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 librados: starting msgr at 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.480+0000 7f171e45d700 1 librados: starting objecter 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 Processor -- start 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 -- start start 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 librados: setting wanted keys 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 librados: calling monclient init 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient: init 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient: _add_conns ranks=[8,0,4,2,3,7,1,6,5] 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17180a09e0 0x7f17181cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): picked mon.noname-e con 0x7f17180a09e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 0x7f17181ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): picked mon.noname-a con 0x7f1718137d90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f171813d1c0 0x7f17181cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): picked mon.noname-c con 0x7f171813d1c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171809dc20 con 0x7f1718137d90 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f171809e9e0 con 0x7f171813d1c0 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f17180a18c0 con 0x7f17180a09e0 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f171e45d700 10 monclient(hunting): authenticate will time out at 2552.882847s 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f17177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 0x7f17181ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f17177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 0x7f17181ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49768/0 (socket says 172.21.15.47:49768) 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.481+0000 7f1717fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17180a09e0 0x7f17181cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 -- 172.21.15.47:0/3071145330 learned_addr learned my addr 172.21.15.47:0/3071145330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): get_auth_request con 0x7f1718137d90 auth_method 0 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f171813d1c0 0x7f17181cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): get_auth_request con 0x7f17180a09e0 auth_method 0 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): get_auth_request con 0x7f171813d1c0 auth_method 0 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient(hunting): handle_auth_done global_id 36693 payload 824 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17180a09e0 msgr2=0x7f17181cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17180a09e0 0x7f17181cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f171813d1c0 msgr2=0x7f17181cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f171813d1c0 0x7f17181cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 monclient: found mon.noname-a 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c009550 con 0x7f1718137d90 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1717fff700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f17180a09e0 0x7f17181cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171c9fa700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f171813d1c0 0x7f17181cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17177fe700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 0x7f17181ccc80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f170c009c40 tx=0x7f170c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 client.?.objecter ms_handle_connect 0x7f1718137d90 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f170c0036a0 con 0x7f1718137d90 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: dump: 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.500 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c003860 con 0x7f1718137d90 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 5 monclient: authenticate success, global_id 36693 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f170c003e40 con 0x7f1718137d90 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 10 monclient: _renew_subs 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17180a2c40 con 0x7f1718137d90 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f17167fc700 4 set_mon_vals no callback set 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f1714ff9700 10 monclient: dump: 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.502 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 10 client.36693.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 10 monclient: _renew_subs 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1718130df0 con 0x7f1718137d90 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.482+0000 7f171e45d700 1 librados: init done 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f171e45d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f171e45d700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 0x7f171812e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718068180 con 0x7f171806aac0 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f170c021020 con 0x7f1718137d90 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f1717fff700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 0x7f171812e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f1717fff700 10 monclient: get_auth_request con 0x7f171806aac0 auth_method 0 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f1717fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f1717fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.483+0000 7f1717fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f170403a2f0 0x7f170403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(599..599 src has 266..599) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f170c04b790 con 0x7f1718137d90 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 client.36693.objecter ms_dispatch 0x7f1718001550 osd_map(599..599 src has 266..599) v4 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 3 client.36693.objecter handle_osd_map got epochs [599,599] > 0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 3 client.36693.objecter handle_osd_map decoding full epoch 599 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 20 client.36693.objecter dump_active .. 0 homeless 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f171c9fa700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f170403a2f0 0x7f170403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f171c9fa700 10 monclient: get_auth_request con 0x7f170403a2f0 auth_method 0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1717fff700 10 monclient(hunting): handle_auth_done global_id 55513 payload 824 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1717fff700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17180a7bc0 con 0x7f171806aac0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 client.36693.objecter ms_handle_connect 0x7f171806aac0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 client.36693.objecter resend_mon_ops 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1717fff700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 0x7f171812e9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f170800fb10 tx=0x7f170800fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 client.36693.objecter ms_handle_connect 0x7f170403a2f0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f171c9fa700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f170403a2f0 0x7f170403c7b0 secure :-1 s=READY pgs=3330 cs=0 l=1 rev1=1 crypto rx=0x7f171809e400 tx=0x7f1710009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1708017070 con 0x7f171806aac0 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.484+0000 7f1714ff9700 10 monclient: dump: 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.506 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f17080052e0 con 0x7f171806aac0 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 msgr2=0x7f171812e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 0x7f171812e9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f170800fb10 tx=0x7f170800fed0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 msgr2=0x7f171812e9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.485+0000 7f1714ff9700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f171806aac0 0x7f171812e9b0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f170800fb10 tx=0x7f170800fed0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.492+0000 7f171e45d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.492+0000 7f171e45d700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 0x7f171812eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.492+0000 7f171e45d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.492+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171806a400 con 0x7f17180a3120 2024-04-20T02:31:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 0x7f171812eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient: get_auth_request con 0x7f17180a3120 auth_method 0 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 10 monclient(hunting): handle_auth_done global_id 55522 payload 824 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 1 -- 172.21.15.47:0/3071145330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f17180a0000 con 0x7f17180a3120 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1714ff9700 10 client.36693.objecter ms_handle_connect 0x7f17180a3120 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1717fff700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 0x7f171812eef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f17080018f0 tx=0x7f1708000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1714ff9700 10 client.36693.objecter resend_mon_ops 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1708022070 con 0x7f17180a3120 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1714ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1714ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.493+0000 7f1714ff9700 10 monclient: dump: 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.511 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f1708003d70 con 0x7f17180a3120 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 msgr2=0x7f171812eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 0x7f171812eef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f17080018f0 tx=0x7f1708000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 msgr2=0x7f171812eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f1714ff9700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17180a3120 0x7f171812eef0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f17080018f0 tx=0x7f1708000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:13.512 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:31:13.513 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:13.514 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:13.515 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.516 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:13.517 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.518 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.521 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.522 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.523 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.524 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:13.525 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.526 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:13.528 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.529 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.531 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:13.532 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 10 librados: watch_flush enter 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 10 librados: watch_flush exit 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 20 client.36693.objecter shutdown clearing up homeless session... 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 10 client.36693.objecter successfully canceled tick 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f170403a2f0 msgr2=0x7f170403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f170403a2f0 0x7f170403c7b0 secure :-1 s=READY pgs=3330 cs=0 l=1 rev1=1 crypto rx=0x7f171809e400 tx=0x7f1710009780 comp rx=0 tx=0).stop 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 10 monclient: shutdown 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 msgr2=0x7f17181ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 0x7f17181ccc80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f170c009c40 tx=0x7f170c004750 comp rx=0 tx=0).stop 2024-04-20T02:31:13.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171c9fa700 1 -- 172.21.15.47:0/3071145330 reap_dead start 2024-04-20T02:31:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 shutdown_connections 2024-04-20T02:31:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 --2- 172.21.15.47:0/3071145330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1718137d90 0x7f17181ccc80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 >> 172.21.15.47:0/3071145330 conn(0x7f17180aa8b0 msgr2=0x7f1718135100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 shutdown_connections 2024-04-20T02:31:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 -- 172.21.15.47:0/3071145330 wait complete. 2024-04-20T02:31:13.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.494+0000 7f171e45d700 1 librados: shutdown 2024-04-20T02:31:13.535 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:13.535 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:31:13.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.677+0000 7f2c8cfcc700 10 monclient: get_monmap_and_config 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.677+0000 7f2c8cfcc700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.677+0000 7f2c8cfcc700 10 monclient: monmap: 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:13.679097+0000 2024-04-20T02:31:13.678 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:13.679097+0000 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.678+0000 7f2c8cfcc700 1 Processor -- start 2024-04-20T02:31:13.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.678+0000 7f2c8cfcc700 1 -- start start 2024-04-20T02:31:13.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.678+0000 7f2c8cfcc700 10 monclient: init 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient: _add_conns ranks=[8,3,10,5,4,7,9,1,2,0,6] 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c880a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): picked mon.noname-e con 0x7f2c880a19d0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88135f00 0x7f2c880701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): picked mon.noname-g con 0x7f2c88135f00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:13.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c88136380 0x7f2c880706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): picked mon.noname-f con 0x7f2c88136380 addr v2:172.21.15.179:3300/0 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c880a4040 con 0x7f2c88136380 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88072020 con 0x7f2c88135f00 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88072300 con 0x7f2c880a19d0 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8cfcc700 10 monclient(hunting): authenticate will time out at 2553.080387s 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c880a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c880a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55122/0 (socket says 172.21.15.47:55122) 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 -- 172.21.15.47:0/1738170863 learned_addr learned my addr 172.21.15.47:0/1738170863 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): get_auth_request con 0x7f2c880a19d0 auth_method 0 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 1 --2- 172.21.15.47:0/1738170863 >> v2:172.21.15.179:3300/0 conn(0x7f2c88136380 0x7f2c880706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 1 --2- 172.21.15.47:0/1738170863 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88135f00 0x7f2c880701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): get_auth_request con 0x7f2c88136380 auth_method 0 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): get_auth_request con 0x7f2c88135f00 auth_method 0 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c85d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient(hunting): handle_auth_done global_id 35936 payload 274 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 -- 172.21.15.47:0/1738170863 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88135f00 msgr2=0x7f2c880701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 --2- 172.21.15.47:0/1738170863 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88135f00 0x7f2c880701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 -- 172.21.15.47:0/1738170863 >> v2:172.21.15.179:3300/0 conn(0x7f2c88136380 msgr2=0x7f2c880706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 --2- 172.21.15.47:0/1738170863 >> v2:172.21.15.179:3300/0 conn(0x7f2c88136380 0x7f2c880706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 monclient: found mon.noname-e 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.679+0000 7f2c8659c700 1 -- 172.21.15.47:0/1738170863 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880751e0 con 0x7f2c880a19d0 2024-04-20T02:31:13.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8659c700 1 --2- 172.21.15.47:0/1738170863 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c880a1db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c780098f0 tx=0x7f2c78009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c75d28364c872f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 1 -- 172.21.15.47:0/1738170863 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c78011aa0 con 0x7f2c880a19d0 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: dump: 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.695 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 1 -- 172.21.15.47:0/1738170863 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78011c00 con 0x7f2c880a19d0 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 1 -- 172.21.15.47:0/1738170863 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c78011d70 con 0x7f2c880a19d0 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 5 monclient: authenticate success, global_id 35936 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 4 set_mon_vals no callback set 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8559a700 10 monclient: dump: 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.697 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 10 monclient: shutdown 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/1738170863 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 msgr2=0x7f2c880a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/1738170863 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c880a1db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c780098f0 tx=0x7f2c78009c00 comp rx=0 tx=0).stop 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/1738170863 shutdown_connections 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/1738170863 >> v2:172.21.15.179:3300/0 conn(0x7f2c88136380 0x7f2c880706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/1738170863 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c880a1db0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/1738170863 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88135f00 0x7f2c880701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.680+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/1738170863 >> 172.21.15.47:0/1738170863 conn(0x7f2c880a4a50 msgr2=0x7f2c880a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.681+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/1738170863 shutdown_connections 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.681+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/1738170863 wait complete. 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.681+0000 7f2c8cfcc700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 10 monclient: monmap: 2024-04-20T02:31:13.698 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 1 librados: starting msgr at 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 1 librados: starting objecter 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 1 Processor -- start 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 1 -- start start 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 1 librados: setting wanted keys 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 1 librados: calling monclient init 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.682+0000 7f2c8cfcc700 10 monclient: init 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient: _add_conns ranks=[8,0,2,3,7,4,5,1,6] 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c881caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): picked mon.noname-e con 0x7f2c880a19d0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c88135f00 0x7f2c881caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): picked mon.noname-a con 0x7f2c88135f00 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c88136380 0x7f2c88120d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): picked mon.noname-b con 0x7f2c88136380 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88122e30 con 0x7f2c88135f00 2024-04-20T02:31:13.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c881249b0 con 0x7f2c88136380 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88123490 con 0x7f2c880a19d0 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8cfcc700 10 monclient(hunting): authenticate will time out at 2553.084417s 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c881caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c88135f00 0x7f2c881caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c881caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55130/0 (socket says 172.21.15.47:55130) 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c88136380 0x7f2c88120d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 -- 172.21.15.47:0/3078422169 learned_addr learned my addr 172.21.15.47:0/3078422169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): get_auth_request con 0x7f2c880a19d0 auth_method 0 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): get_auth_request con 0x7f2c88136380 auth_method 0 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): get_auth_request con 0x7f2c88135f00 auth_method 0 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c85d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient(hunting): handle_auth_done global_id 35945 payload 824 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c88136380 msgr2=0x7f2c88120d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c88136380 0x7f2c88120d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c88135f00 msgr2=0x7f2c881caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c88135f00 0x7f2c881caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 monclient: found mon.noname-e 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.683+0000 7f2c8659c700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009550 con 0x7f2c880a19d0 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8659c700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c881caa50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2c88072440 tx=0x7f2c78004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 client.?.objecter ms_handle_connect 0x7f2c880a19d0 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c78004390 con 0x7f2c880a19d0 2024-04-20T02:31:13.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: dump: 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.704 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78004550 con 0x7f2c880a19d0 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 5 monclient: authenticate success, global_id 35945 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 10 monclient: _renew_subs 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c88071610 con 0x7f2c880a19d0 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c78024470 con 0x7f2c880a19d0 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c84d99700 4 set_mon_vals no callback set 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 10 client.35945.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:13.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c7effd700 10 monclient: dump: 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.706 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 10 monclient: _renew_subs 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c881228c0 con 0x7f2c880a19d0 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.684+0000 7f2c8cfcc700 1 librados: init done 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2c78021070 con 0x7f2c880a19d0 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c6803a2f0 0x7f2c6803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(600..600 src has 266..600) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f2c7804bc50 con 0x7f2c880a19d0 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 10 client.35945.objecter ms_dispatch 0x7f2c88001550 osd_map(600..600 src has 266..600) v4 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 3 client.35945.objecter handle_osd_map got epochs [600,600] > 0 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 3 client.35945.objecter handle_osd_map decoding full epoch 600 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c7effd700 20 client.35945.objecter dump_active .. 0 homeless 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c8cfcc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 0x7f2c881212c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88068430 con 0x7f2c8806aac0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c85d9b700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c6803a2f0 0x7f2c6803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c85d9b700 10 monclient: get_auth_request con 0x7f2c6803a2f0 auth_method 0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c86d9d700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 0x7f2c881212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c86d9d700 10 monclient: get_auth_request con 0x7f2c8806aac0 auth_method 0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c86d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c86d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.685+0000 7f2c86d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_done global_id 94143 payload 824 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c86d9d700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c88067f90 con 0x7f2c8806aac0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 10 client.35945.objecter ms_handle_connect 0x7f2c6803a2f0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c85d9b700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c6803a2f0 0x7f2c6803c7b0 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 crypto rx=0x7f2c88135750 tx=0x7f2c70009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 10 client.35945.objecter ms_handle_connect 0x7f2c8806aac0 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 10 client.35945.objecter resend_mon_ops 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c86d9d700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 0x7f2c881212c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009e60 tx=0x7f2c8000f490 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c8000e040 con 0x7f2c8806aac0 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.686+0000 7f2c7effd700 10 monclient: dump: 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.709 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2c8000f040 con 0x7f2c8806aac0 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 msgr2=0x7f2c881212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 0x7f2c881212c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009e60 tx=0x7f2c8000f490 comp rx=0 tx=0).stop 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 msgr2=0x7f2c881212c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.687+0000 7f2c7effd700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c8806aac0 0x7f2c881212c0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009e60 tx=0x7f2c8000f490 comp rx=0 tx=0).stop 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.694+0000 7f2c8cfcc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.694+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 0x7f2c88121c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.694+0000 7f2c8cfcc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.694+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8806a400 con 0x7f2c88121800 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 0x7f2c88121c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient: get_auth_request con 0x7f2c88121800 auth_method 0 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 10 monclient(hunting): handle_auth_done global_id 94152 payload 824 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 1 -- 172.21.15.47:0/3078422169 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2c880a3c30 con 0x7f2c88121800 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c86d9d700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 0x7f2c88121c10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009e60 tx=0x7f2c80004ad0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c7effd700 10 client.35945.objecter ms_handle_connect 0x7f2c88121800 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c7effd700 10 client.35945.objecter resend_mon_ops 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c80023070 con 0x7f2c88121800 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c7effd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.695+0000 7f2c7effd700 10 monclient: dump: 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.713 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f2c80015430 con 0x7f2c88121800 2024-04-20T02:31:13.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 msgr2=0x7f2c88121c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 0x7f2c88121c10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009e60 tx=0x7f2c80004ad0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 msgr2=0x7f2c88121c10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c7effd700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c88121800 0x7f2c88121c10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:13.715 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.716 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.717 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:13.718 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.719 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.721 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:13.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.724 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.726 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.727 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:13.728 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.729 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.730 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:13.731 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.732 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.734 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:13.735 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 10 librados: watch_flush enter 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 10 librados: watch_flush exit 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 20 client.35945.objecter shutdown clearing up homeless session... 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 10 client.35945.objecter successfully canceled tick 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c6803a2f0 msgr2=0x7f2c6803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c6803a2f0 0x7f2c6803c7b0 secure :-1 s=READY pgs=3331 cs=0 l=1 rev1=1 crypto rx=0x7f2c88135750 tx=0x7f2c70009210 comp rx=0 tx=0).stop 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 10 monclient: shutdown 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 msgr2=0x7f2c881caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 --2- 172.21.15.47:0/3078422169 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f2c880a19d0 0x7f2c881caa50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2c88072440 tx=0x7f2c78004750 comp rx=0 tx=0).stop 2024-04-20T02:31:13.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c86d9d700 1 -- 172.21.15.47:0/3078422169 reap_dead start 2024-04-20T02:31:13.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c86d9d700 1 -- 172.21.15.47:0/3078422169 reap_dead start 2024-04-20T02:31:13.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 shutdown_connections 2024-04-20T02:31:13.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 >> 172.21.15.47:0/3078422169 conn(0x7f2c880a4a50 msgr2=0x7f2c88134f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 shutdown_connections 2024-04-20T02:31:13.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 -- 172.21.15.47:0/3078422169 wait complete. 2024-04-20T02:31:13.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.696+0000 7f2c8cfcc700 1 librados: shutdown 2024-04-20T02:31:13.738 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:13.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:31:13.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:31:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.876+0000 7f43992c5700 10 monclient: get_monmap_and_config 2024-04-20T02:31:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.876+0000 7f43992c5700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.876+0000 7f43992c5700 10 monclient: monmap: 2024-04-20T02:31:13.876 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:13.877397+0000 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:13.877397+0000 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.877 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:13.878 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:13.878 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.876+0000 7f43992c5700 1 Processor -- start 2024-04-20T02:31:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.876+0000 7f43992c5700 1 -- start start 2024-04-20T02:31:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.876+0000 7f43992c5700 10 monclient: init 2024-04-20T02:31:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient: _add_conns ranks=[8,3,0,9,6,5,4,10,1,2,7] 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 0x7f43940719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): picked mon.noname-e con 0x7f4394071610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4394071f30 0x7f43940a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): picked mon.noname-g con 0x7f4394071f30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f43940a2e90 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43940751e0 con 0x7f43940a2e90 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43940727b0 con 0x7f4394071f30 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f43940a0880 con 0x7f4394071610 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43992c5700 10 monclient(hunting): authenticate will time out at 2553.278499s 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 0x7f43940719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43937fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f43940a2e90 0x7f439413b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37836/0 (socket says 172.21.15.47:37836) 2024-04-20T02:31:13.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43937fe700 1 -- 172.21.15.47:0/248510996 learned_addr learned my addr 172.21.15.47:0/248510996 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 0x7f43940719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55140/0 (socket says 172.21.15.47:55140) 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): get_auth_request con 0x7f4394071610 auth_method 0 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43927fc700 1 --2- 172.21.15.47:0/248510996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4394071f30 0x7f43940a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43927fc700 10 monclient(hunting): get_auth_request con 0x7f4394071f30 auth_method 0 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43927fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43927fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f43927fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient(hunting): handle_auth_done global_id 35954 payload 274 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 -- 172.21.15.47:0/248510996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4394071f30 msgr2=0x7f43940a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 --2- 172.21.15.47:0/248510996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4394071f30 0x7f43940a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 -- 172.21.15.47:0/248510996 >> v1:172.21.15.47:6789/0 conn(0x7f43940a2e90 legacy=0x7f439413b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 monclient: found mon.noname-e 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4392ffd700 1 -- 172.21.15.47:0/248510996 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439409e8f0 con 0x7f4394071610 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4391ffb700 1 -- 172.21.15.47:0/248510996 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690362930 0 0) 0x7f43940751e0 con 0x7f43940a2e90 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.877+0000 7f4391ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4392ffd700 1 --2- 172.21.15.47:0/248510996 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 0x7f43940719f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4384009870 tx=0x7f4384009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d9ea93768747b3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 1 -- 172.21.15.47:0/248510996 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4384011aa0 con 0x7f4394071610 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: dump: 2024-04-20T02:31:13.893 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.894 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 5 monclient: authenticate success, global_id 35954 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 1 -- 172.21.15.47:0/248510996 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4384011c00 con 0x7f4394071610 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 4 set_mon_vals no callback set 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 1 -- 172.21.15.47:0/248510996 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4384011d70 con 0x7f4394071610 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:13.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f4391ffb700 10 monclient: dump: 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 10 monclient: shutdown 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 1 -- 172.21.15.47:0/248510996 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 msgr2=0x7f43940719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.878+0000 7f43992c5700 1 --2- 172.21.15.47:0/248510996 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 0x7f43940719f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4384009870 tx=0x7f4384009b80 comp rx=0 tx=0).stop 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 -- 172.21.15.47:0/248510996 shutdown_connections 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 --2- 172.21.15.47:0/248510996 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f4394071610 0x7f43940719f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 --2- 172.21.15.47:0/248510996 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4394071f30 0x7f43940a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 -- 172.21.15.47:0/248510996 >> 172.21.15.47:0/248510996 conn(0x7f43940a7570 msgr2=0x7f43940a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 -- 172.21.15.47:0/248510996 shutdown_connections 2024-04-20T02:31:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 -- 172.21.15.47:0/248510996 wait complete. 2024-04-20T02:31:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 10 monclient: build_initial_monmap 2024-04-20T02:31:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 10 monclient: monmap: 2024-04-20T02:31:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.879+0000 7f43992c5700 1 librados: starting msgr at 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 librados: starting objecter 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 Processor -- start 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 -- start start 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 librados: setting wanted keys 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 librados: calling monclient init 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient: init 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient: _add_conns ranks=[1,0,5,2,6,8,7,4,3] 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4394071610 0x7f43941ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): picked mon.noname-f con 0x7f4394071610 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:13.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 0x7f43941ba710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): picked mon.noname-a con 0x7f4394071f30 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43940a2e90 0x7f43941bac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): picked mon.noname-h con 0x7f43940a2e90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43941a7eb0 con 0x7f4394071f30 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43941a8100 con 0x7f4394071610 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43941a6970 con 0x7f43940a2e90 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.880+0000 7f43992c5700 10 monclient(hunting): authenticate will time out at 2553.281964s 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 0x7f43941ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 0x7f43941ba710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49790/0 (socket says 172.21.15.47:49790) 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 -- 172.21.15.47:0/3455007447 learned_addr learned my addr 172.21.15.47:0/3455007447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): get_auth_request con 0x7f4394071f30 auth_method 0 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4394071610 0x7f43941ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): get_auth_request con 0x7f4394071610 auth_method 0 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43940a2e90 0x7f43941bac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): get_auth_request con 0x7f43940a2e90 auth_method 0 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43937fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient(hunting): handle_auth_done global_id 36720 payload 824 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43940a2e90 msgr2=0x7f43941bac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43940a2e90 0x7f43941bac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4394071610 msgr2=0x7f43941ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4394071610 0x7f43941ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient: _finish_hunting 0 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 monclient: found mon.noname-a 2024-04-20T02:31:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43840094a0 con 0x7f4394071f30 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43927fc700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 0x7f43941ba710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f438800e7e0 tx=0x7f438800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 client.?.objecter ms_handle_connect 0x7f4394071f30 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f438800c070 con 0x7f4394071f30 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: dump: 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43992c5700 5 monclient: authenticate success, global_id 36720 2024-04-20T02:31:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4388013880 con 0x7f4394071f30 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43992c5700 10 monclient: _renew_subs 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4388013e60 con 0x7f4394071f30 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43992c5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43940723f0 con 0x7f4394071f30 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f43917fa700 4 set_mon_vals no callback set 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 10 client.36720.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.881+0000 7f437b7fe700 10 monclient: dump: 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 10 monclient: _renew_subs 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43941a5d00 con 0x7f4394071f30 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 1 librados: init done 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 0x7f43941a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4394068310 con 0x7f439406aac0 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.882+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f43880139f0 con 0x7f4394071f30 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 0x7f43941a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient: get_auth_request con 0x7f439406aac0 auth_method 0 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f438003a2f0 0x7f438003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(600..600 src has 266..600) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f438804a5b0 con 0x7f4394071f30 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 client.36720.objecter ms_dispatch 0x7f4394001550 osd_map(600..600 src has 266..600) v4 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 3 client.36720.objecter handle_osd_map got epochs [600,600] > 0 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 3 client.36720.objecter handle_osd_map decoding full epoch 600 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 20 client.36720.objecter dump_active .. 0 homeless 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f43937fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f438003a2f0 0x7f438003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f43937fe700 10 monclient: get_auth_request con 0x7f438003a2f0 auth_method 0 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 10 monclient(hunting): handle_auth_done global_id 84299 payload 824 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43941a8240 con 0x7f439406aac0 2024-04-20T02:31:13.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 client.36720.objecter ms_handle_connect 0x7f439406aac0 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 client.36720.objecter resend_mon_ops 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f4392ffd700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 0x7f43941a41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4384009870 tx=0x7f4384004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4384020070 con 0x7f439406aac0 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 monclient: dump: 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.912 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f43937fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f438003a2f0 0x7f438003c7b0 secure :-1 s=READY pgs=3332 cs=0 l=1 rev1=1 crypto rx=0x7f438c001c70 tx=0x7f438c001c00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.883+0000 7f437b7fe700 10 client.36720.objecter ms_handle_connect 0x7f438003a2f0 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f43840036a0 con 0x7f439406aac0 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 msgr2=0x7f43941a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 0x7f43941a41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4384009870 tx=0x7f4384004750 comp rx=0 tx=0).stop 2024-04-20T02:31:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 msgr2=0x7f43941a41c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.884+0000 7f437b7fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f439406aac0 0x7f43941a41c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f43992c5700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f43992c5700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 0x7f43941a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f43992c5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f439406a400 con 0x7f43941a4700 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 0x7f43941a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient: get_auth_request con 0x7f43941a4700 auth_method 0 2024-04-20T02:31:13.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 10 monclient(hunting): handle_auth_done global_id 84308 payload 824 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 1 -- 172.21.15.47:0/3455007447 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f439409ffe0 con 0x7f43941a4700 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.892+0000 7f4392ffd700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 0x7f43941a4b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f438400a010 tx=0x7f4384009210 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 client.36720.objecter ms_handle_connect 0x7f43941a4700 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 client.36720.objecter resend_mon_ops 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4384003e70 con 0x7f43941a4700 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 monclient: dump: 2024-04-20T02:31:13.917 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:13.918 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f438402c300 con 0x7f43941a4700 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 msgr2=0x7f43941a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 0x7f43941a4b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f438400a010 tx=0x7f4384009210 comp rx=0 tx=0).stop 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 msgr2=0x7f43941a4b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f437b7fe700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f43941a4700 0x7f43941a4b10 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f438400a010 tx=0x7f4384009210 comp rx=0 tx=0).stop 2024-04-20T02:31:13.919 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:13.920 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.921 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.922 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:13.923 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:13.924 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.926 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.928 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.929 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:13.930 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.931 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.933 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.934 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.937 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:13.938 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.939 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.940 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:13.941 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:13.942 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 10 librados: watch_flush enter 2024-04-20T02:31:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 10 librados: watch_flush exit 2024-04-20T02:31:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 20 client.36720.objecter shutdown clearing up homeless session... 2024-04-20T02:31:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 10 client.36720.objecter successfully canceled tick 2024-04-20T02:31:13.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f438003a2f0 msgr2=0x7f438003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f438003a2f0 0x7f438003c7b0 secure :-1 s=READY pgs=3332 cs=0 l=1 rev1=1 crypto rx=0x7f438c001c70 tx=0x7f438c001c00 comp rx=0 tx=0).stop 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 10 monclient: shutdown 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 msgr2=0x7f43941ba710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43992c5700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 0x7f43941ba710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f438800e7e0 tx=0x7f438800eaf0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.893+0000 7f43937fe700 1 -- 172.21.15.47:0/3455007447 reap_dead start 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.894+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 shutdown_connections 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.894+0000 7f43992c5700 1 --2- 172.21.15.47:0/3455007447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4394071f30 0x7f43941ba710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.894+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 >> 172.21.15.47:0/3455007447 conn(0x7f43940a7570 msgr2=0x7f43940a6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.894+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 shutdown_connections 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.894+0000 7f43992c5700 1 -- 172.21.15.47:0/3455007447 wait complete. 2024-04-20T02:31:13.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:13.894+0000 7f43992c5700 1 librados: shutdown 2024-04-20T02:31:13.945 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:13.945 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:31:13.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:31:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.083+0000 7f45bf576700 10 monclient: get_monmap_and_config 2024-04-20T02:31:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.083+0000 7f45bf576700 10 monclient: build_initial_monmap 2024-04-20T02:31:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.083+0000 7f45bf576700 10 monclient: monmap: 2024-04-20T02:31:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:14.083 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:14.084572+0000 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:14.084572+0000 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:14.084 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:14.085 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 Processor -- start 2024-04-20T02:31:14.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 -- start start 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient: init 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient: _add_conns ranks=[3,6,2,10,9,5,8,7,4,0,1] 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b809f9e0 0x7f45b8135e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): picked mon.noname-g con 0x7f45b809f9e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 0x7f45b80a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): picked mon.noname-d con 0x7f45b809fdc0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 0x7f45b80a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): picked mon.noname-b con 0x7f45b80a1f10 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8136350 con 0x7f45b80a1f10 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80a0740 con 0x7f45b809f9e0 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f45b809e520 con 0x7f45b809fdc0 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.084+0000 7f45bf576700 10 monclient(hunting): authenticate will time out at 2553.485937s 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 0x7f45b80a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 0x7f45b80a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 0x7f45b80a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45712/0 (socket says 172.21.15.47:45712) 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 0x7f45b80a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36784/0 (socket says 172.21.15.47:36784) 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 -- 172.21.15.47:0/3155412510 learned_addr learned my addr 172.21.15.47:0/3155412510 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): get_auth_request con 0x7f45b809fdc0 auth_method 0 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): get_auth_request con 0x7f45b80a1f10 auth_method 0 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bd312700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b809f9e0 0x7f45b8135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bd312700 10 monclient(hunting): get_auth_request con 0x7f45b809f9e0 auth_method 0 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bd312700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bd312700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bd312700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient(hunting): handle_auth_done global_id 55131 payload 274 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 -- 172.21.15.47:0/3155412510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b809f9e0 msgr2=0x7f45b8135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b809f9e0 0x7f45b8135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 -- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 msgr2=0x7f45b80a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 0x7f45b80a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient: _finish_hunting 0 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 monclient: found mon.noname-d 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 -- 172.21.15.47:0/3155412510 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80751e0 con 0x7f45b809fdc0 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bdb13700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 0x7f45b80a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bcb11700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 0x7f45b80a0200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009920 tx=0x7f45a8009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d7e4ac79ec5fb735 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 1 -- 172.21.15.47:0/3155412510 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45a8011aa0 con 0x7f45b809fdc0 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: dump: 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: _finish_auth 0 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bf576700 5 monclient: authenticate success, global_id 55131 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bf576700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 1 -- 172.21.15.47:0/3155412510 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8011c00 con 0x7f45b809fdc0 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bf576700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45bf576700 4 set_mon_vals no callback set 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 1 -- 172.21.15.47:0/3155412510 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45a8011d70 con 0x7f45b809fdc0 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.085+0000 7f45affff700 10 monclient: dump: 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.101 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 10 monclient: shutdown 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:14.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 -- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 msgr2=0x7f45b80a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 0x7f45b80a0200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009920 tx=0x7f45a8009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 -- 172.21.15.47:0/3155412510 shutdown_connections 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f45b80a1f10 0x7f45b80a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b809f9e0 0x7f45b8135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 --2- 172.21.15.47:0/3155412510 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f45b809fdc0 0x7f45b80a0200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 -- 172.21.15.47:0/3155412510 >> 172.21.15.47:0/3155412510 conn(0x7f45b80a3000 msgr2=0x7f45b80a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 -- 172.21.15.47:0/3155412510 shutdown_connections 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.086+0000 7f45bf576700 1 -- 172.21.15.47:0/3155412510 wait complete. 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 10 monclient: build_initial_monmap 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 10 monclient: monmap: 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:14.103 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 1 librados: starting msgr at 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 1 librados: starting objecter 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 1 Processor -- start 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 1 -- start start 2024-04-20T02:31:14.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 1 librados: setting wanted keys 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 1 librados: calling monclient init 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.087+0000 7f45bf576700 10 monclient: init 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient: _add_conns ranks=[4,1,3,7,2,0,6,5,8] 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f45b809f9e0 0x7f45b81c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): picked mon.noname-c con 0x7f45b809f9e0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b809fdc0 0x7f45b8134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): picked mon.noname-f con 0x7f45b809fdc0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b80a1f10 0x7f45b8135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): picked mon.noname-g con 0x7f45b80a1f10 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8130880 con 0x7f45b809fdc0 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b809cfc0 con 0x7f45b80a1f10 2024-04-20T02:31:14.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45b812f280 con 0x7f45b809f9e0 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bf576700 10 monclient(hunting): authenticate will time out at 2553.489291s 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f45b809f9e0 0x7f45b81c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f45b809f9e0 0x7f45b81c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55516/0 (socket says 172.21.15.47:55516) 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 -- 172.21.15.47:0/4220084470 learned_addr learned my addr 172.21.15.47:0/4220084470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): get_auth_request con 0x7f45b809f9e0 auth_method 0 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bcb11700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b809fdc0 0x7f45b8134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bdb13700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b80a1f10 0x7f45b8135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bcb11700 10 monclient(hunting): get_auth_request con 0x7f45b809fdc0 auth_method 0 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bcb11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bcb11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bcb11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bdb13700 10 monclient(hunting): get_auth_request con 0x7f45b80a1f10 auth_method 0 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bdb13700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bdb13700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bdb13700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient(hunting): handle_auth_done global_id 84370 payload 824 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b80a1f10 msgr2=0x7f45b8135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f45b80a1f10 0x7f45b8135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b809fdc0 msgr2=0x7f45b8134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45b809fdc0 0x7f45b8134d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient: _finish_hunting 0 2024-04-20T02:31:14.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 monclient: found mon.noname-c 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0009a70 con 0x7f45b809f9e0 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45bd312700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f45b809f9e0 0x7f45b81c6ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e840 tx=0x7f45b400ec00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45adffb700 10 client.?.objecter ms_handle_connect 0x7f45b809f9e0 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.088+0000 7f45adffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45b4013880 con 0x7f45b809f9e0 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: dump: 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.108 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4013a20 con 0x7f45b809f9e0 2024-04-20T02:31:14.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 5 monclient: authenticate success, global_id 84370 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 10 monclient: _renew_subs 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a8009550 con 0x7f45b809f9e0 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45b4012370 con 0x7f45b809f9e0 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45adffb700 10 monclient: dump: 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.110 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 10 client.84370.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 10 monclient: _renew_subs 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b809ca90 con 0x7f45b809f9e0 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45af7fe700 4 set_mon_vals no callback set 2024-04-20T02:31:14.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.089+0000 7f45bf576700 1 librados: init done 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bf576700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bf576700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 0x7f45b81ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8068210 con 0x7f45b806aac0 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f45b4013b90 con 0x7f45b809f9e0 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f459c03a320 0x7f459c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(600..600 src has 266..600) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f45b404a4c0 con 0x7f45b809f9e0 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 10 client.84370.objecter ms_dispatch 0x7f45b8001550 osd_map(600..600 src has 266..600) v4 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 3 client.84370.objecter handle_osd_map got epochs [600,600] > 0 2024-04-20T02:31:14.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 3 client.84370.objecter handle_osd_map decoding full epoch 600 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 0x7f45b81ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient: get_auth_request con 0x7f45b806aac0 auth_method 0 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45adffb700 20 client.84370.objecter dump_active .. 0 homeless 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bdb13700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f459c03a320 0x7f459c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bdb13700 10 monclient: get_auth_request con 0x7f459c03a320 auth_method 0 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 10 monclient(hunting): handle_auth_done global_id 84328 payload 824 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.090+0000 7f45bcb11700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45b80a0a00 con 0x7f45b806aac0 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 client.84370.objecter ms_handle_connect 0x7f45b806aac0 2024-04-20T02:31:14.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 client.84370.objecter resend_mon_ops 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45bcb11700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 0x7f45b81ca400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45b8074d40 tx=0x7f45a8004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 client.84370.objecter ms_handle_connect 0x7f459c03a320 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45bdb13700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f459c03a320 0x7f459c03c7e0 secure :-1 s=READY pgs=3333 cs=0 l=1 rev1=1 crypto rx=0x7f45b8073020 tx=0x7f45b000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45a8004400 con 0x7f45b806aac0 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 monclient: dump: 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.114 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f45a8004560 con 0x7f45b806aac0 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 msgr2=0x7f45b81ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 0x7f45b81ca400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45b8074d40 tx=0x7f45a8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 msgr2=0x7f45b81ca400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:14.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.091+0000 7f45adffb700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b806aac0 0x7f45b81ca400 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45b8074d40 tx=0x7f45a8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bf576700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bf576700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 0x7f45b81cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bf576700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f45b806a400 con 0x7f45b81ca940 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bcb11700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 0x7f45b81cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bcb11700 10 monclient: get_auth_request con 0x7f45b81ca940 auth_method 0 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bcb11700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bcb11700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.099+0000 7f45bcb11700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45bcb11700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45bcb11700 10 monclient(hunting): handle_auth_done global_id 84337 payload 824 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45bcb11700 1 -- 172.21.15.47:0/4220084470 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f45b80a3d60 con 0x7f45b81ca940 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 client.84370.objecter ms_handle_connect 0x7f45b81ca940 2024-04-20T02:31:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45bcb11700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 0x7f45b81cad50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45a8000c00 tx=0x7f45a8011000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 client.84370.objecter resend_mon_ops 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f45a802b070 con 0x7f45b81ca940 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 monclient: dump: 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.119 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f45a8026850 con 0x7f45b81ca940 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 msgr2=0x7f45b81cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 0x7f45b81cad50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45a8000c00 tx=0x7f45a8011000 comp rx=0 tx=0).stop 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 msgr2=0x7f45b81cad50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:14.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.100+0000 7f45adffb700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f45b81ca940 0x7f45b81cad50 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45a8000c00 tx=0x7f45a8011000 comp rx=0 tx=0).stop 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:14.121 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:14.122 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:14.123 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:14.124 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:14.125 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:14.127 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.130 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:14.132 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.133 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.134 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:14.137 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:14.138 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:14.139 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.140 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:14.141 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 10 librados: watch_flush enter 2024-04-20T02:31:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 10 librados: watch_flush exit 2024-04-20T02:31:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 20 client.84370.objecter shutdown clearing up homeless session... 2024-04-20T02:31:14.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 10 client.84370.objecter successfully canceled tick 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f459c03a320 msgr2=0x7f459c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f459c03a320 0x7f459c03c7e0 secure :-1 s=READY pgs=3333 cs=0 l=1 rev1=1 crypto rx=0x7f45b8073020 tx=0x7f45b000b040 comp rx=0 tx=0).stop 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 10 monclient: shutdown 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f45b809f9e0 msgr2=0x7f45b81c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 --2- 172.21.15.47:0/4220084470 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f45b809f9e0 0x7f45b81c6ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e840 tx=0x7f45b400ec00 comp rx=0 tx=0).stop 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bdb13700 1 -- 172.21.15.47:0/4220084470 reap_dead start 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bdb13700 1 -- 172.21.15.47:0/4220084470 reap_dead start 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 shutdown_connections 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 >> 172.21.15.47:0/4220084470 conn(0x7f45b80a3000 msgr2=0x7f45b81bdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 shutdown_connections 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 -- 172.21.15.47:0/4220084470 wait complete. 2024-04-20T02:31:14.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.101+0000 7f45bf576700 1 librados: shutdown 2024-04-20T02:31:14.145 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:14.145 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:31:14.145 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:31:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.297+0000 7f0a673ea700 10 monclient: get_monmap_and_config 2024-04-20T02:31:14.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.297+0000 7f0a673ea700 10 monclient: build_initial_monmap 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.297+0000 7f0a673ea700 10 monclient: monmap: 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:14.298781+0000 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:14.298781+0000 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:14.298 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:14.299 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:14.299 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 1 Processor -- start 2024-04-20T02:31:14.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 1 -- start start 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 10 monclient: init 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 10 monclient: _add_conns ranks=[0,4,7,9,2,8,1,3,10,5,6] 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0a60071660 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.298+0000 7f0a673ea700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a60137de0 0x7f0a6013cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 10 monclient(hunting): picked mon.noname-c con 0x7f0a60137de0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0a6013d150 0x7f0a6013f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 10 monclient(hunting): picked mon.noname-i con 0x7f0a6013d150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a600751e0 con 0x7f0a60071660 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6006ca10 con 0x7f0a60137de0 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6006e140 con 0x7f0a6013d150 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:14.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a673ea700 10 monclient(hunting): authenticate will time out at 2553.700057s 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65186700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a60071660 0x7f0a60071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37852/0 (socket says 172.21.15.47:37852) 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65186700 1 -- 172.21.15.47:0/432054600 learned_addr learned my addr 172.21.15.47:0/432054600 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a60137de0 0x7f0a6013cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): get_auth_request con 0x7f0a60137de0 auth_method 0 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0a6013d150 0x7f0a6013f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): get_auth_request con 0x7f0a6013d150 auth_method 0 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a65987700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 1 -- 172.21.15.47:0/432054600 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830731265 0 0) 0x7f0a600751e0 con 0x7f0a60071660 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient(hunting): _negotiate 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient(hunting): my global_id is 36729 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 1 -- 172.21.15.47:0/432054600 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a4c0035b0 con 0x7f0a60071660 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient(hunting): handle_auth_done global_id 84379 payload 274 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 -- 172.21.15.47:0/432054600 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0a6013d150 msgr2=0x7f0a6013f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0a6013d150 0x7f0a6013f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 -- 172.21.15.47:0/432054600 >> v1:172.21.15.47:6789/0 conn(0x7f0a60071660 legacy=0x7f0a60071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient: _finish_hunting 0 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 monclient: found mon.noname-c 2024-04-20T02:31:14.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 -- 172.21.15.47:0/432054600 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6006e420 con 0x7f0a60137de0 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a64985700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a60137de0 0x7f0a6013cc10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009870 tx=0x7f0a50009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3c28b96dbaa9a4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 1 -- 172.21.15.47:0/432054600 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0a50011aa0 con 0x7f0a60137de0 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.299+0000 7f0a57fff700 10 monclient: dump: 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 10 monclient: _finish_auth 0 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 5 monclient: authenticate success, global_id 84379 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 1 -- 172.21.15.47:0/432054600 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a50011c00 con 0x7f0a60137de0 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 4 set_mon_vals no callback set 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 1 -- 172.21.15.47:0/432054600 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0a50011d70 con 0x7f0a60137de0 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a57fff700 10 monclient: dump: 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 10 monclient: shutdown 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 -- 172.21.15.47:0/432054600 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a60137de0 msgr2=0x7f0a6013cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a60137de0 0x7f0a6013cc10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009870 tx=0x7f0a50009b80 comp rx=0 tx=0).stop 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 -- 172.21.15.47:0/432054600 shutdown_connections 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0a6013d150 0x7f0a6013f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 --2- 172.21.15.47:0/432054600 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a60137de0 0x7f0a6013cc10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 -- 172.21.15.47:0/432054600 >> 172.21.15.47:0/432054600 conn(0x7f0a600a84a0 msgr2=0x7f0a600a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 -- 172.21.15.47:0/432054600 shutdown_connections 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.300+0000 7f0a673ea700 1 -- 172.21.15.47:0/432054600 wait complete. 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 10 monclient: build_initial_monmap 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 10 monclient: monmap: 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 1 librados: starting msgr at 2024-04-20T02:31:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 1 librados: starting objecter 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 1 Processor -- start 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 1 -- start start 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 1 librados: setting wanted keys 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 1 librados: calling monclient init 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.301+0000 7f0a673ea700 10 monclient: init 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient: _add_conns ranks=[5,2,6,1,8,0,4,7,3] 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0a60071660 0x7f0a6011e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): picked mon.noname-h con 0x7f0a60071660 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 0x7f0a6011ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): picked mon.noname-b con 0x7f0a60137de0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0a6013d150 0x7f0a6011ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): picked mon.noname-d con 0x7f0a6013d150 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a60120c80 con 0x7f0a60137de0 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a601205c0 con 0x7f0a60071660 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6006d3c0 con 0x7f0a6013d150 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a673ea700 10 monclient(hunting): authenticate will time out at 2553.703446s 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 0x7f0a6011ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0a6013d150 0x7f0a6011ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 0x7f0a6011ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36786/0 (socket says 172.21.15.47:36786) 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 -- 172.21.15.47:0/980576221 learned_addr learned my addr 172.21.15.47:0/980576221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): get_auth_request con 0x7f0a60137de0 auth_method 0 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0a60071660 0x7f0a6011e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): get_auth_request con 0x7f0a6013d150 auth_method 0 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): get_auth_request con 0x7f0a60071660 auth_method 0 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65987700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a65186700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient(hunting): handle_auth_done global_id 24734 payload 824 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 -- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0a6013d150 msgr2=0x7f0a6011ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0a6013d150 0x7f0a6011ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 -- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0a60071660 msgr2=0x7f0a6011e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0a60071660 0x7f0a6011e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient: _finish_hunting 0 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 monclient: found mon.noname-b 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:14.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.302+0000 7f0a64985700 1 -- 172.21.15.47:0/980576221 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500094a0 con 0x7f0a60137de0 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a64985700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 0x7f0a6011ea00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f0a500090b0 tx=0x7f0a50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 client.?.objecter ms_handle_connect 0x7f0a60137de0 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0a50004410 con 0x7f0a60137de0 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: dump: 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.324 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 5 monclient: authenticate success, global_id 24734 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5001f070 con 0x7f0a60137de0 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0a500045d0 con 0x7f0a60137de0 2024-04-20T02:31:14.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 10 monclient: _renew_subs 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6013c5d0 con 0x7f0a60137de0 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a577fe700 4 set_mon_vals no callback set 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a55ffb700 10 monclient: dump: 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:14.326 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 10 client.24734.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 10 monclient: _renew_subs 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6013be50 con 0x7f0a60137de0 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.303+0000 7f0a673ea700 1 librados: init done 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.304+0000 7f0a673ea700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.304+0000 7f0a673ea700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.304+0000 7f0a673ea700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:14.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.304+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a60120ae0 con 0x7f0a60137de0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0a500036a0 con 0x7f0a60137de0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0a4403a2f0 0x7f0a4403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(601..601 src has 266..601) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f0a50021070 con 0x7f0a60137de0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 10 client.24734.objecter ms_dispatch 0x7f0a60001550 osd_map(601..601 src has 266..601) v4 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 3 client.24734.objecter handle_osd_map got epochs [601,601] > 0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 3 client.24734.objecter handle_osd_map decoding full epoch 601 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a55ffb700 20 client.24734.objecter dump_active .. 0 homeless 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a65186700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0a4403a2f0 0x7f0a4403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.305+0000 7f0a65186700 10 monclient: get_auth_request con 0x7f0a4403a2f0 auth_method 0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.306+0000 7f0a55ffb700 10 client.24734.objecter ms_handle_connect 0x7f0a4403a2f0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.306+0000 7f0a65186700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0a4403a2f0 0x7f0a4403c7b0 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c007fd0 tx=0x7f0a5c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.310+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0a5002f3a0 con 0x7f0a60137de0 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.310+0000 7f0a55ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:14.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.310+0000 7f0a55ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:14.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.619+0000 7f0a673ea700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:31:14.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.620+0000 7f0a673ea700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:31:14.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.620+0000 7f0a673ea700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:14.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.620+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f0a60120ae0 con 0x7f0a60137de0 2024-04-20T02:31:14.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.621+0000 7f0a55ffb700 1 -- 172.21.15.47:0/980576221 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f0a60120ae0 con 0x7f0a60137de0 2024-04-20T02:31:14.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.621+0000 7f0a55ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:31:14.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.621+0000 7f0a55ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=12,kv=10,logm=60} crc {auth=3378291304,config=692619927,health=1628097280,kv=3003869462,logm=327779224}) 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=17,config=2,health=12,kv=11,logm=58} crc {auth=147632553,config=692619927,health=1628097280,kv=1530195301,logm=8333304}) 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.627 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=11,logm=58} crc {auth=2652010395,config=692619927,health=1628097280,kv=1530195301,logm=2981793282}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=18,config=2,health=12,kv=9,logm=59} crc {auth=1846043071,config=692619927,health=1628097280,kv=2700652957,logm=3142842014}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=2336531876,config=692619927,health=2173695685,kv=1530195301,logm=3782215511}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.628 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=15,config=2,health=10,kv=11,logm=62} crc {auth=2566183789,config=692619927,health=903489274,kv=1530195301,logm=2390312559}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=17,config=2,health=12,kv=11,logm=58} crc {auth=2336531876,config=692619927,health=1628097280,kv=1530195301,logm=3829842466}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=11,kv=11,logm=59} crc {auth=1810511646,config=692619927,health=2835044255,kv=1530195301,logm=3156631099}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.622+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=1478174646,config=692619927,health=1628097280,kv=105211281,logm=1441356360}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2595547880}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.629 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3954609146}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.624+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2581784679}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=564123444}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3787806966}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.630 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823140399}) 2024-04-20T02:31:14.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2204355567}) 2024-04-20T02:31:14.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569656580}) 2024-04-20T02:31:14.631 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.625+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622692881}) 2024-04-20T02:31:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 10 librados: watch_flush enter 2024-04-20T02:31:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 10 librados: watch_flush exit 2024-04-20T02:31:14.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 20 client.24734.objecter shutdown clearing up homeless session... 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 10 client.24734.objecter successfully canceled tick 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0a4403a2f0 msgr2=0x7f0a4403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0a4403a2f0 0x7f0a4403c7b0 secure :-1 s=READY pgs=3334 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c007fd0 tx=0x7f0a5c009040 comp rx=0 tx=0).stop 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 10 monclient: shutdown 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 msgr2=0x7f0a6011ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.626+0000 7f0a673ea700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 0x7f0a6011ea00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f0a500090b0 tx=0x7f0a50004750 comp rx=0 tx=0).stop 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 shutdown_connections 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0a4403a2f0 0x7f0a4403c7b0 unknown :-1 s=CLOSED pgs=3334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0a6013d150 0x7f0a6011ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0a60071660 0x7f0a6011e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 --2- 172.21.15.47:0/980576221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a60137de0 0x7f0a6011ea00 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:14.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 >> 172.21.15.47:0/980576221 conn(0x7f0a600a84a0 msgr2=0x7f0a60136e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:14.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 shutdown_connections 2024-04-20T02:31:14.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 -- 172.21.15.47:0/980576221 wait complete. 2024-04-20T02:31:14.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:14.627+0000 7f0a673ea700 1 librados: shutdown 2024-04-20T02:31:14.656 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2567187784}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3812047396}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=133731689}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=854279698}) 2024-04-20T02:31:14.657 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2790188139}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=423891385}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1492276952}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.627+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.628+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2998903245}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.628+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3421997193}) 2024-04-20T02:31:14.658 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355092615}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2447246594}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.630+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1752354706}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3331389969}) 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.659 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2240979525}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2270310380}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=413406264}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889183494}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.631+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3702069586}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.660 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2161752416}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4019316414}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3395700275}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1936854778}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3504635237}) 2024-04-20T02:31:14.661 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2766446949}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=377282705}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142126160}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.634+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3555483017}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2916665216}) 2024-04-20T02:31:14.662 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3507800688}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=378306935}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3878470048}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1194727750}) 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.663 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=751407508}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1894481183}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1480217406}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.636+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3246880047}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674152240}) 2024-04-20T02:31:14.664 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=91240738}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=545606659}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3947785677}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.665 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4048844495}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=47576964}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2986560977}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2016931442}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.638+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3724532733}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.639+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3429465371}) 2024-04-20T02:31:14.666 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2514727438}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3690866371}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1692075894}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1852222594}) 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.667 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=853400675}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1504408358}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079963702}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.640+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=572145917}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.641+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.641+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2944898561}) 2024-04-20T02:31:14.668 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1090450742}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3507209162}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1242333547}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1433459976}) 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.669 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3275168090}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=687328414}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3450295150}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.642+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1058701492}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.643+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.643+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.643+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1803180820}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.643+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.670 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2913542363}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1351106962}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4174129605}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1581146275}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1930426874}) 2024-04-20T02:31:14.671 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=543969678}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913417929}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.644+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933898753}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3106646880}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1537436848}) 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.672 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=101264363}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1374878647}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3361884168}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1571355497}) 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.673 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4221867779}) 2024-04-20T02:31:14.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3819548691}) 2024-04-20T02:31:14.674 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.646+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2202319280}) 2024-04-20T02:31:14.675 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3845763175}) 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2621392990}) 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.976 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=654325782}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=64247441}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3844680608}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2631198518}) 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.977 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=785349611}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899490030}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.649+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4277536066}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1361396774}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=637269923}) 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.978 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2942442908}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.650+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1048972905}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=940646065}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1262029634}) 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.979 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2980036444}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=947780867}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.651+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4145202586}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=940998846}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4280581449}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.980 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1146040509}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1918147076}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=750134463}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1655740014}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.981 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1353975432}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330195574}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.661+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3645456340}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1358535593}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=613591304}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.982 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3142242433}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1545063322}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3375412689}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2539080610}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.983 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1583701980}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2102848184}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.662+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2484460379}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3694292843}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2764079575}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.984 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=762642137}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3858082688}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3601072488}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=979771799}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2740672919}) 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.985 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2527353583}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.664+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4119345649}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4179448478}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4182960090}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1016024077}) 2024-04-20T02:31:14.986 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4187768806}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.665+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3312790563}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2402091940}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2457061563}) 2024-04-20T02:31:14.987 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2874654391}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.666+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1822403123}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1189964856}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734664855}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4141966316}) 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.988 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=828743363}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2087616601}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1205614211}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3412777008}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1683615726}) 2024-04-20T02:31:14.989 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.667+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3138951107}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=413730454}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1244767743}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3364984207}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3747750415}) 2024-04-20T02:31:14.990 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3007150993}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=781951428}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3345604221}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=322880944}) 2024-04-20T02:31:14.991 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.669+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4148087250}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1264508507}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3230120813}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1820886258}) 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.992 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3754274677}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3837081719}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2082316495}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=23177198}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1863502082}) 2024-04-20T02:31:14.993 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.670+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=199268498}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2569433791}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3670277730}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2367857710}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.994 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=273551810}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273056761}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3182668555}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3092620788}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430482314}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.672+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4248524976}) 2024-04-20T02:31:14.995 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029276279}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106518934}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2098353225}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3926740171}) 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.996 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1439226582}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3761973105}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1629451858}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=880766399}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.673+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2769528541}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.997 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3599454141}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1063046704}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=139932958}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=498000697}) 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.998 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1221252969}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3416245021}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3747907214}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=489449288}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.675+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1040941284}) 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:14.999 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=980215000}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3550720245}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=789399721}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1795845548}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.000 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=297371765}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1884477849}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1314304115}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1941009575}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.677+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=780625090}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.001 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=601168254}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=498617653}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2009444659}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=90725620}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.002 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3998794040}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1890688926}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3667535920}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853847731}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.678+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1055453735}) 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.003 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3946112418}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3068061538}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2893341856}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=52844905}) 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.004 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=993944071}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=644908948}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1841021534}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1127260818}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.680+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1125982280}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.005 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=569499858}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3090781728}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=6446120}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=837402493}) 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.006 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2903787764}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1817957895}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1651738000}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2246789941}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.681+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3659484959}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.007 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3396016127}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3350943886}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1626076775}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3024652920}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.008 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1874062711}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1755846354}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=248260961}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715163100}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.683+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2279490207}) 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.009 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2450321669}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3028737690}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1082231371}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3623781449}) 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.010 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1567073990}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1918774415}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2214268051}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.684+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664280459}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.685+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2044052907}) 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.011 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1386446623}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3449958832}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=323354154}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1852581964}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.012 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2556315855}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2766560053}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4007632791}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3249306262}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.686+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3972355763}) 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.013 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1140470685}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=925547709}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=570870835}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4162865602}) 2024-04-20T02:31:15.014 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=254610964}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3370433079}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.687+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=35510676}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179641719}) 2024-04-20T02:31:15.015 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.688+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1451927149}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3423472208}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3870520349}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=609170514}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.016 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4085849195}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3909831715}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3654898955}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1303642557}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448586441}) 2024-04-20T02:31:15.017 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.689+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=141570805}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.690+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366840891}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416832112}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2003020269}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.018 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=857664050}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2647278203}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1348324105}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2280470187}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1773380097}) 2024-04-20T02:31:15.019 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.691+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=663950222}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796377115}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2505967801}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=41980960}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.020 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3082199360}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1427196255}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.692+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1367531635}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.693+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.693+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.693+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4198974958}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.693+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.693+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750622375}) 2024-04-20T02:31:15.021 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.693+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1585574958}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2657500264}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=212688070}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2242664210}) 2024-04-20T02:31:15.022 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=260991457}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1652496876}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570067387}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4043911767}) 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.023 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389299374}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.694+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3323947507}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=98228172}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2745942420}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2859833447}) 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.024 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3974395692}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115631354}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3709915632}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1803937297}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=715581377}) 2024-04-20T02:31:15.025 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.696+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1851554180}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=936260734}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1865352985}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2413988262}) 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.026 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1735128523}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=718569100}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=939432660}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.698+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.699+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.699+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3594025635}) 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.699+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.027 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.699+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2712139199}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.699+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2164923589}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4098610329}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=851915044}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2708902615}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.028 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=805695791}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=30103739}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261749279}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4237316465}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3098386693}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.700+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3378273617}) 2024-04-20T02:31:15.029 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=394357205}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2729556706}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2355015686}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2598925957}) 2024-04-20T02:31:15.030 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3798570716}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=487315791}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3026096394}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=596270}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.702+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=789031019}) 2024-04-20T02:31:15.031 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2262960452}) 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=134258271}) 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=308926166}) 2024-04-20T02:31:15.032 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3501810343}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2227191048}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4210129074}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2305749914}) 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.703+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.033 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.704+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514234658}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.704+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=563139502}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=962744876}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3119558445}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2213389141}) 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.034 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3955080277}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2594950038}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2742393846}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1116769404}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1014237381}) 2024-04-20T02:31:15.035 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.705+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1100895752}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.706+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.706+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.706+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2156578980}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3561055519}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2088368134}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.036 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2907564316}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3996671154}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1014189650}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3604113823}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3855321428}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.707+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911305276}) 2024-04-20T02:31:15.037 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=273747861}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=774177165}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=764222204}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.038 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2154994764}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283102710}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1422929102}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.708+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.709+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.709+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3892475124}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.709+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.709+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3062446945}) 2024-04-20T02:31:15.039 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.709+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=940594904}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3566588778}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=878191271}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1746737395}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1207876001}) 2024-04-20T02:31:15.040 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=238903820}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3741482770}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=622777972}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2175503715}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.710+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=844521172}) 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.041 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1141402938}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1738336859}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1159183775}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3432445944}) 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.042 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3716969652}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1599055411}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=225252375}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3023086627}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.712+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2565734906}) 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.043 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1886787589}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1085959911}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2650420644}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3671266492}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.044 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1773155993}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.713+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3569386094}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2176756483}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3840178542}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.714+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=657687860}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432331801}) 2024-04-20T02:31:15.045 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3440138137}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=219141153}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2204117734}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.046 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1198951559}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1844608616}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2237386513}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3182479930}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.715+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3933023564}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1877618343}) 2024-04-20T02:31:15.047 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2163758163}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4022901805}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3185934608}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=710215418}) 2024-04-20T02:31:15.048 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1490448608}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3593996138}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3193201294}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.717+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2132336964}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=172855445}) 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.049 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1881263908}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3680467005}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1641390983}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.718+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1002357272}) 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.050 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4143310703}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=56876835}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25165118}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.719+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3550949673}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1017128883}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.051 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126703313}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.720+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=652546905}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3105071621}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=547593492}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.052 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=837216772}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3227207514}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667243962}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.721+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2558332229}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.722+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.722+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.722+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=746592665}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.722+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.053 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4169465618}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1047982113}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2236688376}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3064254440}) 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.054 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3909243633}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1522617574}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4082054724}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.723+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2152535932}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.724+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.724+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1864419}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.055 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3989529235}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3901337852}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=181189227}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2288262572}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.056 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2429446508}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3414489507}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4232307248}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.725+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1560997245}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.726+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.726+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.726+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3542326461}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.057 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1296213792}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3219060544}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4068542411}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1761689670}) 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.058 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4195544034}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2445285712}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=689510393}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.727+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2179664179}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965316045}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.059 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2469292329}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3567044246}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2227429915}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2026925922}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.060 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2777526936}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=787762998}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1770883551}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.729+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1225946252}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2452740748}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.061 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=535927873}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=981772308}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2816940926}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4287274540}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.730+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2594841727}) 2024-04-20T02:31:15.062 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1644711461}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060820529}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.731+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=762838844}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2427658666}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38641180}) 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.063 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4161437144}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3387983288}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1271605356}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3468183372}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.064 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3879315231}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1170047151}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.732+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2474631653}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.733+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.733+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.733+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1113133146}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.733+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4125158661}) 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.065 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2637069163}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=899431944}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2620433651}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2515397458}) 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.066 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=573750794}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1940387595}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.734+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=943697965}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4156324852}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1360906220}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.735+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.067 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=186239118}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=451010393}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917996863}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4006854319}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.068 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2202899242}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066901709}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.736+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2285776005}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.737+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3299877240}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3760159595}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1674166700}) 2024-04-20T02:31:15.069 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2694111211}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2154871109}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3357444207}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.070 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1973880682}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3247951351}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.738+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3636478451}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.739+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194810208}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4176222652}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.071 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3798963912}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=223107798}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3030518737}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1233891087}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4226119684}) 2024-04-20T02:31:15.072 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=224259077}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.740+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2685221566}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.741+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.741+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.741+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3902356534}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.741+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3019619972}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1154843397}) 2024-04-20T02:31:15.073 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3118836460}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2327046991}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2222955104}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=820111777}) 2024-04-20T02:31:15.074 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=910980581}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.742+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1126063735}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3632541390}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5655728}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3069367861}) 2024-04-20T02:31:15.075 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=333317001}) 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1335650150}) 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2938230454}) 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.076 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=322034064}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3833404488}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.744+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3750577125}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3667883235}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030348121}) 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.077 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=369356065}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2489478242}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1839166813}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=110833306}) 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.078 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1044801105}) 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3125312828}) 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.746+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2782339599}) 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2827707982}) 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.079 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2860458459}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3731878088}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2348828597}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37555126}) 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.080 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=373805006}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2335457984}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.748+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4129007792}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.749+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4230163056}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054360172}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.081 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3803574350}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=442005070}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1809908107}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4083258867}) 2024-04-20T02:31:15.082 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3901188720}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1114116858}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.750+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.751+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=396204160}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.751+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=886837163}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.083 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3425111287}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3825858743}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1954909172}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1837605093}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2667185195}) 2024-04-20T02:31:15.084 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3952789362}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.752+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1130481258}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.753+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.753+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141692154}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.753+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1690679926}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3895310167}) 2024-04-20T02:31:15.085 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519462863}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2595655822}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2663671730}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=297346894}) 2024-04-20T02:31:15.086 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1404621888}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.754+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3615713211}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.755+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.755+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773898292}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.755+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=15902800}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.087 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=431108612}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1148066338}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3988045270}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1838256572}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.088 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1741502907}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3928129704}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=138798767}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.756+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.757+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231102862}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.757+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3458025856}) 2024-04-20T02:31:15.089 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1511606783}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=306259270}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3730637278}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.090 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=970169699}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3483063305}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2413537479}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1700735971}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384049306}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.758+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2288031254}) 2024-04-20T02:31:15.091 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.759+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.759+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3757044526}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2472323117}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3633262560}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2192814580}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.092 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3641134002}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344133493}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=858831915}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=498799184}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.760+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=820063312}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.093 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3359252051}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4027822726}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2428133103}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1307216532}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.094 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791433134}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3859259150}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3890154345}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3041455953}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.761+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1595421134}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=716417695}) 2024-04-20T02:31:15.095 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176927060}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2774607467}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2434001384}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1612882314}) 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.096 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=943831518}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3620408562}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3773003388}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.763+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3690222641}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=868447121}) 2024-04-20T02:31:15.097 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=303365060}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=474474363}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4018731047}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1081115475}) 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.098 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1204856006}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2305943343}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1178929597}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.765+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1198483971}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092895676}) 2024-04-20T02:31:15.099 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=662176485}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.766+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1633706803}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3045201278}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=17891754}) 2024-04-20T02:31:15.100 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2275235018}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2525999872}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4284111395}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.767+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2593564359}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4007214369}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.101 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1337331443}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2920282387}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3293724054}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4277626804}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.102 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3202238453}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1150319111}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136232892}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.768+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3966451246}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1001432471}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1806896127}) 2024-04-20T02:31:15.103 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3442363338}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2368225273}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=108675191}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3486427656}) 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.104 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2962422415}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1559912713}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.770+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=291818379}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1418329198}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3912815807}) 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.105 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4076996224}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1692874176}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.771+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3371715971}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=868227345}) 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.106 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1802642112}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1570551691}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.772+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1570019233}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2285408668}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=917588255}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.107 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=511325673}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1590267828}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4236063153}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3391666777}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=716621127}) 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.108 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4070899918}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.773+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2425129339}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4070335296}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3805660880}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3126392165}) 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.109 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1750252405}) 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=881788275}) 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2061635222}) 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.110 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2133100848}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575743543}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.775+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1768652359}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3771325105}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082929400}) 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.111 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=845867605}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1591713436}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2499825170}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117514976}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.112 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1337366153}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3425407964}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.777+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=904245260}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1204758603}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2612103300}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.113 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2823810429}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2544700853}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=462146322}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1853492689}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.114 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3420804115}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3290904945}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.779+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1531430985}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4223434358}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580513775}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.115 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1692979152}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1130442643}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2423496339}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.780+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=287308919}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3050085984}) 2024-04-20T02:31:15.116 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290675301}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.781+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=276225094}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1424747218}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2730921482}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2439650255}) 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.117 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1410495543}) 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=417908903}) 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3520677385}) 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.118 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2140490719}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317950109}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.782+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1852134100}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.783+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.783+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.783+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4139060855}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3047297274}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.119 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2423092324}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3620885258}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1379226799}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=980398830}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3880712821}) 2024-04-20T02:31:15.120 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1961501942}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.784+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2673788695}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4201250550}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=433353672}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3884150733}) 2024-04-20T02:31:15.121 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=42615420}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2292559090}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=815317468}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.785+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2386505101}) 2024-04-20T02:31:15.122 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.786+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.786+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3408274561}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.786+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1219563651}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3672530709}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2381712913}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2921825031}) 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.123 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1712448425}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4058574868}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2992318482}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3739105581}) 2024-04-20T02:31:15.124 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=842440505}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.787+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1899680542}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.788+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.788+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3283630439}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1174585958}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3724174819}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.125 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1468954121}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=652689049}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4284995498}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2019113898}) 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.126 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2482455343}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.789+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=209312413}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2402484345}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38505145}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=865737865}) 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.127 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2996346600}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4196909035}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1819586996}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.790+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3723384118}) 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.791+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.128 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.791+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1180779766}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.791+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=128505328}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4138331413}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1240813883}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2970983119}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.792+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.129 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2652451305}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=538600970}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1974756465}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=607280732}) 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.130 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3291157765}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.793+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=376196761}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2206903728}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2359546086}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=302756327}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.131 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4273424614}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.794+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=110661490}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3551108127}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1838687040}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1340680513}) 2024-04-20T02:31:15.132 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.795+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3020246754}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3309157683}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3041081055}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1014169595}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.796+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.133 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1806687416}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=374785260}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=249718393}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=203853097}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240015707}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.797+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=770368752}) 2024-04-20T02:31:15.134 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3233621962}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2392743539}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3722990794}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1021044942}) 2024-04-20T02:31:15.135 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3222553460}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.798+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3601406196}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.799+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.799+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.799+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=318261433}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.799+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.799+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1727796899}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.799+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1413756280}) 2024-04-20T02:31:15.136 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=604552027}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2800818817}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2482817421}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1067186933}) 2024-04-20T02:31:15.137 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.800+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2544176682}) 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3744476428}) 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2409173873}) 2024-04-20T02:31:15.138 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3410346380}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.801+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779700727}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3198483786}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1060259056}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1985437484}) 2024-04-20T02:31:15.139 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2665203740}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=952114645}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1738285309}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3962489570}) 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.802+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.140 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.803+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038810183}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.803+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4021755951}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1476268426}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3546396001}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3438762943}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.141 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3736247022}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3756048860}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3392995620}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3758282230}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2944761504}) 2024-04-20T02:31:15.142 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.804+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=369541203}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1504363724}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2781160661}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3715053691}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1865455680}) 2024-04-20T02:31:15.143 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=735457626}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2855913281}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3558656201}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3975247756}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.806+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2424734464}) 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.807+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.144 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.807+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.807+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3520560890}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.807+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.807+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.807+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2206436458}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=352508177}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2689104259}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.145 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2708284622}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3568637159}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=982836920}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174147262}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.808+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4054720438}) 2024-04-20T02:31:15.146 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=71699330}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=569057336}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=921389402}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1063829410}) 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.147 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.809+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3607464687}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3393848180}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1377987273}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3307581132}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.810+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3760204439}) 2024-04-20T02:31:15.148 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=527322320}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=396909003}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4148054643}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2002086023}) 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.149 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=769234273}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=610479904}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2253303566}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1254317502}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.811+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=887559862}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.150 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3254057847}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=423174535}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2040364816}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3377011985}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.151 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2409121531}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3361720525}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1007402798}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3267345553}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.813+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1692122353}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.152 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2721587012}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1356033678}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2666628535}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3014543895}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1417707245}) 2024-04-20T02:31:15.153 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.815+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3252762082}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3421342537}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2217735226}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.816+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1473373932}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.154 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3750434278}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2755161843}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1927315894}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4188036495}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.155 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=21491062}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1003554913}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1539947442}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107321236}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.817+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3048524040}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1375571929}) 2024-04-20T02:31:15.156 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017846400}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1520022162}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=220289002}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1740564011}) 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.157 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=844339473}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4030941345}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379577838}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.819+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2643020102}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3624954911}) 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.158 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2892514677}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3661151968}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.820+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=881986314}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=896868792}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.159 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4135846495}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2151492618}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2800980618}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.821+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=173368278}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4250417182}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=656790840}) 2024-04-20T02:31:15.160 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=341365530}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2554591102}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2213188778}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4291071139}) 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.161 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1584885193}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3253739833}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.822+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=595338337}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=121540958}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2867722591}) 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.162 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=289573372}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=203238801}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3570589334}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=151387910}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.163 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2442916289}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825112444}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.824+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2033862995}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.825+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.825+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422009462}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1460625369}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4264209919}) 2024-04-20T02:31:15.164 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3218177321}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1232435300}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2619810267}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=405510486}) 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.165 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=403365158}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.826+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1095193902}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=942541255}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=139748602}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1484975725}) 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.166 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2911896626}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=211270177}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.827+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927786672}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1720014117}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940275735}) 2024-04-20T02:31:15.167 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.828+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693318697}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1765580585}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2738771869}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=296007054}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=164755649}) 2024-04-20T02:31:15.168 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2666193882}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1312920662}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2244222695}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2416456924}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.829+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4053914268}) 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.169 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2714417585}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=591085371}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3686177159}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3472427444}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.170 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2773486323}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4153814096}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3944241211}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3302549685}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.831+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1082264558}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4000132710}) 2024-04-20T02:31:15.171 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3955967723}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.832+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=35170753}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2113206254}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=251101552}) 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.172 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292533509}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=220158624}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=49916980}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.833+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140039644}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=364920665}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.173 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=456577559}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2656102228}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1799182443}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=82741050}) 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.174 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.834+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2112991334}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.835+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.835+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.835+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2816053053}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.835+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.835+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=320874671}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.835+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2015945184}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3064331211}) 2024-04-20T02:31:15.175 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4021330790}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1314263255}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1459287544}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3265906886}) 2024-04-20T02:31:15.176 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3062399437}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3450813439}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472545214}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.836+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1201847977}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2514979707}) 2024-04-20T02:31:15.177 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=998648154}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1846646526}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1488492745}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2634056624}) 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.178 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1431171264}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3592138227}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.838+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3798474947}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.839+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3708462612}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4040098573}) 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.179 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=931756536}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4258196767}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=179526541}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=713434011}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2280387704}) 2024-04-20T02:31:15.180 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.840+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.841+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3386737730}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.841+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.841+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742658892}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.841+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682737353}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2859785775}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274065412}) 2024-04-20T02:31:15.181 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=480505942}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1010119075}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3322866773}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=268137168}) 2024-04-20T02:31:15.182 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=682974612}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3594358783}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.842+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1542490317}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=887071070}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3706225860}) 2024-04-20T02:31:15.183 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3196783324}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2082453707}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3437608587}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1178557828}) 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.184 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3588329533}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1589657314}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.844+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=892120291}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2838775671}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1250549712}) 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.185 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2181172046}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.845+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1990915548}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457353006}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1154723068}) 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.186 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1546110723}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37375134}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.846+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=615157175}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4060721201}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2432419135}) 2024-04-20T02:31:15.187 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=841603757}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3120224278}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3716205521}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3164038032}) 2024-04-20T02:31:15.188 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3850194913}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.847+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1364325559}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.848+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3358405792}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2205514416}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2766274493}) 2024-04-20T02:31:15.189 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2908928849}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3159461623}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=268216242}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.190 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2873703391}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1079545685}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=744368650}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.850+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=897969795}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1286359703}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.191 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2864459007}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=721315114}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=345273773}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3600921622}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.192 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3406434596}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.852+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.853+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2647681375}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.853+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.853+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324865886}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.853+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2384682586}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1630807773}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3952790880}) 2024-04-20T02:31:15.193 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2728662095}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=430107836}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2373196868}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2493817043}) 2024-04-20T02:31:15.194 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4090644042}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2434617460}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.855+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=208086820}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68826985}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=423176104}) 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.195 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3140934709}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1517645062}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3171981254}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1683923089}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.196 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3879897630}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2027408146}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.857+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3729080491}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3042684346}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328375286}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.858+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.197 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4252554183}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3293190001}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2302707132}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1880383018}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.198 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3267895408}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2134673372}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.859+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2991442186}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=60576732}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2581126246}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.199 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3274768081}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=879157387}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2807679504}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.860+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4082396896}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.861+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.861+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.861+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1824796191}) 2024-04-20T02:31:15.200 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.861+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.861+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989356255}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.861+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2615199568}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4046988698}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4019669290}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1723708502}) 2024-04-20T02:31:15.201 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=633316493}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1820619886}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3112990151}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.862+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1702717989}) 2024-04-20T02:31:15.202 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.863+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.863+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3766691127}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.863+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3176125903}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1670286389}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=704025433}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3534074334}) 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.203 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1724938383}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3622963477}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3032151779}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2031059272}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=161409758}) 2024-04-20T02:31:15.204 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.864+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3559764341}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1991195130}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1406366781}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=174073040}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.205 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=813966397}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3819893221}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3929013511}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1533090356}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1783896941}) 2024-04-20T02:31:15.206 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.866+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3373124333}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1115059140}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1652703837}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1935857817}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.867+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3641660664}) 2024-04-20T02:31:15.207 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3553428678}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3882857559}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=54554839}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183398049}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.868+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2648471824}) 2024-04-20T02:31:15.208 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2108212891}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1687489049}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=894422313}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1543637570}) 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.209 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3394819221}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4268335969}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=265066994}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586677018}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.869+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=393022796}) 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.210 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=551237307,mdsmap=1531151339,mgr=3609017154}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=2815283758,mdsmap=1531151339}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.870+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=2348222002,mdsmap=1531151339,mgr=899648751}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=846069462,mdsmap=2826658659}) 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.211 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=3645733535,mdsmap=1531151339,mgr=2487754059}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=94,mdsmap=3,mgr=3} crc {logm=1398736718,mdsmap=1531151339,mgr=2487754059}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=34327119,mdsmap=3585761405}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3559332408,mdsmap=1531151339}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.871+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=4075491482,mdsmap=1531151339}) 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.212 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=38,osdmap=24} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4239401579,mon_config_key=2868358642,monmap=3385036551,osd_metadata=1493347452,osd_pg_creating=3755012269,osd_snap=2952527711,osdmap=58087211}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=40,osdmap=23} crc {mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4239401579,mon_config_key=2868358642,monmap=3385036551,osd_metadata=1881248415,osd_pg_creating=3755012269,osd_snap=3068482293,osdmap=1007986080}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=38,osdmap=24} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4239401579,mon_config_key=2888989162,monmap=3385036551,osd_metadata=1881248416,osd_pg_creating=3755012269,osd_snap=3969853831,osdmap=2063205319}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=9,monmap=3,osd_metadata=6,osd_snap=39,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2886876016,monmap=3385036551,osd_metadata=3566517105,osd_snap=1477224246,osdmap=188873204}) 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.213 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=40,osdmap=22} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4239401579,mon_config_key=2868358642,monmap=3385036551,osd_metadata=1493347452,osd_pg_creating=3755012269,osd_snap=3445240628,osdmap=1935571594}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.873+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=23} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2306111053,osdmap=1340192256}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10,monmap=3,osd_metadata=6,osd_snap=38,osdmap=25} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4239401579,mon_config_key=3064537996,monmap=3385036551,osd_metadata=3566517105,osd_snap=3992083473,osdmap=190310418}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=36,osdmap=26} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=530358493,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=2060791160,osdmap=3628909529}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.874+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=10,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=38,osdmap=25} crc {mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3650041901,mon_config_key=3064537996,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=3755012269,osd_snap=1855276820,osdmap=3034771367}) 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.214 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=112205152}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2146730467}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3909283597}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2687118391}) 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.215 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=679604216}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=713279655}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3822553749}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2916722383}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.876+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=321111765}) 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.216 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3463760433}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3999922497}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3397888143}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1268227279}) 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.217 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4044641763}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=545486742}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=415031160}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3114175514}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.878+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1022820023}) 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.218 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1798387596}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2467379869}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2220036115}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3695858995}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.219 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=44986088}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=734634436}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2998346151}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3228135712}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.880+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=944359679}) 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.220 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3667559892}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1878701083}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.883+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=542576149}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3550618478}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.221 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2112384698}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3656581456}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4186424918}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=529349293}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.884+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3771849943}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=1965019991}) 2024-04-20T02:31:15.222 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=80} crc {osdmap=4193191063}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=81} crc {osdmap=2260383820}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=76} crc {osdmap=2670574074}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=81} crc {osdmap=850587724}) 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.223 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=81} crc {osdmap=390949134}) 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=78} crc {osdmap=2751212539}) 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=82} crc {osdmap=3106276013}) 2024-04-20T02:31:15.224 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:31:14.895+0000 7fde5f0f7700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=76} crc {osdmap=2792997139}) 2024-04-20T02:31:15.675 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:31:15.676 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:31:15.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.871+0000 7f12dd9ef700 10 monclient: get_monmap_and_config 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.871+0000 7f12dd9ef700 10 monclient: build_initial_monmap 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.871+0000 7f12dd9ef700 10 monclient: monmap: 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:15.872632+0000 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:15.872632+0000 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:15.872 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 Processor -- start 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 -- start start 2024-04-20T02:31:15.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient: init 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient: _add_conns ranks=[7,10,3,5,4,1,8,6,9,2,0] 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 0x7f12d80a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): picked mon.noname-i con 0x7f12d80a4c80 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12d80a0d20 0x7f12d80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): picked mon.noname-f con 0x7f12d80a0d20 addr v2:172.21.15.179:3300/0 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 0x7f12d8137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): picked mon.noname-g con 0x7f12d80a55a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:15.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80a0000 con 0x7f12d80a0d20 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80a6a10 con 0x7f12d80a55a0 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f12d809e440 con 0x7f12d80a4c80 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12dd9ef700 10 monclient(hunting): authenticate will time out at 2555.273661s 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12d67fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12d80a0d20 0x7f12d80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 0x7f12d80a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.872+0000 7f12d77fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 0x7f12d8137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f12d80a0d20 0x7f12d80a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:34174/0 (socket says 172.21.15.47:34174) 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 0x7f12d8137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:47338/0 (socket says 172.21.15.47:47338) 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 -- 172.21.15.47:0/2287177776 learned_addr learned my addr 172.21.15.47:0/2287177776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:15.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 0x7f12d80a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:54074/0 (socket says 172.21.15.47:54074) 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): get_auth_request con 0x7f12d80a4c80 auth_method 0 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): get_auth_request con 0x7f12d80a0d20 auth_method 0 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): get_auth_request con 0x7f12d80a55a0 auth_method 0 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:15.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient(hunting): handle_auth_done global_id 55558 payload 274 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 -- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 msgr2=0x7f12d80a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 0x7f12d80a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 -- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 msgr2=0x7f12d8137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 0x7f12d8137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient: _finish_hunting 0 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 monclient: found mon.noname-f 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 -- 172.21.15.47:0/2287177776 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80751e0 con 0x7f12d80a0d20 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d6ffd700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 0x7f12d80a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d77fe700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 0x7f12d8137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d67fc700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d80a0d20 0x7f12d80a5060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0098f0 tx=0x7f12cc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0cc38e39c591d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 1 -- 172.21.15.47:0/2287177776 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12cc011aa0 con 0x7f12d80a0d20 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: dump: 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:15.889 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 1 -- 172.21.15.47:0/2287177776 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc011c00 con 0x7f12d80a0d20 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 1 -- 172.21.15.47:0/2287177776 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12cc011d70 con 0x7f12d80a0d20 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12d5ffb700 10 monclient: dump: 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12dd9ef700 5 monclient: authenticate success, global_id 55558 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12dd9ef700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.873+0000 7f12dd9ef700 4 set_mon_vals no callback set 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 10 monclient: shutdown 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 -- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d80a0d20 msgr2=0x7f12d80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d80a0d20 0x7f12d80a5060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0098f0 tx=0x7f12cc009c00 comp rx=0 tx=0).stop 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 -- 172.21.15.47:0/2287177776 shutdown_connections 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f12d80a55a0 0x7f12d8137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a4c80 0x7f12d80a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/2287177776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d80a0d20 0x7f12d80a5060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 -- 172.21.15.47:0/2287177776 >> 172.21.15.47:0/2287177776 conn(0x7f12d80aa280 msgr2=0x7f12d80aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 -- 172.21.15.47:0/2287177776 shutdown_connections 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.874+0000 7f12dd9ef700 1 -- 172.21.15.47:0/2287177776 wait complete. 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.875+0000 7f12dd9ef700 10 monclient: build_initial_monmap 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.875+0000 7f12dd9ef700 10 monclient: monmap: 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.875+0000 7f12dd9ef700 1 librados: starting msgr at 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.875+0000 7f12dd9ef700 1 librados: starting objecter 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.875+0000 7f12dd9ef700 1 Processor -- start 2024-04-20T02:31:15.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 -- start start 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 librados: setting wanted keys 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 librados: calling monclient init 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient: init 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient: _add_conns ranks=[2,6,7,1,5,4,3,8,0] 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 0x7f12d81cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): picked mon.noname-b con 0x7f12d80a0d20 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f12d80a4c80 0x7f12d8135a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): picked mon.noname-d con 0x7f12d80a4c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a55a0 0x7f12d8135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): picked mon.noname-i con 0x7f12d80a55a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d81cc910 con 0x7f12d80a0d20 2024-04-20T02:31:15.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8127190 con 0x7f12d80a4c80 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f12d809dd20 con 0x7f12d80a55a0 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.876+0000 7f12dd9ef700 10 monclient(hunting): authenticate will time out at 2555.277931s 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 0x7f12d81cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 0x7f12d81cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36810/0 (socket says 172.21.15.47:36810) 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 -- 172.21.15.47:0/1805150909 learned_addr learned my addr 172.21.15.47:0/1805150909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): get_auth_request con 0x7f12d80a0d20 auth_method 0 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f12d80a4c80 0x7f12d8135a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): get_auth_request con 0x7f12d80a4c80 auth_method 0 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a55a0 0x7f12d8135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): get_auth_request con 0x7f12d80a55a0 auth_method 0 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:15.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d67fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient(hunting): handle_auth_done global_id 24743 payload 824 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 -- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a55a0 msgr2=0x7f12d8135fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a55a0 0x7f12d8135fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 -- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f12d80a4c80 msgr2=0x7f12d8135a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f12d80a4c80 0x7f12d8135a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 monclient: found mon.noname-b 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 -- 172.21.15.47:0/1805150909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80094a0 con 0x7f12d80a0d20 2024-04-20T02:31:15.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d77fe700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a55a0 0x7f12d8135fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d6ffd700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 0x7f12d81cc3d0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f12c800ebe0 tx=0x7f12c800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 client.?.objecter ms_handle_connect 0x7f12d80a0d20 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12c8009800 con 0x7f12d80a0d20 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: dump: 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.899 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:15.900 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: _finish_auth 0 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800ccc0 con 0x7f12d80a0d20 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 5 monclient: authenticate success, global_id 24743 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12c800ce30 con 0x7f12d80a0d20 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 10 monclient: _renew_subs 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d0009a70 con 0x7f12d80a0d20 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12d57fa700 4 set_mon_vals no callback set 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 10 client.24743.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:15.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12bffff700 10 monclient: dump: 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:15.902 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 10 monclient: _renew_subs 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc009550 con 0x7f12d80a0d20 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.877+0000 7f12dd9ef700 1 librados: init done 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.878+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f12c800c850 con 0x7f12d80a0d20 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.878+0000 7f12dd9ef700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.878+0000 7f12dd9ef700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.878+0000 7f12dd9ef700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.878+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d81273d0 con 0x7f12d80a0d20 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f12c403a2f0 0x7f12c403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(602..602 src has 266..602) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f12c804a760 con 0x7f12d80a0d20 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 10 client.24743.objecter ms_dispatch 0x7f12d8001550 osd_map(602..602 src has 266..602) v4 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 3 client.24743.objecter handle_osd_map got epochs [602,602] > 0 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 3 client.24743.objecter handle_osd_map decoding full epoch 602 2024-04-20T02:31:15.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 20 client.24743.objecter dump_active .. 0 homeless 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12d67fc700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f12c403a2f0 0x7f12c403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12d67fc700 10 monclient: get_auth_request con 0x7f12c403a2f0 auth_method 0 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12d67fc700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f12c403a2f0 0x7f12c403c7b0 secure :-1 s=READY pgs=3335 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0098f0 tx=0x7f12cc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.879+0000 7f12bffff700 10 client.24743.objecter ms_handle_connect 0x7f12c403a2f0 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.884+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f12c80102b0 con 0x7f12d80a0d20 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.884+0000 7f12bffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:15.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:15.884+0000 7f12bffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.199+0000 7f12dd9ef700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.199+0000 7f12dd9ef700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.199+0000 7f12dd9ef700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.199+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f12d81250a0 con 0x7f12d80a0d20 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.200+0000 7f12bffff700 1 -- 172.21.15.47:0/1805150909 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7f12c8017c80 con 0x7f12d80a0d20 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.200+0000 7f12bffff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.200+0000 7f12bffff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:16.200 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.202+0000 7f12dd9ef700 10 librados: watch_flush enter 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 10 librados: watch_flush exit 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 20 client.24743.objecter shutdown clearing up homeless session... 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 10 client.24743.objecter successfully canceled tick 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f12c403a2f0 msgr2=0x7f12c403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f12c403a2f0 0x7f12c403c7b0 secure :-1 s=READY pgs=3335 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0098f0 tx=0x7f12cc008030 comp rx=0 tx=0).stop 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 10 monclient: shutdown 2024-04-20T02:31:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 msgr2=0x7f12d81cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 0x7f12d81cc3d0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f12c800ebe0 tx=0x7f12c800eef0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 shutdown_connections 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f12c403a2f0 0x7f12c403c7b0 unknown :-1 s=CLOSED pgs=3335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f12d80a55a0 0x7f12d8135fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12d80a0d20 0x7f12d81cc3d0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 --2- 172.21.15.47:0/1805150909 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f12d80a4c80 0x7f12d8135a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 >> 172.21.15.47:0/1805150909 conn(0x7f12d80aa280 msgr2=0x7f12d80a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 shutdown_connections 2024-04-20T02:31:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 -- 172.21.15.47:0/1805150909 wait complete. 2024-04-20T02:31:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.203+0000 7f12dd9ef700 1 librados: shutdown 2024-04-20T02:31:16.218 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:31:16.218 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:31:16.218 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:31:16.218 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.355+0000 7ff4795b0700 10 monclient: get_monmap_and_config 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.355+0000 7ff4795b0700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.356+0000 7ff4795b0700 10 monclient: monmap: 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:16.357123+0000 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:16.357123+0000 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:16.356 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:16.357 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.356+0000 7ff4795b0700 1 Processor -- start 2024-04-20T02:31:16.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.356+0000 7ff4795b0700 1 -- start start 2024-04-20T02:31:16.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.356+0000 7ff4795b0700 10 monclient: init 2024-04-20T02:31:16.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient: _add_conns ranks=[9,8,5,4,6,0,1,7,3,10,2] 2024-04-20T02:31:16.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 0x7ff4740a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): picked mon.noname-a con 0x7ff474137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 0x7ff4740a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): picked mon.noname-e con 0x7ff474138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff4741385b0 0x7ff4740a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): picked mon.noname-h con 0x7ff4741385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47409fa40 con 0x7ff474137d90 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff47409dfc0 con 0x7ff4741385b0 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff4740a7bc0 con 0x7ff474138170 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4795b0700 10 monclient(hunting): authenticate will time out at 2555.758222s 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 0x7ff4740a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 0x7ff4740a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 0x7ff4740a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55150/0 (socket says 172.21.15.47:55150) 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 0x7ff4740a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49808/0 (socket says 172.21.15.47:49808) 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 -- 172.21.15.47:0/2552234056 learned_addr learned my addr 172.21.15.47:0/2552234056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): get_auth_request con 0x7ff474138170 auth_method 0 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): get_auth_request con 0x7ff474137d90 auth_method 0 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4737fe700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff4741385b0 0x7ff4740a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4737fe700 10 monclient(hunting): get_auth_request con 0x7ff4741385b0 auth_method 0 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient(hunting): handle_auth_done global_id 35963 payload 274 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 -- 172.21.15.47:0/2552234056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff4741385b0 msgr2=0x7ff4740a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff4741385b0 0x7ff4740a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 -- 172.21.15.47:0/2552234056 >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 msgr2=0x7ff4740a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 --2- 172.21.15.47:0/2552234056 >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 0x7ff4740a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 monclient: found mon.noname-e 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 -- 172.21.15.47:0/2552234056 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740751e0 con 0x7ff474138170 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff472ffd700 1 --2- 172.21.15.47:0/2552234056 >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 0x7ff4740a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4737fe700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff4741385b0 0x7ff4740a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.357+0000 7ff4727fc700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 0x7ff4740a5c90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff468009920 tx=0x7ff468009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=baa4ea1157756a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 1 -- 172.21.15.47:0/2552234056 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff468011aa0 con 0x7ff474138170 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: dump: 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.371 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 1 -- 172.21.15.47:0/2552234056 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff468011c00 con 0x7ff474138170 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 5 monclient: authenticate success, global_id 35963 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:16.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 1 -- 172.21.15.47:0/2552234056 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff468011d70 con 0x7ff474138170 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 4 set_mon_vals no callback set 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff471ffb700 10 monclient: dump: 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.373 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 10 monclient: shutdown 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 -- 172.21.15.47:0/2552234056 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 msgr2=0x7ff4740a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 0x7ff4740a5c90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff468009920 tx=0x7ff468009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 -- 172.21.15.47:0/2552234056 shutdown_connections 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff4741385b0 0x7ff4740a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 --2- 172.21.15.47:0/2552234056 >> v2:172.21.15.47:3300/0 conn(0x7ff474137d90 0x7ff4740a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 --2- 172.21.15.47:0/2552234056 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff474138170 0x7ff4740a5c90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 -- 172.21.15.47:0/2552234056 >> 172.21.15.47:0/2552234056 conn(0x7ff4740aa8b0 msgr2=0x7ff4740a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 -- 172.21.15.47:0/2552234056 shutdown_connections 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.358+0000 7ff4795b0700 1 -- 172.21.15.47:0/2552234056 wait complete. 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.359+0000 7ff4795b0700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.359+0000 7ff4795b0700 10 monclient: monmap: 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.375 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.359+0000 7ff4795b0700 1 librados: starting msgr at 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.359+0000 7ff4795b0700 1 librados: starting objecter 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 Processor -- start 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 -- start start 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 librados: setting wanted keys 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 librados: calling monclient init 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient: init 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient: _add_conns ranks=[1,0,8,7,5,4,6,2,3] 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff474137d90 0x7ff4741cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): picked mon.noname-f con 0x7ff474137d90 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 0x7ff474136fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): picked mon.noname-a con 0x7ff474138170 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4741385b0 0x7ff4741c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): picked mon.noname-e con 0x7ff4741385b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474124a20 con 0x7ff474138170 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741cccd0 con 0x7ff474137d90 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff47409dc20 con 0x7ff4741385b0 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4795b0700 10 monclient(hunting): authenticate will time out at 2555.761690s 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 0x7ff474136fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 0x7ff474136fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49814/0 (socket says 172.21.15.47:49814) 2024-04-20T02:31:16.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 1 -- 172.21.15.47:0/1385048052 learned_addr learned my addr 172.21.15.47:0/1385048052 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): get_auth_request con 0x7ff474138170 auth_method 0 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4741385b0 0x7ff4741c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): get_auth_request con 0x7ff4741385b0 auth_method 0 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4727fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff472ffd700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff474137d90 0x7ff4741cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff4737fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff472ffd700 10 monclient(hunting): get_auth_request con 0x7ff474137d90 auth_method 0 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff472ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff472ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.360+0000 7ff472ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 10 monclient(hunting): handle_auth_done global_id 36747 payload 824 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4741385b0 msgr2=0x7ff4741c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4741385b0 0x7ff4741c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff474137d90 msgr2=0x7ff4741cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff474137d90 0x7ff4741cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 monclient: found mon.noname-a 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640094a0 con 0x7ff474138170 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4737fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff4741385b0 0x7ff4741c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4727fc700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 0x7ff474136fd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff468013000 tx=0x7ff468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 client.?.objecter ms_handle_connect 0x7ff474138170 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff468004490 con 0x7ff474138170 2024-04-20T02:31:16.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: dump: 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.381 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff468003b10 con 0x7ff474138170 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 5 monclient: authenticate success, global_id 36747 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 10 monclient: _renew_subs 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff468009550 con 0x7ff474138170 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff468003c80 con 0x7ff474138170 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:16.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff45b7fe700 10 monclient: dump: 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.383 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 10 client.36747.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4717fa700 4 set_mon_vals no callback set 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 10 monclient: _renew_subs 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4741225c0 con 0x7ff474138170 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.361+0000 7ff4795b0700 1 librados: init done 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff4795b0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff4795b0700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 0x7ff4741c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474068180 con 0x7ff47406aac0 2024-04-20T02:31:16.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff468021070 con 0x7ff474138170 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 0x7ff4741c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient: get_auth_request con 0x7ff47406aac0 auth_method 0 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46003a2f0 0x7ff46003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(603..603 src has 266..603) v4 ==== 6641+0+0 (secure 0 0 0) 0x7ff46804bc30 con 0x7ff474138170 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 10 client.36747.objecter ms_dispatch 0x7ff474001550 osd_map(603..603 src has 266..603) v4 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 3 client.36747.objecter handle_osd_map got epochs [603,603] > 0 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 3 client.36747.objecter handle_osd_map decoding full epoch 603 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 10 monclient(hunting): handle_auth_done global_id 36756 payload 824 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff47409f760 con 0x7ff47406aac0 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 20 client.36747.objecter dump_active .. 0 homeless 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff4737fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46003a2f0 0x7ff46003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff4737fe700 10 monclient: get_auth_request con 0x7ff46003a2f0 auth_method 0 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff472ffd700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 0x7ff4741c92b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff464011bd0 tx=0x7ff464011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 10 client.36747.objecter ms_handle_connect 0x7ff47406aac0 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.362+0000 7ff45b7fe700 10 client.36747.objecter resend_mon_ops 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff464014440 con 0x7ff47406aac0 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:16.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 10 monclient: dump: 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.388 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.389 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 10 client.36747.objecter ms_handle_connect 0x7ff46003a2f0 2024-04-20T02:31:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff4737fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46003a2f0 0x7ff46003c7b0 secure :-1 s=READY pgs=3336 cs=0 l=1 rev1=1 crypto rx=0x7ff474138a20 tx=0x7ff46c009750 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff46400fa90 con 0x7ff47406aac0 2024-04-20T02:31:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:16.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 msgr2=0x7ff4741c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 0x7ff4741c92b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff464011bd0 tx=0x7ff464011f90 comp rx=0 tx=0).stop 2024-04-20T02:31:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 msgr2=0x7ff4741c92b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.363+0000 7ff45b7fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff47406aac0 0x7ff4741c92b0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff464011bd0 tx=0x7ff464011f90 comp rx=0 tx=0).stop 2024-04-20T02:31:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff4795b0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff4795b0700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 0x7ff4741c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff4795b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47406a400 con 0x7ff4741240f0 2024-04-20T02:31:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 0x7ff4741c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient: get_auth_request con 0x7ff4741240f0 auth_method 0 2024-04-20T02:31:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 10 monclient(hunting): handle_auth_done global_id 36765 payload 824 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 1 -- 172.21.15.47:0/1385048052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff47409e430 con 0x7ff4741240f0 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff472ffd700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 0x7ff4741c97f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff45b7fe700 10 client.36747.objecter ms_handle_connect 0x7ff4741240f0 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff45b7fe700 10 client.36747.objecter resend_mon_ops 2024-04-20T02:31:16.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4640044a0 con 0x7ff4741240f0 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff45b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff45b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.371+0000 7ff45b7fe700 10 monclient: dump: 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7ff464004030 con 0x7ff4741240f0 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 msgr2=0x7ff4741c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 0x7ff4741c97f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464004750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 msgr2=0x7ff4741c97f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff45b7fe700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4741240f0 0x7ff4741c97f0 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff464000c00 tx=0x7ff464004750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:16.398 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.400 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:16.401 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.402 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.403 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.404 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.407 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.408 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:16.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:16.410 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.412 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:16.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.417 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.419 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:31:16.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.421 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.421 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:16.421 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 10 librados: watch_flush enter 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 10 librados: watch_flush exit 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 20 client.36747.objecter shutdown clearing up homeless session... 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 10 client.36747.objecter successfully canceled tick 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46003a2f0 msgr2=0x7ff46003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff46003a2f0 0x7ff46003c7b0 secure :-1 s=READY pgs=3336 cs=0 l=1 rev1=1 crypto rx=0x7ff474138a20 tx=0x7ff46c009750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 10 monclient: shutdown 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 msgr2=0x7ff474136fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 0x7ff474136fd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff468013000 tx=0x7ff468004750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4737fe700 1 -- 172.21.15.47:0/1385048052 reap_dead start 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 shutdown_connections 2024-04-20T02:31:16.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 --2- 172.21.15.47:0/1385048052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff474138170 0x7ff474136fd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 >> 172.21.15.47:0/1385048052 conn(0x7ff4740aa8b0 msgr2=0x7ff4740a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 shutdown_connections 2024-04-20T02:31:16.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 -- 172.21.15.47:0/1385048052 wait complete. 2024-04-20T02:31:16.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.372+0000 7ff4795b0700 1 librados: shutdown 2024-04-20T02:31:16.424 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:16.424 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:31:16.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:31:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.564+0000 7fb974169700 10 monclient: get_monmap_and_config 2024-04-20T02:31:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.564+0000 7fb974169700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.564+0000 7fb974169700 10 monclient: monmap: 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:16.565789+0000 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:16.565789+0000 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:16.566 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:16.566 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 1 Processor -- start 2024-04-20T02:31:16.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 1 -- start start 2024-04-20T02:31:16.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 10 monclient: init 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 10 monclient: _add_conns ranks=[0,4,2,5,9,10,8,7,6,1,3] 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb96c071630 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb96c137930 0x7fb96c139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 10 monclient(hunting): picked mon.noname-c con 0x7fb96c137930 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.565+0000 7fb974169700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c13a260 0x7fb96c13c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 10 monclient(hunting): picked mon.noname-b con 0x7fb96c13a260 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96c0751e0 con 0x7fb96c071630 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c072720 con 0x7fb96c13a260 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0a0820 con 0x7fb96c137930 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb974169700 10 monclient(hunting): authenticate will time out at 2555.967060s 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb96c137930 0x7fb96c139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971f05700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb96c071630 0x7fb96c071a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37862/0 (socket says 172.21.15.47:37862) 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c13a260 0x7fb96c13c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971f05700 1 -- 172.21.15.47:0/3056032956 learned_addr learned my addr 172.21.15.47:0/3056032956 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): get_auth_request con 0x7fb96c137930 auth_method 0 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): get_auth_request con 0x7fb96c13a260 auth_method 0 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient(hunting): handle_auth_done global_id 84388 payload 274 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb970f03700 1 -- 172.21.15.47:0/3056032956 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74851818 0 0) 0x7fb96c0751e0 con 0x7fb96c071630 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 -- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c13a260 msgr2=0x7fb96c13c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 --2- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c13a260 0x7fb96c13c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 -- 172.21.15.47:0/3056032956 >> v1:172.21.15.47:6789/0 conn(0x7fb96c071630 legacy=0x7fb96c071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 monclient: found mon.noname-c 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 -- 172.21.15.47:0/3056032956 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c09e890 con 0x7fb96c137930 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb970f03700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb972706700 1 --2- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c13a260 0x7fb96c13c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb971704700 1 --2- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb96c137930 0x7fb96c139d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0098a0 tx=0x7fb95c009bb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=45bbe79fcfd8e0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb970f03700 1 -- 172.21.15.47:0/3056032956 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb95c011aa0 con 0x7fb96c137930 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb970f03700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb970f03700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.566+0000 7fb970f03700 10 monclient: dump: 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.582 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 1 -- 172.21.15.47:0/3056032956 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c011c00 con 0x7fb96c137930 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 5 monclient: authenticate success, global_id 84388 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 1 -- 172.21.15.47:0/3056032956 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb95c011d70 con 0x7fb96c137930 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 4 set_mon_vals no callback set 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb970f03700 10 monclient: dump: 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.585 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 10 monclient: shutdown 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 -- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb96c137930 msgr2=0x7fb96c139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 --2- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb96c137930 0x7fb96c139d20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0098a0 tx=0x7fb95c009bb0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 -- 172.21.15.47:0/3056032956 shutdown_connections 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 --2- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c13a260 0x7fb96c13c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 --2- 172.21.15.47:0/3056032956 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb96c137930 0x7fb96c139d20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 -- 172.21.15.47:0/3056032956 >> 172.21.15.47:0/3056032956 conn(0x7fb96c0a7570 msgr2=0x7fb96c0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 -- 172.21.15.47:0/3056032956 shutdown_connections 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.567+0000 7fb974169700 1 -- 172.21.15.47:0/3056032956 wait complete. 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.568+0000 7fb974169700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.568+0000 7fb974169700 10 monclient: monmap: 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:16.587 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.568+0000 7fb974169700 1 librados: starting msgr at 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.568+0000 7fb974169700 1 librados: starting objecter 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 Processor -- start 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 -- start start 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 librados: setting wanted keys 2024-04-20T02:31:16.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 librados: calling monclient init 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient: init 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient: _add_conns ranks=[3,0,5,2,6,1,7,8,4] 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb96c071630 0x7fb96c1324e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): picked mon.noname-g con 0x7fb96c071630 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb96c137930 0x7fb96c132a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): picked mon.noname-a con 0x7fb96c137930 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb96c13a260 0x7fb96c11d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): picked mon.noname-h con 0x7fb96c13a260 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c11fea0 con 0x7fb96c137930 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c09f200 con 0x7fb96c071630 2024-04-20T02:31:16.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c072380 con 0x7fb96c13a260 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb974169700 10 monclient(hunting): authenticate will time out at 2555.970879s 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb971704700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb96c137930 0x7fb96c132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.569+0000 7fb971704700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb96c137930 0x7fb96c132a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49842/0 (socket says 172.21.15.47:49842) 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 -- 172.21.15.47:0/97412573 learned_addr learned my addr 172.21.15.47:0/97412573 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): get_auth_request con 0x7fb96c137930 auth_method 0 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb96c13a260 0x7fb96c11d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971f05700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb96c071630 0x7fb96c1324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): get_auth_request con 0x7fb96c13a260 auth_method 0 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971f05700 10 monclient(hunting): get_auth_request con 0x7fb96c071630 auth_method 0 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971f05700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971f05700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971f05700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb972706700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient(hunting): handle_auth_done global_id 36783 payload 824 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb96c13a260 msgr2=0x7fb96c11d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb96c13a260 0x7fb96c11d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb96c071630 msgr2=0x7fb96c1324e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb96c071630 0x7fb96c1324e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 monclient: found mon.noname-a 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964009a70 con 0x7fb96c137930 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971f05700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb96c071630 0x7fb96c1324e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb971704700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb96c137930 0x7fb96c132a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0090b0 tx=0x7fb95c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 client.?.objecter ms_handle_connect 0x7fb96c137930 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb95c0045d0 con 0x7fb96c137930 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: dump: 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c01f070 con 0x7fb96c137930 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 5 monclient: authenticate success, global_id 36783 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb95c011a40 con 0x7fb96c137930 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 10 monclient: _renew_subs 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c0094a0 con 0x7fb96c137930 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb963fff700 4 set_mon_vals no callback set 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb9627fc700 10 monclient: dump: 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 10 client.36783.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 10 monclient: _renew_subs 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c11f8e0 con 0x7fb96c137930 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.570+0000 7fb974169700 1 librados: init done 2024-04-20T02:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb974169700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb974169700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 0x7fb96c11dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c068310 con 0x7fb96c06aac0 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb95c021030 con 0x7fb96c137930 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 0x7fb96c11dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient: get_auth_request con 0x7fb96c06aac0 auth_method 0 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb971f05700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.571+0000 7fb9627fc700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb95003a2f0 0x7fb95003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb971f05700 10 monclient(hunting): handle_auth_done global_id 24761 payload 824 2024-04-20T02:31:16.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb971f05700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb96c06bdb0 con 0x7fb96c06aac0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(603..603 src has 266..603) v4 ==== 6641+0+0 (secure 0 0 0) 0x7fb95c04bb30 con 0x7fb96c137930 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 client.36783.objecter ms_dispatch 0x7fb96c001550 osd_map(603..603 src has 266..603) v4 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 3 client.36783.objecter handle_osd_map got epochs [603,603] > 0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 3 client.36783.objecter handle_osd_map decoding full epoch 603 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb971f05700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 0x7fb96c11dad0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968001870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb972706700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb95003a2f0 0x7fb95003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 20 client.36783.objecter dump_active .. 0 homeless 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 client.36783.objecter ms_handle_connect 0x7fb96c06aac0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 client.36783.objecter resend_mon_ops 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb972706700 10 monclient: get_auth_request con 0x7fb95003a2f0 auth_method 0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb968011960 con 0x7fb96c06aac0 2024-04-20T02:31:16.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 monclient: dump: 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.600 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb9680114f0 con 0x7fb96c06aac0 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 msgr2=0x7fb96c11dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 0x7fb96c11dad0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968001870 comp rx=0 tx=0).stop 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 msgr2=0x7fb96c11dad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c06aac0 0x7fb96c11dad0 secure :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968001870 comp rx=0 tx=0).stop 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb9627fc700 10 client.36783.objecter ms_handle_connect 0x7fb95003a2f0 2024-04-20T02:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.572+0000 7fb972706700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb95003a2f0 0x7fb95003c7b0 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7fb96c139000 tx=0x7fb96400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb974169700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb974169700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 0x7fb96c0a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb974169700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c06b960 con 0x7fb96c11e010 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 0x7fb96c0a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient: get_auth_request con 0x7fb96c11e010 auth_method 0 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 10 monclient(hunting): handle_auth_done global_id 24770 payload 824 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.580+0000 7fb971f05700 1 -- 172.21.15.47:0/97412573 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb96c072120 con 0x7fb96c11e010 2024-04-20T02:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 client.36783.objecter ms_handle_connect 0x7fb96c11e010 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 client.36783.objecter resend_mon_ops 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb971f05700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 0x7fb96c0a1840 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fb968016000 tx=0x7fb968000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb968021070 con 0x7fb96c11e010 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 monclient: dump: 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fb96801c8e0 con 0x7fb96c11e010 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 msgr2=0x7fb96c0a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 0x7fb96c0a1840 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fb968016000 tx=0x7fb968000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 msgr2=0x7fb96c0a1840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb9627fc700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb96c11e010 0x7fb96c0a1840 secure :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fb968016000 tx=0x7fb968000ed0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.607 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:16.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:16.607 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:16.607 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:16.608 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.609 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.610 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:16.611 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.612 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.613 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.614 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.615 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.616 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:16.617 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:16.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:16.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:16.629 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:16.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb974169700 10 librados: watch_flush enter 2024-04-20T02:31:16.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb974169700 10 librados: watch_flush exit 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb974169700 20 client.36783.objecter shutdown clearing up homeless session... 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb974169700 10 client.36783.objecter successfully canceled tick 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb95003a2f0 msgr2=0x7fb95003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.581+0000 7fb974169700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb95003a2f0 0x7fb95003c7b0 secure :-1 s=READY pgs=3337 cs=0 l=1 rev1=1 crypto rx=0x7fb96c139000 tx=0x7fb96400b040 comp rx=0 tx=0).stop 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 10 monclient: shutdown 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb96c137930 msgr2=0x7fb96c132a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 --2- 172.21.15.47:0/97412573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb96c137930 0x7fb96c132a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0090b0 tx=0x7fb95c004750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb972706700 1 -- 172.21.15.47:0/97412573 reap_dead start 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb972706700 1 -- 172.21.15.47:0/97412573 reap_dead start 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 shutdown_connections 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 >> 172.21.15.47:0/97412573 conn(0x7fb96c0a7570 msgr2=0x7fb96c0a57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 shutdown_connections 2024-04-20T02:31:16.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 -- 172.21.15.47:0/97412573 wait complete. 2024-04-20T02:31:16.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.582+0000 7fb974169700 1 librados: shutdown 2024-04-20T02:31:16.632 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:16.632 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:31:16.632 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:31:16.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.771+0000 7faa8c257700 10 monclient: get_monmap_and_config 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.771+0000 7faa8c257700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.771+0000 7faa8c257700 10 monclient: monmap: 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:16.772803+0000 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:16.772803+0000 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.772 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:16.773 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.772+0000 7faa8c257700 1 Processor -- start 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.772+0000 7faa8c257700 1 -- start start 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.772+0000 7faa8c257700 10 monclient: init 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient: _add_conns ranks=[9,5,1,2,0,8,7,3,6,10,4] 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa84137d90 0x7faa840a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): picked mon.noname-a con 0x7faa84137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faa84138170 0x7faa840a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): picked mon.noname-h con 0x7faa84138170 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7faa841385b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa840751e0 con 0x7faa841385b0 2024-04-20T02:31:16.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8409fa40 con 0x7faa84137d90 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa8409dfc0 con 0x7faa84138170 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa8c257700 10 monclient(hunting): authenticate will time out at 2556.174266s 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa84137d90 0x7faa840a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa84137d90 0x7faa840a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49858/0 (socket says 172.21.15.47:49858) 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 -- 172.21.15.47:0/3293203378 learned_addr learned my addr 172.21.15.47:0/3293203378 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): get_auth_request con 0x7faa84137d90 auth_method 0 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa897f2700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faa84138170 0x7faa840a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa897f2700 10 monclient(hunting): get_auth_request con 0x7faa84138170 auth_method 0 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa897f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa897f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa897f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient(hunting): handle_auth_done global_id 36792 payload 274 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 -- 172.21.15.47:0/3293203378 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faa84138170 msgr2=0x7faa840a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faa84138170 0x7faa840a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 -- 172.21.15.47:0/3293203378 >> v1:172.21.15.179:6789/0 conn(0x7faa841385b0 legacy=0x7faa840a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 monclient: found mon.noname-a 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:16.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 -- 172.21.15.47:0/3293203378 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840a7bc0 con 0x7faa84137d90 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa897f2700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faa84138170 0x7faa840a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.773+0000 7faa89ff3700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa84137d90 0x7faa840a09e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faa80009920 tx=0x7faa80009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a116928c9d86f48a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 1 -- 172.21.15.47:0/3293203378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa80009e70 con 0x7faa84137d90 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: dump: 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.787 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 1 -- 172.21.15.47:0/3293203378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000bc90 con 0x7faa84137d90 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 1 -- 172.21.15.47:0/3293203378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa8000be00 con 0x7faa84137d90 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 5 monclient: authenticate success, global_id 36792 2024-04-20T02:31:16.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 4 set_mon_vals no callback set 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa88ff1700 10 monclient: dump: 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.789 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 10 monclient: shutdown 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 -- 172.21.15.47:0/3293203378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa84137d90 msgr2=0x7faa840a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa84137d90 0x7faa840a09e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faa80009920 tx=0x7faa80009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 -- 172.21.15.47:0/3293203378 shutdown_connections 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa84137d90 0x7faa840a09e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 --2- 172.21.15.47:0/3293203378 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7faa84138170 0x7faa840a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 -- 172.21.15.47:0/3293203378 >> 172.21.15.47:0/3293203378 conn(0x7faa840aa8b0 msgr2=0x7faa840a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 -- 172.21.15.47:0/3293203378 shutdown_connections 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.774+0000 7faa8c257700 1 -- 172.21.15.47:0/3293203378 wait complete. 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.775+0000 7faa8c257700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.775+0000 7faa8c257700 10 monclient: monmap: 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.791 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.775+0000 7faa8c257700 1 librados: starting msgr at 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.775+0000 7faa8c257700 1 librados: starting objecter 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.775+0000 7faa8c257700 1 Processor -- start 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 -- start start 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 librados: setting wanted keys 2024-04-20T02:31:16.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 librados: calling monclient init 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient: init 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient: _add_conns ranks=[4,6,2,1,5,3,8,0,7] 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84137d90 0x7faa841cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): picked mon.noname-c con 0x7faa84137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faa84138170 0x7faa84136d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): picked mon.noname-d con 0x7faa84138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa841385b0 0x7faa841372d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): picked mon.noname-b con 0x7faa841385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8409e9e0 con 0x7faa841385b0 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa84122f50 con 0x7faa84137d90 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7faa8409dc20 con 0x7faa84138170 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8c257700 10 monclient(hunting): authenticate will time out at 2556.177580s 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84137d90 0x7faa841cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84137d90 0x7faa841cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55544/0 (socket says 172.21.15.47:55544) 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 1 -- 172.21.15.47:0/2000184701 learned_addr learned my addr 172.21.15.47:0/2000184701 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): get_auth_request con 0x7faa84137d90 auth_method 0 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faa84138170 0x7faa84136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa841385b0 0x7faa841372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): get_auth_request con 0x7faa84138170 auth_method 0 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): get_auth_request con 0x7faa841385b0 auth_method 0 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa897f2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.776+0000 7faa8a7f4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 10 monclient(hunting): handle_auth_done global_id 55158 payload 824 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84137d90 msgr2=0x7faa841cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84137d90 0x7faa841cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa841385b0 msgr2=0x7faa841372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa841385b0 0x7faa841372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 monclient: found mon.noname-d 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80009550 con 0x7faa84138170 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa89ff3700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84137d90 0x7faa841cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8a7f4700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa841385b0 0x7faa841372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa897f2700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faa84138170 0x7faa84136d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e7e0 tx=0x7faa7400eaf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 client.?.objecter ms_handle_connect 0x7faa84138170 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:16.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa7400c070 con 0x7faa84138170 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: dump: 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.797 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa74013880 con 0x7faa84138170 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 5 monclient: authenticate success, global_id 55158 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 10 monclient: _renew_subs 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa841cc6b0 con 0x7faa84138170 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa74013e60 con 0x7faa84138170 2024-04-20T02:31:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7bfff700 4 set_mon_vals no callback set 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 10 client.55158.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa7a7fc700 10 monclient: dump: 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 10 monclient: _renew_subs 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa84137810 con 0x7faa84138170 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.777+0000 7faa8c257700 1 librados: init done 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7faa740139f0 con 0x7faa84138170 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa8c257700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:16.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa8c257700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 0x7faa841208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa84068180 con 0x7faa8406aac0 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 0x7faa841208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient: get_auth_request con 0x7faa8406aac0 auth_method 0 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faa6803a2f0 0x7faa6803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(603..603 src has 266..603) v4 ==== 6641+0+0 (secure 0 0 0) 0x7faa7404a4e0 con 0x7faa84138170 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 10 client.55158.objecter ms_dispatch 0x7faa84001550 osd_map(603..603 src has 266..603) v4 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 3 client.55158.objecter handle_osd_map got epochs [603,603] > 0 2024-04-20T02:31:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 3 client.55158.objecter handle_osd_map decoding full epoch 603 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 10 monclient(hunting): handle_auth_done global_id 84406 payload 824 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 20 client.55158.objecter dump_active .. 0 homeless 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa84123f40 con 0x7faa8406aac0 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa8a7f4700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faa6803a2f0 0x7faa6803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 10 client.55158.objecter ms_handle_connect 0x7faa8406aac0 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa89ff3700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 0x7faa841208d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa80009130 tx=0x7faa80004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa7a7fc700 10 client.55158.objecter resend_mon_ops 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.778+0000 7faa8a7f4700 10 monclient: get_auth_request con 0x7faa6803a2f0 auth_method 0 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa80020070 con 0x7faa8406aac0 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 10 monclient: dump: 2024-04-20T02:31:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faa8000bed0 con 0x7faa8406aac0 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 msgr2=0x7faa841208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 0x7faa841208d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa80009130 tx=0x7faa80004750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 msgr2=0x7faa841208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa8406aac0 0x7faa841208d0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa80009130 tx=0x7faa80004750 comp rx=0 tx=0).stop 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa7a7fc700 10 client.55158.objecter ms_handle_connect 0x7faa6803a2f0 2024-04-20T02:31:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.779+0000 7faa8a7f4700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faa6803a2f0 0x7faa6803c7b0 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7faa84124900 tx=0x7faa7c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa8c257700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:16.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa8c257700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 0x7faa84121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa8c257700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa8406a400 con 0x7faa84120e10 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 0x7faa84121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient: get_auth_request con 0x7faa84120e10 auth_method 0 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 10 monclient(hunting): handle_auth_done global_id 84415 payload 824 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.787+0000 7faa89ff3700 1 -- 172.21.15.47:0/2000184701 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faa8409f760 con 0x7faa84120e10 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 client.55158.objecter ms_handle_connect 0x7faa84120e10 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa89ff3700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 0x7faa84121220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa80023000 tx=0x7faa80011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 client.55158.objecter resend_mon_ops 2024-04-20T02:31:16.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa8002c300 con 0x7faa84120e10 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 monclient: dump: 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.808 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7faa800268c0 con 0x7faa84120e10 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 msgr2=0x7faa84121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 0x7faa84121220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa80023000 tx=0x7faa80011000 comp rx=0 tx=0).stop 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 msgr2=0x7faa84121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa7a7fc700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa84120e10 0x7faa84121220 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa80023000 tx=0x7faa80011000 comp rx=0 tx=0).stop 2024-04-20T02:31:16.809 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:16.810 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.811 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.812 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:16.813 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.814 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.815 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:16.817 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:16.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:16.820 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:16.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.828 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:16.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:16.830 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:16.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:16.830 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:16.830 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:16.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa8c257700 10 librados: watch_flush enter 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa8c257700 10 librados: watch_flush exit 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa8c257700 20 client.55158.objecter shutdown clearing up homeless session... 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.788+0000 7faa8c257700 10 client.55158.objecter successfully canceled tick 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faa6803a2f0 msgr2=0x7faa6803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7faa6803a2f0 0x7faa6803c7b0 secure :-1 s=READY pgs=3338 cs=0 l=1 rev1=1 crypto rx=0x7faa84124900 tx=0x7faa7c009040 comp rx=0 tx=0).stop 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 10 monclient: shutdown 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faa84138170 msgr2=0x7faa84136d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 --2- 172.21.15.47:0/2000184701 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7faa84138170 0x7faa84136d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e7e0 tx=0x7faa7400eaf0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8a7f4700 1 -- 172.21.15.47:0/2000184701 reap_dead start 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8a7f4700 1 -- 172.21.15.47:0/2000184701 reap_dead start 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 shutdown_connections 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 >> 172.21.15.47:0/2000184701 conn(0x7faa840aa8b0 msgr2=0x7faa841bf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 shutdown_connections 2024-04-20T02:31:16.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 -- 172.21.15.47:0/2000184701 wait complete. 2024-04-20T02:31:16.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.789+0000 7faa8c257700 1 librados: shutdown 2024-04-20T02:31:16.833 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:16.833 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:31:16.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:31:16.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.971+0000 7f8c1cc81700 10 monclient: get_monmap_and_config 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.971+0000 7f8c1cc81700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.971+0000 7f8c1cc81700 10 monclient: monmap: 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:16.972550+0000 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:16.972550+0000 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.972 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.971+0000 7f8c1cc81700 1 Processor -- start 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 -- start start 2024-04-20T02:31:16.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient: init 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient: _add_conns ranks=[3,2,4,5,1,0,6,7,9,10,8] 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18071710 0x7f8c1809e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): picked mon.noname-g con 0x7f8c18071710 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 0x7f8c18071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): picked mon.noname-b con 0x7f8c1806f090 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 0x7f8c180a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): picked mon.noname-c con 0x7f8c18072030 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1809ee80 con 0x7f8c1806f090 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180a0680 con 0x7f8c18071710 2024-04-20T02:31:16.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180a0030 con 0x7f8c18072030 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c1cc81700 10 monclient(hunting): authenticate will time out at 2556.373822s 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c16d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 0x7f8c180a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.972+0000 7f8c15d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 0x7f8c18071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 0x7f8c18071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36844/0 (socket says 172.21.15.47:36844) 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 0x7f8c180a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55576/0 (socket says 172.21.15.47:55576) 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 1 -- 172.21.15.47:0/3513116506 learned_addr learned my addr 172.21.15.47:0/3513116506 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): get_auth_request con 0x7f8c1806f090 auth_method 0 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): get_auth_request con 0x7f8c18072030 auth_method 0 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1659c700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18071710 0x7f8c1809e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1659c700 10 monclient(hunting): get_auth_request con 0x7f8c18071710 auth_method 0 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_done global_id 84424 payload 274 2024-04-20T02:31:16.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 -- 172.21.15.47:0/3513116506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18071710 msgr2=0x7f8c1809e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18071710 0x7f8c1809e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 -- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 msgr2=0x7f8c18071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 0x7f8c18071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 monclient: found mon.noname-c 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 -- 172.21.15.47:0/3513116506 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180751e0 con 0x7f8c18072030 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c15d9b700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 0x7f8c18071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1659c700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18071710 0x7f8c1809e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c16d9d700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 0x7f8c180a2950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009df0 tx=0x7f8c1000b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=755be975750a5d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 1 -- 172.21.15.47:0/3513116506 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c1000e070 con 0x7f8c18072030 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: dump: 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.988 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 1 -- 172.21.15.47:0/3513116506 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c100098f0 con 0x7f8c18072030 2024-04-20T02:31:16.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1cc81700 5 monclient: authenticate success, global_id 84424 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1cc81700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1cc81700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 1 -- 172.21.15.47:0/3513116506 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c100128a0 con 0x7f8c18072030 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1cc81700 4 set_mon_vals no callback set 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.973+0000 7f8c1559a700 10 monclient: dump: 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.990 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 10 monclient: shutdown 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 -- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 msgr2=0x7f8c180a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 0x7f8c180a2950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009df0 tx=0x7f8c1000b330 comp rx=0 tx=0).stop 2024-04-20T02:31:16.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 -- 172.21.15.47:0/3513116506 shutdown_connections 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8c18072030 0x7f8c180a2950 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18071710 0x7f8c1809e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/3513116506 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c1806f090 0x7f8c18071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 -- 172.21.15.47:0/3513116506 >> 172.21.15.47:0/3513116506 conn(0x7f8c180a7570 msgr2=0x7f8c180a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 -- 172.21.15.47:0/3513116506 shutdown_connections 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.974+0000 7f8c1cc81700 1 -- 172.21.15.47:0/3513116506 wait complete. 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.975+0000 7f8c1cc81700 10 monclient: build_initial_monmap 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.975+0000 7f8c1cc81700 10 monclient: monmap: 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:16.992 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.975+0000 7f8c1cc81700 1 librados: starting msgr at 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.975+0000 7f8c1cc81700 1 librados: starting objecter 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.975+0000 7f8c1cc81700 1 Processor -- start 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 -- start start 2024-04-20T02:31:16.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 librados: setting wanted keys 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 librados: calling monclient init 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient: init 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient: _add_conns ranks=[7,2,3,1,4,8,0,6,5] 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8c1806f090 0x7f8c18066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): picked mon.noname-i con 0x7f8c1806f090 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c18071710 0x7f8c181c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): picked mon.noname-b con 0x7f8c18071710 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18072030 0x7f8c18133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): picked mon.noname-g con 0x7f8c18072030 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18067800 con 0x7f8c18071710 2024-04-20T02:31:16.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18121d40 con 0x7f8c18072030 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18122020 con 0x7f8c1806f090 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c1cc81700 10 monclient(hunting): authenticate will time out at 2556.377878s 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.976+0000 7f8c15d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c18071710 0x7f8c181c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c18071710 0x7f8c181c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36852/0 (socket says 172.21.15.47:36852) 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 -- 172.21.15.47:0/4061294134 learned_addr learned my addr 172.21.15.47:0/4061294134 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): get_auth_request con 0x7f8c18071710 auth_method 0 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8c1806f090 0x7f8c18066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18072030 0x7f8c18133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:16.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): get_auth_request con 0x7f8c1806f090 auth_method 0 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): get_auth_request con 0x7f8c18072030 auth_method 0 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:16.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c16d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient(hunting): handle_auth_done global_id 24797 payload 824 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8c1806f090 msgr2=0x7f8c18066ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8c1806f090 0x7f8c18066ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18072030 msgr2=0x7f8c18133c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8c18072030 0x7f8c18133c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 monclient: found mon.noname-b 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009550 con 0x7f8c18071710 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c15d9b700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c18071710 0x7f8c181c6ee0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00ebe0 tx=0x7f8c0c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 client.?.objecter ms_handle_connect 0x7f8c18071710 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0c00ccc0 con 0x7f8c18071710 2024-04-20T02:31:16.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: dump: 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:16.998 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: _finish_auth 0 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00ce40 con 0x7f8c18071710 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 5 monclient: authenticate success, global_id 24797 2024-04-20T02:31:16.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 10 monclient: _renew_subs 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c10009a70 con 0x7f8c18071710 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0c0138e0 con 0x7f8c18071710 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c06ffd700 10 monclient: dump: 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.000 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c14d99700 4 set_mon_vals no callback set 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 10 client.24797.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 10 monclient: _renew_subs 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:17.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c18120ee0 con 0x7f8c18071710 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.977+0000 7f8c1cc81700 1 librados: init done 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f8c0c013470 con 0x7f8c18071710 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c1cc81700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 0x7f8c18134150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18068310 con 0x7f8c1806aac0 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c1659c700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 0x7f8c18134150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.978+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8bf803a320 0x7f8bf803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient: get_auth_request con 0x7f8c1806aac0 auth_method 0 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(603..603 src has 266..603) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f8c0c04ab60 con 0x7f8c18071710 2024-04-20T02:31:17.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 client.24797.objecter ms_dispatch 0x7f8c18001550 osd_map(603..603 src has 266..603) v4 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 3 client.24797.objecter handle_osd_map got epochs [603,603] > 0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 3 client.24797.objecter handle_osd_map decoding full epoch 603 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c16d9d700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8bf803a320 0x7f8bf803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c16d9d700 10 monclient: get_auth_request con 0x7f8bf803a320 auth_method 0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 10 monclient(hunting): handle_auth_done global_id 55167 payload 824 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 20 client.24797.objecter dump_active .. 0 homeless 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c18121eb0 con 0x7f8c1806aac0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c1659c700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 0x7f8c18134150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c08010800 tx=0x7f8c08010bc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 client.24797.objecter ms_handle_connect 0x7f8c1806aac0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 client.24797.objecter resend_mon_ops 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0800ec10 con 0x7f8c1806aac0 2024-04-20T02:31:17.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 monclient: dump: 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.004 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 client.24797.objecter ms_handle_connect 0x7f8bf803a320 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c16d9d700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8bf803a320 0x7f8bf803c7e0 secure :-1 s=READY pgs=3339 cs=0 l=1 rev1=1 crypto rx=0x7f8c1806f490 tx=0x7f8c10011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8c0800ed70 con 0x7f8c1806aac0 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 msgr2=0x7f8c18134150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 0x7f8c18134150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c08010800 tx=0x7f8c08010bc0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 msgr2=0x7f8c18134150 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.979+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c1806aac0 0x7f8c18134150 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c08010800 tx=0x7f8c08010bc0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1cc81700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 0x7f8c181ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1cc81700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1806a400 con 0x7f8c181c9f20 2024-04-20T02:31:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 0x7f8c181ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 10 monclient: get_auth_request con 0x7f8c181c9f20 auth_method 0 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.987+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c1659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c1659c700 10 monclient(hunting): handle_auth_done global_id 55176 payload 824 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c1659c700 1 -- 172.21.15.47:0/4061294134 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8c1806c3a0 con 0x7f8c181c9f20 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c1659c700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 0x7f8c181ca330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c08010800 tx=0x7f8c08004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 client.24797.objecter ms_handle_connect 0x7f8c181c9f20 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 client.24797.objecter resend_mon_ops 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c08021ce0 con 0x7f8c181c9f20 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:17.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 monclient: dump: 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.009 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f8c08021e40 con 0x7f8c181c9f20 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 msgr2=0x7f8c181ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 0x7f8c181ca330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c08010800 tx=0x7f8c08004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 msgr2=0x7f8c181ca330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.988+0000 7f8c06ffd700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f8c181c9f20 0x7f8c181ca330 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c08010800 tx=0x7f8c08004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:17.011 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.012 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.013 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:17.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.015 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:17.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.018 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:17.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.025 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.027 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:17.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:17.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.030 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.031 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.033 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.033 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:17.033 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 10 librados: watch_flush enter 2024-04-20T02:31:17.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 10 librados: watch_flush exit 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 20 client.24797.objecter shutdown clearing up homeless session... 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 10 client.24797.objecter successfully canceled tick 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8bf803a320 msgr2=0x7f8bf803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f8bf803a320 0x7f8bf803c7e0 secure :-1 s=READY pgs=3339 cs=0 l=1 rev1=1 crypto rx=0x7f8c1806f490 tx=0x7f8c10011000 comp rx=0 tx=0).stop 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 10 monclient: shutdown 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c18071710 msgr2=0x7f8c181c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 --2- 172.21.15.47:0/4061294134 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8c18071710 0x7f8c181c6ee0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00ebe0 tx=0x7f8c0c00eef0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c16d9d700 1 -- 172.21.15.47:0/4061294134 reap_dead start 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c16d9d700 1 -- 172.21.15.47:0/4061294134 reap_dead start 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 shutdown_connections 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 >> 172.21.15.47:0/4061294134 conn(0x7f8c180a7570 msgr2=0x7f8c18134c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 shutdown_connections 2024-04-20T02:31:17.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 -- 172.21.15.47:0/4061294134 wait complete. 2024-04-20T02:31:17.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:16.989+0000 7f8c1cc81700 1 librados: shutdown 2024-04-20T02:31:17.036 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:17.036 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:31:17.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:31:17.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.176+0000 7f222bdf0700 10 monclient: get_monmap_and_config 2024-04-20T02:31:17.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.176+0000 7f222bdf0700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.176+0000 7f222bdf0700 10 monclient: monmap: 2024-04-20T02:31:17.177 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.177 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:17.177902+0000 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:17.177902+0000 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.178 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:17.179 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.177+0000 7f222bdf0700 1 Processor -- start 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 -- start start 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient: init 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient: _add_conns ranks=[2,8,7,9,1,10,5,6,0,3,4] 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f22240a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): picked mon.noname-b con 0x7f22240a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f22240a3270 0x7f2224137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): picked mon.noname-e con 0x7f22240a3270 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2224138040 0x7f22241384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): picked mon.noname-i con 0x7f2224138040 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222409f150 con 0x7f22240a2950 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f222409ee70 con 0x7f2224138040 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f222409f710 con 0x7f22240a3270 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.178+0000 7f222bdf0700 10 monclient(hunting): authenticate will time out at 2556.579849s 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f22240a3270 0x7f2224137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f22240a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f22240a3270 0x7f2224137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55176/0 (socket says 172.21.15.47:55176) 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f22240a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36868/0 (socket says 172.21.15.47:36868) 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 1 -- 172.21.15.47:0/611587319 learned_addr learned my addr 172.21.15.47:0/611587319 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): get_auth_request con 0x7f22240a3270 auth_method 0 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222938b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): get_auth_request con 0x7f22240a2950 auth_method 0 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2224138040 0x7f22241384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): get_auth_request con 0x7f2224138040 auth_method 0 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f222a38d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient(hunting): handle_auth_done global_id 24806 payload 274 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 -- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f22240a3270 msgr2=0x7f2224137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f22240a3270 0x7f2224137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 -- 172.21.15.47:0/611587319 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2224138040 msgr2=0x7f22241384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2224138040 0x7f22241384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 monclient: found mon.noname-b 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.179+0000 7f2229b8c700 1 -- 172.21.15.47:0/611587319 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22240751e0 con 0x7f22240a2950 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2229b8c700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f22240a2d30 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f221c009870 tx=0x7f221c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5840322a5c3ba0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 1 -- 172.21.15.47:0/611587319 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f221c011aa0 con 0x7f22240a2950 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: dump: 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.186 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 1 -- 172.21.15.47:0/611587319 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c011c00 con 0x7f22240a2950 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f222bdf0700 5 monclient: authenticate success, global_id 24806 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f222bdf0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f222bdf0700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 1 -- 172.21.15.47:0/611587319 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f221c011d70 con 0x7f22240a2950 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f222bdf0700 4 set_mon_vals no callback set 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.180+0000 7f2228b8a700 10 monclient: dump: 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.189 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 10 monclient: shutdown 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 -- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 msgr2=0x7f22240a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f22240a2d30 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f221c009870 tx=0x7f221c009b80 comp rx=0 tx=0).stop 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 -- 172.21.15.47:0/611587319 shutdown_connections 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2224138040 0x7f22241384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f22240a2d30 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 --2- 172.21.15.47:0/611587319 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f22240a3270 0x7f2224137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 -- 172.21.15.47:0/611587319 >> 172.21.15.47:0/611587319 conn(0x7f22240a7570 msgr2=0x7f22240a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 -- 172.21.15.47:0/611587319 shutdown_connections 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.181+0000 7f222bdf0700 1 -- 172.21.15.47:0/611587319 wait complete. 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 10 monclient: monmap: 2024-04-20T02:31:17.190 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 1 librados: starting msgr at 2024-04-20T02:31:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 1 librados: starting objecter 2024-04-20T02:31:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 1 Processor -- start 2024-04-20T02:31:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 1 -- start start 2024-04-20T02:31:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 1 librados: setting wanted keys 2024-04-20T02:31:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 1 librados: calling monclient init 2024-04-20T02:31:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.182+0000 7f222bdf0700 10 monclient: init 2024-04-20T02:31:17.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient: _add_conns ranks=[2,7,4,1,6,0,3,8,5] 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f222412e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): picked mon.noname-b con 0x7f22240a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 0x7f222412ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): picked mon.noname-i con 0x7f22240a3270 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 0x7f222412f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): picked mon.noname-c con 0x7f2224138040 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222413b890 con 0x7f22240a2950 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f222409d960 con 0x7f2224138040 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f222413e310 con 0x7f22240a3270 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222bdf0700 10 monclient(hunting): authenticate will time out at 2556.584192s 2024-04-20T02:31:17.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 0x7f222412f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 0x7f222412f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55584/0 (socket says 172.21.15.47:55584) 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 1 -- 172.21.15.47:0/112254769 learned_addr learned my addr 172.21.15.47:0/112254769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): get_auth_request con 0x7f2224138040 auth_method 0 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222a38d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222938b700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 0x7f222412ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222938b700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 msgr2=0x7f222412ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222938b700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 msgr2=0x7f222412ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222938b700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 0x7f222412ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:17.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.183+0000 7f222938b700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 0x7f222412ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 10 monclient(hunting): handle_auth_done global_id 84433 payload 824 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 msgr2=0x7f222412ebe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f22240a3270 0x7f222412ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 msgr2=0x7f222412e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f22240a2950 0x7f222412e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 monclient: found mon.noname-c 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c0094a0 con 0x7f2224138040 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f222a38d700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 0x7f222412f120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f222000e950 tx=0x7f222000ed10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f22167fc700 10 client.?.objecter ms_handle_connect 0x7f2224138040 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.184+0000 7f22167fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2220009730 con 0x7f2224138040 2024-04-20T02:31:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: dump: 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.200 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222000ccc0 con 0x7f2224138040 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 5 monclient: authenticate success, global_id 84433 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 10 monclient: _renew_subs 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222409dd70 con 0x7f2224138040 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f222000ce30 con 0x7f2224138040 2024-04-20T02:31:17.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f22167fc700 10 monclient: dump: 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 10 client.84433.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 10 monclient: _renew_subs 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f2217fff700 4 set_mon_vals no callback set 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:17.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222413d260 con 0x7f2224138040 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.191+0000 7f222bdf0700 1 librados: init done 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f222bdf0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f222bdf0700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 0x7f2224139300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f222413d3f0 con 0x7f222413da20 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f2229b8c700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 0x7f2224139300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f2229b8c700 10 monclient: get_auth_request con 0x7f222413da20 auth_method 0 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f2229b8c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f2229b8c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.192+0000 7f2229b8c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f2229b8c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f2229b8c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f2229b8c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f2229b8c700 10 monclient(hunting): handle_auth_done global_id 35990 payload 824 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f2229b8c700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22240672b0 con 0x7f222413da20 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f2229b8c700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 0x7f2224139300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f221c009870 tx=0x7f221c006d70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f22167fc700 10 client.84433.objecter ms_handle_connect 0x7f222413da20 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f22167fc700 10 client.84433.objecter resend_mon_ops 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f221c004440 con 0x7f222413da20 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f22167fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f22167fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.193+0000 7f22167fc700 10 monclient: dump: 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.205 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f221c01e070 con 0x7f222413da20 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:17.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 msgr2=0x7f2224139300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 0x7f2224139300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f221c009870 tx=0x7f221c006d70 comp rx=0 tx=0).stop 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 msgr2=0x7f2224139300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222413da20 0x7f2224139300 secure :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f221c009870 tx=0x7f221c006d70 comp rx=0 tx=0).stop 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.194+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f222000c850 con 0x7f2224138040 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f220803a520 0x7f220803c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f222004aa90 con 0x7f2224138040 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 10 client.84433.objecter ms_dispatch 0x7f2224001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 3 client.84433.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 3 client.84433.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 20 client.84433.objecter dump_active .. 0 homeless 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f2229b8c700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f220803a520 0x7f220803c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f2229b8c700 10 monclient: get_auth_request con 0x7f220803a520 auth_method 0 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f2229b8c700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f220803a520 0x7f220803c9e0 secure :-1 s=READY pgs=3340 cs=0 l=1 rev1=1 crypto rx=0x7f221c022000 tx=0x7f221c020000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.195+0000 7f22167fc700 10 client.84433.objecter ms_handle_connect 0x7f220803a520 2024-04-20T02:31:17.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222bdf0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:17.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222bdf0700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 0x7f2224139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222bdf0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2224068310 con 0x7f222406aac0 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 0x7f2224139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient: get_auth_request con 0x7f222406aac0 auth_method 0 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.202+0000 7f222938b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f222938b700 10 monclient(hunting): handle_auth_done global_id 35999 payload 824 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f222938b700 1 -- 172.21.15.47:0/112254769 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22240a36b0 con 0x7f222406aac0 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 client.84433.objecter ms_handle_connect 0x7f222406aac0 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f222938b700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 0x7f2224139840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2218007d00 tx=0x7f221800f430 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 client.84433.objecter resend_mon_ops 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f221801a070 con 0x7f222406aac0 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 monclient: dump: 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7f221800fad0 con 0x7f222406aac0 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 msgr2=0x7f2224139840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 0x7f2224139840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2218007d00 tx=0x7f221800f430 comp rx=0 tx=0).stop 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 msgr2=0x7f2224139840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f22167fc700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f222406aac0 0x7f2224139840 secure :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2218007d00 tx=0x7f221800f430 comp rx=0 tx=0).stop 2024-04-20T02:31:17.212 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:17.213 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.214 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.215 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:17.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.217 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.220 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:17.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.223 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:17.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.231 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.232 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:17.233 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:17.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.203+0000 7f222bdf0700 10 librados: watch_flush enter 2024-04-20T02:31:17.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 10 librados: watch_flush exit 2024-04-20T02:31:17.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 20 client.84433.objecter shutdown clearing up homeless session... 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 10 client.84433.objecter successfully canceled tick 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f220803a520 msgr2=0x7f220803c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f220803a520 0x7f220803c9e0 secure :-1 s=READY pgs=3340 cs=0 l=1 rev1=1 crypto rx=0x7f221c022000 tx=0x7f221c020000 comp rx=0 tx=0).stop 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 10 monclient: shutdown 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 msgr2=0x7f222412f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 0x7f222412f120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f222000e950 tx=0x7f222000ed10 comp rx=0 tx=0).stop 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222a38d700 1 -- 172.21.15.47:0/112254769 reap_dead start 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 shutdown_connections 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 --2- 172.21.15.47:0/112254769 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2224138040 0x7f222412f120 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 >> 172.21.15.47:0/112254769 conn(0x7f22240a7570 msgr2=0x7f222409c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 shutdown_connections 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 -- 172.21.15.47:0/112254769 wait complete. 2024-04-20T02:31:17.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.204+0000 7f222bdf0700 1 librados: shutdown 2024-04-20T02:31:17.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:17.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:31:17.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:31:17.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.374+0000 7fc819b6c700 10 monclient: get_monmap_and_config 2024-04-20T02:31:17.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.374+0000 7fc819b6c700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.374+0000 7fc819b6c700 10 monclient: monmap: 2024-04-20T02:31:17.374 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.374 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.374 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:17.375452+0000 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:17.375452+0000 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:17.375 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 Processor -- start 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 -- start start 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient: init 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient: _add_conns ranks=[3,8,10,9,6,0,7,4,5,2,1] 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc814135ad0 0x7fc81409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): picked mon.noname-g con 0x7fc814135ad0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 0x7fc8141362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): picked mon.noname-e con 0x7fc814135eb0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc8140a2950 0x7fc8140a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): picked mon.noname-f con 0x7fc8140a2950 addr v2:172.21.15.179:3300/0 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81409bc20 con 0x7fc8140a2950 2024-04-20T02:31:17.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc814067db0 con 0x7fc814135ad0 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140a0ae0 con 0x7fc814135eb0 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc819b6c700 10 monclient(hunting): authenticate will time out at 2556.776704s 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 0x7fc8141362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 0x7fc8141362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55212/0 (socket says 172.21.15.47:55212) 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 1 -- 172.21.15.47:0/2337273020 learned_addr learned my addr 172.21.15.47:0/2337273020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 10 monclient(hunting): get_auth_request con 0x7fc814135eb0 auth_method 0 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc813fff700 1 --2- 172.21.15.47:0/2337273020 >> v2:172.21.15.179:3300/0 conn(0x7fc8140a2950 0x7fc8140a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): get_auth_request con 0x7fc8140a2950 auth_method 0 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.375+0000 7fc8137fe700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc814135ad0 0x7fc81409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8137fe700 10 monclient(hunting): get_auth_request con 0x7fc814135ad0 auth_method 0 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8137fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8137fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8137fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc813fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 10 monclient(hunting): handle_auth_done global_id 36008 payload 274 2024-04-20T02:31:17.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 -- 172.21.15.47:0/2337273020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc814135ad0 msgr2=0x7fc81409b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc814135ad0 0x7fc81409b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 -- 172.21.15.47:0/2337273020 >> v2:172.21.15.179:3300/0 conn(0x7fc8140a2950 msgr2=0x7fc8140a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2337273020 >> v2:172.21.15.179:3300/0 conn(0x7fc8140a2950 0x7fc8140a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 monclient: found mon.noname-e 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 -- 172.21.15.47:0/2337273020 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8140751e0 con 0x7fc814135eb0 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8137fe700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc814135ad0 0x7fc81409b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 0x7fc8141362b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc808009920 tx=0x7fc808009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=372c2f160ede9a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 1 -- 172.21.15.47:0/2337273020 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc808011aa0 con 0x7fc814135eb0 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:17.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: dump: 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.390 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 1 -- 172.21.15.47:0/2337273020 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808011c00 con 0x7fc814135eb0 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc819b6c700 5 monclient: authenticate success, global_id 36008 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc819b6c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 1 -- 172.21.15.47:0/2337273020 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc808011d70 con 0x7fc814135eb0 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc819b6c700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc819b6c700 4 set_mon_vals no callback set 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.376+0000 7fc8127fc700 10 monclient: dump: 2024-04-20T02:31:17.391 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.392 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 10 monclient: shutdown 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 -- 172.21.15.47:0/2337273020 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 msgr2=0x7fc8141362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 0x7fc8141362b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc808009920 tx=0x7fc808009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 -- 172.21.15.47:0/2337273020 shutdown_connections 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc814135ad0 0x7fc81409b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2337273020 >> v2:172.21.15.179:3300/0 conn(0x7fc8140a2950 0x7fc8140a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2337273020 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc814135eb0 0x7fc8141362b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 -- 172.21.15.47:0/2337273020 >> 172.21.15.47:0/2337273020 conn(0x7fc8140a7570 msgr2=0x7fc8140a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 -- 172.21.15.47:0/2337273020 shutdown_connections 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.377+0000 7fc819b6c700 1 -- 172.21.15.47:0/2337273020 wait complete. 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 10 monclient: monmap: 2024-04-20T02:31:17.393 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.394 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 1 librados: starting msgr at 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 1 librados: starting objecter 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 1 Processor -- start 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 1 -- start start 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 1 librados: setting wanted keys 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 1 librados: calling monclient init 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.378+0000 7fc819b6c700 10 monclient: init 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient: _add_conns ranks=[6,5,7,1,4,2,3,0,8] 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8140a2950 0x7fc8141cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): picked mon.noname-d con 0x7fc8140a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc814135ad0 0x7fc8141cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): picked mon.noname-h con 0x7fc814135ad0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:17.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc814135eb0 0x7fc8141235b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): picked mon.noname-i con 0x7fc814135eb0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc814072380 con 0x7fc814135ad0 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc814066ea0 con 0x7fc8140a2950 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc814134690 con 0x7fc814135eb0 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc819b6c700 10 monclient(hunting): authenticate will time out at 2556.780256s 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8140a2950 0x7fc8141cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8140a2950 0x7fc8141cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45738/0 (socket says 172.21.15.47:45738) 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 -- 172.21.15.47:0/2293603537 learned_addr learned my addr 172.21.15.47:0/2293603537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): get_auth_request con 0x7fc8140a2950 auth_method 0 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc814135eb0 0x7fc8141235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc814135ad0 0x7fc8141cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): get_auth_request con 0x7fc814135eb0 auth_method 0 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc812ffd700 10 monclient(hunting): get_auth_request con 0x7fc814135ad0 auth_method 0 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc812ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc812ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc812ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc813fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient(hunting): handle_auth_done global_id 55185 payload 824 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc814135eb0 msgr2=0x7fc8141235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc814135eb0 0x7fc8141235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc814135ad0 msgr2=0x7fc8141cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc814135ad0 0x7fc8141cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 monclient: found mon.noname-d 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:17.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808009550 con 0x7fc8140a2950 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc814135ad0 0x7fc8141cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc8137fe700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8140a2950 0x7fc8141cc500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc80400e810 tx=0x7fc80400eb20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc7fbfff700 10 client.?.objecter ms_handle_connect 0x7fc8140a2950 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.379+0000 7fc7fbfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8040044a0 con 0x7fc8140a2950 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: dump: 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.399 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8040036a0 con 0x7fc8140a2950 2024-04-20T02:31:17.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 5 monclient: authenticate success, global_id 55185 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 10 monclient: _renew_subs 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc814127270 con 0x7fc8140a2950 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc804003c80 con 0x7fc8140a2950 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 10 monclient: dump: 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.401 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 10 client.55185.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 10 monclient: _renew_subs 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8141256c0 con 0x7fc8140a2950 2024-04-20T02:31:17.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc811ffb700 4 set_mon_vals no callback set 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc819b6c700 1 librados: init done 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.380+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc804003810 con 0x7fc8140a2950 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc819b6c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 0x7fc814123af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814068310 con 0x7fc81406aac0 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc80003a2f0 0x7fc80003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7fc80404a2a0 con 0x7fc8140a2950 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 10 client.55185.objecter ms_dispatch 0x7fc814001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 3 client.55185.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 3 client.55185.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 0x7fc814123af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient: get_auth_request con 0x7fc81406aac0 auth_method 0 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc813fff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc80003a2f0 0x7fc80003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc813fff700 10 monclient: get_auth_request con 0x7fc80003a2f0 auth_method 0 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 20 client.55185.objecter dump_active .. 0 homeless 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 10 monclient(hunting): handle_auth_done global_id 55594 payload 824 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc814126cb0 con 0x7fc81406aac0 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 10 client.55185.objecter ms_handle_connect 0x7fc81406aac0 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 0x7fc814123af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a7e80 tx=0x7fc808004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.381+0000 7fc7fbfff700 10 client.55185.objecter resend_mon_ops 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 10 client.55185.objecter ms_handle_connect 0x7fc80003a2f0 2024-04-20T02:31:17.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc813fff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc80003a2f0 0x7fc80003c7b0 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a1370 tx=0x7fc80c00d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc808004570 con 0x7fc81406aac0 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 10 monclient: dump: 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.405 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc8080036a0 con 0x7fc81406aac0 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 msgr2=0x7fc814123af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 0x7fc814123af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a7e80 tx=0x7fc808004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 msgr2=0x7fc814123af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.382+0000 7fc7fbfff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc81406aac0 0x7fc814123af0 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a7e80 tx=0x7fc808004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc819b6c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 0x7fc81406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc819b6c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814127400 con 0x7fc814124030 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 0x7fc81406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient: get_auth_request con 0x7fc814124030 auth_method 0 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 10 monclient(hunting): handle_auth_done global_id 55603 payload 824 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.390+0000 7fc812ffd700 1 -- 172.21.15.47:0/2293603537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc814067a40 con 0x7fc814124030 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc812ffd700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 0x7fc81406f090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc808009130 tx=0x7fc808011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 client.55185.objecter ms_handle_connect 0x7fc814124030 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 client.55185.objecter resend_mon_ops 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc808003d60 con 0x7fc814124030 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 monclient: dump: 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.410 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7fc80802e070 con 0x7fc814124030 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 msgr2=0x7fc81406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 0x7fc81406f090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc808009130 tx=0x7fc808011000 comp rx=0 tx=0).stop 2024-04-20T02:31:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 msgr2=0x7fc81406f090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc7fbfff700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc814124030 0x7fc81406f090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:17.412 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.413 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.415 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.418 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.423 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:17.424 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.426 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.427 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.428 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.429 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:17.431 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.433 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:17.433 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:17.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.391+0000 7fc819b6c700 10 librados: watch_flush enter 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 10 librados: watch_flush exit 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 20 client.55185.objecter shutdown clearing up homeless session... 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 10 client.55185.objecter successfully canceled tick 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc80003a2f0 msgr2=0x7fc80003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc80003a2f0 0x7fc80003c7b0 secure :-1 s=READY pgs=3341 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a1370 tx=0x7fc80c00d040 comp rx=0 tx=0).stop 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 10 monclient: shutdown 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8140a2950 msgr2=0x7fc8141cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 --2- 172.21.15.47:0/2293603537 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc8140a2950 0x7fc8141cc500 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc80400e810 tx=0x7fc80400eb20 comp rx=0 tx=0).stop 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc813fff700 1 -- 172.21.15.47:0/2293603537 reap_dead start 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc813fff700 1 -- 172.21.15.47:0/2293603537 reap_dead start 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 shutdown_connections 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 >> 172.21.15.47:0/2293603537 conn(0x7fc8140a7570 msgr2=0x7fc81409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 shutdown_connections 2024-04-20T02:31:17.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 -- 172.21.15.47:0/2293603537 wait complete. 2024-04-20T02:31:17.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.392+0000 7fc819b6c700 1 librados: shutdown 2024-04-20T02:31:17.436 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:17.436 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:31:17.436 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:31:17.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.575+0000 7f03c6c2c700 10 monclient: get_monmap_and_config 2024-04-20T02:31:17.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.575+0000 7f03c6c2c700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.575+0000 7f03c6c2c700 10 monclient: monmap: 2024-04-20T02:31:17.575 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.575 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:17.576469+0000 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:17.576469+0000 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:17.576 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 Processor -- start 2024-04-20T02:31:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 -- start start 2024-04-20T02:31:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient: init 2024-04-20T02:31:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient: _add_conns ranks=[0,9,5,4,1,6,8,10,3,7,2] 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f03c006cf70 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03c006d350 0x7f03c013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): picked mon.noname-a con 0x7f03c006d350 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c013f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): picked mon.noname-h con 0x7f03c013d3f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c00751e0 con 0x7f03c006cf70 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c00702e0 con 0x7f03c006d350 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00a3fb0 con 0x7f03c013d3f0 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c6c2c700 10 monclient(hunting): authenticate will time out at 2556.977946s 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03c49c8700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f03c006cf70 0x7f03c006c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37864/0 (socket says 172.21.15.47:37864) 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.576+0000 7f03bffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f03c006d350 0x7f03c013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c49c8700 1 -- 172.21.15.47:0/947125885 learned_addr learned my addr 172.21.15.47:0/947125885 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): get_auth_request con 0x7f03c006d350 auth_method 0 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c51c9700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c013f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c51c9700 10 monclient(hunting): get_auth_request con 0x7f03c013d3f0 auth_method 0 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c51c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c51c9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c51c9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient(hunting): handle_auth_done global_id 36801 payload 274 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 1 -- 172.21.15.47:0/947125885 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 msgr2=0x7f03c013f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c013f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 1 -- 172.21.15.47:0/947125885 >> v1:172.21.15.47:6789/0 conn(0x7f03c006cf70 legacy=0x7f03c006c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 1 monclient: found mon.noname-a 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 1 -- 172.21.15.47:0/947125885 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c006e4c0 con 0x7f03c006d350 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c51c9700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c013f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bffff700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006d350 0x7f03c013ceb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009840 tx=0x7f03a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94853e999d8f08c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 1 -- 172.21.15.47:0/947125885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03a8011aa0 con 0x7f03c006d350 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: dump: 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c6c2c700 5 monclient: authenticate success, global_id 36801 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 1 -- 172.21.15.47:0/947125885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8011c00 con 0x7f03c006d350 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c6c2c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c6c2c700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 1 -- 172.21.15.47:0/947125885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03a8011d70 con 0x7f03c006d350 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03c6c2c700 4 set_mon_vals no callback set 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.577+0000 7f03bf7fe700 10 monclient: dump: 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 10 monclient: shutdown 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 -- 172.21.15.47:0/947125885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006d350 msgr2=0x7f03c013ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006d350 0x7f03c013ceb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009840 tx=0x7f03a8009b50 comp rx=0 tx=0).stop 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 -- 172.21.15.47:0/947125885 shutdown_connections 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c013f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/947125885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006d350 0x7f03c013ceb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 -- 172.21.15.47:0/947125885 >> 172.21.15.47:0/947125885 conn(0x7f03c00a4a50 msgr2=0x7f03c00a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 -- 172.21.15.47:0/947125885 shutdown_connections 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.578+0000 7f03c6c2c700 1 -- 172.21.15.47:0/947125885 wait complete. 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 10 monclient: monmap: 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 1 librados: starting msgr at 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 1 librados: starting objecter 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 1 Processor -- start 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 1 -- start start 2024-04-20T02:31:17.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 1 librados: setting wanted keys 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 1 librados: calling monclient init 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.579+0000 7f03c6c2c700 10 monclient: init 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient: _add_conns ranks=[0,1,5,3,4,7,2,8,6] 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006cf70 0x7f03c011e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): picked mon.noname-a con 0x7f03c006cf70 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c006d350 0x7f03c01ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): picked mon.noname-f con 0x7f03c006d350 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c01cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): picked mon.noname-h con 0x7f03c013d3f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c006da70 con 0x7f03c006cf70 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c011e6c0 con 0x7f03c006d350 2024-04-20T02:31:17.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00720f0 con 0x7f03c013d3f0 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c6c2c700 10 monclient(hunting): authenticate will time out at 2556.981331s 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006cf70 0x7f03c011e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006cf70 0x7f03c011e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49884/0 (socket says 172.21.15.47:49884) 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 -- 172.21.15.47:0/4211320784 learned_addr learned my addr 172.21.15.47:0/4211320784 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): get_auth_request con 0x7f03c006cf70 auth_method 0 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bffff700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c006d350 0x7f03c01ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c51c9700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c01cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bffff700 10 monclient(hunting): get_auth_request con 0x7f03c006d350 auth_method 0 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c51c9700 10 monclient(hunting): get_auth_request con 0x7f03c013d3f0 auth_method 0 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c51c9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c51c9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c51c9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient(hunting): handle_auth_done global_id 36819 payload 824 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 msgr2=0x7f03c01cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f03c013d3f0 0x7f03c01cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c006d350 msgr2=0x7f03c01ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c006d350 0x7f03c01ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 monclient: found mon.noname-a 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80094a0 con 0x7f03c006cf70 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bffff700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03c006d350 0x7f03c01ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03c49c8700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006cf70 0x7f03c011e8e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c490 tx=0x7f03b000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bd7fa700 10 client.?.objecter ms_handle_connect 0x7f03c006cf70 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.580+0000 7f03bd7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03b00070d0 con 0x7f03c006cf70 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: dump: 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.599 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 5 monclient: authenticate success, global_id 36819 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 10 monclient: _renew_subs 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0007230 con 0x7f03c006cf70 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c011fec0 con 0x7f03c006cf70 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03b0007810 con 0x7f03c006cf70 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 10 client.36819.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03beffd700 4 set_mon_vals no callback set 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03bd7fa700 10 monclient: dump: 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.601 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 10 monclient: _renew_subs 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c00a1ae0 con 0x7f03c006cf70 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 1 librados: init done 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.581+0000 7f03c6c2c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 0x7f03c00a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0068430 con 0x7f03c006aac0 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f03b00073a0 con 0x7f03c006cf70 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 0x7f03c00a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient: get_auth_request con 0x7f03c006aac0 auth_method 0 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f03a003a2f0 0x7f03a003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f03b0049970 con 0x7f03c006cf70 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 10 client.36819.objecter ms_dispatch 0x7f03c0001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 3 client.36819.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 3 client.36819.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 20 client.36819.objecter dump_active .. 0 homeless 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03c51c9700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f03a003a2f0 0x7f03a003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03c51c9700 10 monclient: get_auth_request con 0x7f03a003a2f0 auth_method 0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 10 monclient(hunting): handle_auth_done global_id 94233 payload 824 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03c0071f20 con 0x7f03c006aac0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bffff700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 0x7f03c00a0010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009840 tx=0x7f03a8004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 10 client.36819.objecter ms_handle_connect 0x7f03c006aac0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 10 client.36819.objecter resend_mon_ops 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03bd7fa700 10 client.36819.objecter ms_handle_connect 0x7f03a003a2f0 2024-04-20T02:31:17.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.582+0000 7f03c51c9700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f03a003a2f0 0x7f03a003c7b0 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 crypto rx=0x7f03c00676f0 tx=0x7f03b8003a80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03a801f070 con 0x7f03c006aac0 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 10 monclient: dump: 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.605 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f03a80036a0 con 0x7f03c006aac0 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 msgr2=0x7f03c00a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 0x7f03c00a0010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009840 tx=0x7f03a8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 msgr2=0x7f03c00a0010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.583+0000 7f03bd7fa700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c006aac0 0x7f03c00a0010 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009840 tx=0x7f03a8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03c6c2c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 0x7f03c00a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03c6c2c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c006a400 con 0x7f03c00a0550 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 0x7f03c00a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient: get_auth_request con 0x7f03c00a0550 auth_method 0 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 10 monclient(hunting): handle_auth_done global_id 94242 payload 824 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.591+0000 7f03bffff700 1 -- 172.21.15.47:0/4211320784 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03c006e1e0 con 0x7f03c00a0550 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bffff700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 0x7f03c00a0960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03c00a1cd0 tx=0x7f03a8011000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 client.36819.objecter ms_handle_connect 0x7f03c00a0550 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 client.36819.objecter resend_mon_ops 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03a8004660 con 0x7f03c00a0550 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 monclient: dump: 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.609 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9183 (secure 0 0 0) 0x7f03a8036440 con 0x7f03c00a0550 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 msgr2=0x7f03c00a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 0x7f03c00a0960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03c00a1cd0 tx=0x7f03a8011000 comp rx=0 tx=0).stop 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 msgr2=0x7f03c00a0960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03bd7fa700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03c00a0550 0x7f03c00a0960 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03c00a1cd0 tx=0x7f03a8011000 comp rx=0 tx=0).stop 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:17.611 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:17.612 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:17.613 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.614 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:17.615 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:17.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.617 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.618 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.619 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.620 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.621 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.622 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.623 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.624 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.626 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.628 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.629 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.631 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.632 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.632 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.633 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:17.634 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03c6c2c700 10 librados: watch_flush enter 2024-04-20T02:31:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03c6c2c700 10 librados: watch_flush exit 2024-04-20T02:31:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03c6c2c700 20 client.36819.objecter shutdown clearing up homeless session... 2024-04-20T02:31:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03c6c2c700 10 client.36819.objecter successfully canceled tick 2024-04-20T02:31:17.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f03a003a2f0 msgr2=0x7f03a003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.592+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f03a003a2f0 0x7f03a003c7b0 secure :-1 s=READY pgs=3342 cs=0 l=1 rev1=1 crypto rx=0x7f03c00676f0 tx=0x7f03b8003a80 comp rx=0 tx=0).stop 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 10 monclient: shutdown 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006cf70 msgr2=0x7f03c011e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 --2- 172.21.15.47:0/4211320784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03c006cf70 0x7f03c011e8e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c490 tx=0x7f03b000c850 comp rx=0 tx=0).stop 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c51c9700 1 -- 172.21.15.47:0/4211320784 reap_dead start 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c51c9700 1 -- 172.21.15.47:0/4211320784 reap_dead start 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 shutdown_connections 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 >> 172.21.15.47:0/4211320784 conn(0x7f03c00a4a50 msgr2=0x7f03c00a6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 shutdown_connections 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 -- 172.21.15.47:0/4211320784 wait complete. 2024-04-20T02:31:17.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.593+0000 7f03c6c2c700 1 librados: shutdown 2024-04-20T02:31:17.638 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:17.638 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:31:17.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:31:17.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.775+0000 7fcc41aa1700 10 monclient: get_monmap_and_config 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.775+0000 7fcc41aa1700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.775+0000 7fcc41aa1700 10 monclient: monmap: 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:17.776873+0000 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:17.776873+0000 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.776 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:17.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.776+0000 7fcc41aa1700 1 Processor -- start 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.776+0000 7fcc41aa1700 1 -- start start 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.776+0000 7fcc41aa1700 10 monclient: init 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient: _add_conns ranks=[8,0,10,5,3,9,6,4,1,7,2] 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcc3c09ce50 0x7fcc3c09c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): picked mon.noname-e con 0x7fcc3c09ce50 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:17.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcc3c0a1f10 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcc3c0a2a40 0x7fcc3c13f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): picked mon.noname-f con 0x7fcc3c0a2a40 addr v2:172.21.15.179:3300/0 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3c0751e0 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3c09bc40 con 0x7fcc3c0a2a40 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c09ab10 con 0x7fcc3c09ce50 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc41aa1700 10 monclient(hunting): authenticate will time out at 2557.178247s 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3affd700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcc3c0a1f10 0x7fcc3c0a2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37870/0 (socket says 172.21.15.47:37870) 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3affd700 1 -- 172.21.15.47:0/3375022899 learned_addr learned my addr 172.21.15.47:0/3375022899 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/3375022899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcc3c09ce50 0x7fcc3c09c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): get_auth_request con 0x7fcc3c09ce50 auth_method 0 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 1 --2- 172.21.15.47:0/3375022899 >> v2:172.21.15.179:3300/0 conn(0x7fcc3c0a2a40 0x7fcc3c13f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): get_auth_request con 0x7fcc3c0a2a40 auth_method 0 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3a7fc700 1 -- 172.21.15.47:0/3375022899 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196287040 0 0) 0x7fcc3c0751e0 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3a7fc700 10 monclient(hunting): _negotiate 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3a7fc700 10 monclient(hunting): my global_id is 36828 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3a7fc700 1 -- 172.21.15.47:0/3375022899 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc280035b0 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_done global_id 36017 payload 274 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/3375022899 >> v2:172.21.15.179:3300/0 conn(0x7fcc3c0a2a40 msgr2=0x7fcc3c13f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/3375022899 >> v2:172.21.15.179:3300/0 conn(0x7fcc3c0a2a40 0x7fcc3c13f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/3375022899 >> v1:172.21.15.47:6789/0 conn(0x7fcc3c0a1f10 legacy=0x7fcc3c0a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 monclient: found mon.noname-e 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/3375022899 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c09b680 con 0x7fcc3c09ce50 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.777+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/3375022899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcc3c09ce50 0x7fcc3c09c3b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009840 tx=0x7fcc2c009b50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5381e6df1cb235b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 1 -- 172.21.15.47:0/3375022899 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc2c011aa0 con 0x7fcc3c09ce50 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: dump: 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.791 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 1 -- 172.21.15.47:0/3375022899 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c011c00 con 0x7fcc3c09ce50 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 1 -- 172.21.15.47:0/3375022899 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc2c011d70 con 0x7fcc3c09ce50 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 5 monclient: authenticate success, global_id 36017 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 4 set_mon_vals no callback set 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc3a7fc700 10 monclient: dump: 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.792 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 10 monclient: shutdown 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 -- 172.21.15.47:0/3375022899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcc3c09ce50 msgr2=0x7fcc3c09c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/3375022899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcc3c09ce50 0x7fcc3c09c3b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009840 tx=0x7fcc2c009b50 comp rx=0 tx=0).stop 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 -- 172.21.15.47:0/3375022899 shutdown_connections 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/3375022899 >> v2:172.21.15.179:3300/0 conn(0x7fcc3c0a2a40 0x7fcc3c13f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/3375022899 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fcc3c09ce50 0x7fcc3c09c3b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 -- 172.21.15.47:0/3375022899 >> 172.21.15.47:0/3375022899 conn(0x7fcc3c0a3000 msgr2=0x7fcc3c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 -- 172.21.15.47:0/3375022899 shutdown_connections 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.778+0000 7fcc41aa1700 1 -- 172.21.15.47:0/3375022899 wait complete. 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.779+0000 7fcc41aa1700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.779+0000 7fcc41aa1700 10 monclient: monmap: 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:17.794 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.779+0000 7fcc41aa1700 1 librados: starting msgr at 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.779+0000 7fcc41aa1700 1 librados: starting objecter 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.779+0000 7fcc41aa1700 1 Processor -- start 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 -- start start 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 librados: setting wanted keys 2024-04-20T02:31:17.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 librados: calling monclient init 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient: init 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient: _add_conns ranks=[7,0,5,6,2,1,4,3,8] 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc3c09ce50 0x7fcc3c0902d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): picked mon.noname-i con 0x7fcc3c09ce50 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 0x7fcc3c090810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): picked mon.noname-a con 0x7fcc3c0a1f10 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c0a2a40 0x7fcc3c090d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): picked mon.noname-h con 0x7fcc3c0a2a40 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c093030 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c091aa0 con 0x7fcc3c0a2a40 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c09ad50 con 0x7fcc3c09ce50 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc41aa1700 10 monclient(hunting): authenticate will time out at 2557.181764s 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 0x7fcc3c090810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 0x7fcc3c090810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49890/0 (socket says 172.21.15.47:49890) 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 1 -- 172.21.15.47:0/4181679420 learned_addr learned my addr 172.21.15.47:0/4181679420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 10 monclient(hunting): get_auth_request con 0x7fcc3c0a1f10 auth_method 0 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.780+0000 7fcc3affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3bfff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c0a2a40 0x7fcc3c090d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc3c09ce50 0x7fcc3c0902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3bfff700 10 monclient(hunting): get_auth_request con 0x7fcc3c0a2a40 auth_method 0 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 10 monclient(hunting): handle_auth_done global_id 36837 payload 824 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc3c09ce50 msgr2=0x7fcc3c0902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc3c09ce50 0x7fcc3c0902d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c0a2a40 msgr2=0x7fcc3c090d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c0a2a40 0x7fcc3c090d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 monclient: found mon.noname-a 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0094a0 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3b7fe700 10 monclient: get_auth_request con 0x7fcc3c09ce50 auth_method 0 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fcc3c09ce50 0x7fcc3c0902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:31:17.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc3affd700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 0x7fcc3c090810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcc24001eb0 tx=0x7fcc2400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 client.?.objecter ms_handle_connect 0x7fcc3c0a1f10 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc240070d0 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: dump: 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.799 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24007270 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 5 monclient: authenticate success, global_id 36837 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 10 monclient: _renew_subs 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3c09a900 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc24010300 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc33fff700 10 monclient: dump: 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.801 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc39ffb700 4 set_mon_vals no callback set 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 10 client.36837.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 10 monclient: _renew_subs 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c091570 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.781+0000 7fcc41aa1700 1 librados: init done 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fcc24005b70 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc41aa1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 0x7fcc3c06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c068180 con 0x7fcc3c06aac0 2024-04-20T02:31:17.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc1c03a2f0 0x7fcc1c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7fcc24049ad0 con 0x7fcc3c0a1f10 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 10 client.36837.objecter ms_dispatch 0x7fcc3c001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 3 client.36837.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 3 client.36837.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 0x7fcc3c06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient: get_auth_request con 0x7fcc3c06aac0 auth_method 0 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc33fff700 20 client.36837.objecter dump_active .. 0 homeless 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3bfff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc1c03a2f0 0x7fcc1c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3bfff700 10 monclient: get_auth_request con 0x7fcc1c03a2f0 auth_method 0 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.782+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_done global_id 84389 payload 824 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc3c0668b0 con 0x7fcc3c06aac0 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 client.36837.objecter ms_handle_connect 0x7fcc3c06aac0 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 client.36837.objecter resend_mon_ops 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 client.36837.objecter ms_handle_connect 0x7fcc1c03a2f0 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 0x7fcc3c06ef30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc3bfff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc1c03a2f0 0x7fcc1c03c7b0 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0a2ef0 tx=0x7fcc3400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc2c004430 con 0x7fcc3c06aac0 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 monclient: dump: 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.804 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcc2c004590 con 0x7fcc3c06aac0 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 msgr2=0x7fcc3c06ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 0x7fcc3c06ef30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 msgr2=0x7fcc3c06ef30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.783+0000 7fcc33fff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06aac0 0x7fcc3c06ef30 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c004750 comp rx=0 tx=0).stop 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc41aa1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 0x7fcc3c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc41aa1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c06a400 con 0x7fcc3c06f470 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 0x7fcc3c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient: get_auth_request con 0x7fcc3c06f470 auth_method 0 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 10 monclient(hunting): handle_auth_done global_id 84398 payload 824 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.792+0000 7fcc3b7fe700 1 -- 172.21.15.47:0/4181679420 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc3c09fae0 con 0x7fcc3c06f470 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 client.36837.objecter ms_handle_connect 0x7fcc3c06f470 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 client.36837.objecter resend_mon_ops 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc3b7fe700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 0x7fcc3c06f880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009b90 tx=0x7fcc2c009210 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc2c02b300 con 0x7fcc3c06f470 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 monclient: dump: 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.809 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7fcc2c02b8e0 con 0x7fcc3c06f470 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 msgr2=0x7fcc3c06f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 0x7fcc3c06f880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009b90 tx=0x7fcc2c009210 comp rx=0 tx=0).stop 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 msgr2=0x7fcc3c06f880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc33fff700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fcc3c06f470 0x7fcc3c06f880 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009b90 tx=0x7fcc2c009210 comp rx=0 tx=0).stop 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:17.811 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.812 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.813 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:17.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:17.815 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:17.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.818 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:17.820 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.821 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.823 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.824 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.825 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.826 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.827 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:17.828 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.829 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:17.830 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:17.831 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:17.831 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:17.831 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:17.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:17.831 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:17.831 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:17.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 10 librados: watch_flush enter 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 10 librados: watch_flush exit 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 20 client.36837.objecter shutdown clearing up homeless session... 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 10 client.36837.objecter successfully canceled tick 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc1c03a2f0 msgr2=0x7fcc1c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fcc1c03a2f0 0x7fcc1c03c7b0 secure :-1 s=READY pgs=3343 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0a2ef0 tx=0x7fcc3400b040 comp rx=0 tx=0).stop 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 10 monclient: shutdown 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.793+0000 7fcc41aa1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 msgr2=0x7fcc3c090810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 0x7fcc3c090810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcc24001eb0 tx=0x7fcc2400c860 comp rx=0 tx=0).stop 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc3bfff700 1 -- 172.21.15.47:0/4181679420 reap_dead start 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 shutdown_connections 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 --2- 172.21.15.47:0/4181679420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc3c0a1f10 0x7fcc3c090810 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 >> 172.21.15.47:0/4181679420 conn(0x7fcc3c0a3000 msgr2=0x7fcc3c1bf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 shutdown_connections 2024-04-20T02:31:17.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 -- 172.21.15.47:0/4181679420 wait complete. 2024-04-20T02:31:17.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.794+0000 7fcc41aa1700 1 librados: shutdown 2024-04-20T02:31:17.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:17.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:31:17.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:31:17.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.972+0000 7efe74b41700 10 monclient: get_monmap_and_config 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.972+0000 7efe74b41700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.972+0000 7efe74b41700 10 monclient: monmap: 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:17.973952+0000 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:17.973952+0000 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.973 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:17.974 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.973+0000 7efe74b41700 1 Processor -- start 2024-04-20T02:31:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.973+0000 7efe74b41700 1 -- start start 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.973+0000 7efe74b41700 10 monclient: init 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient: _add_conns ranks=[2,4,6,9,7,5,0,10,8,1,3] 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe7009fa80 0x7efe70135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): picked mon.noname-b con 0x7efe7009fa80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 0x7efe700a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): picked mon.noname-c con 0x7efe7009fe60 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe700a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): picked mon.noname-d con 0x7efe700a1f10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700a0780 con 0x7efe7009fa80 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe7009df90 con 0x7efe7009fe60 2024-04-20T02:31:17.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efe7009dcb0 con 0x7efe700a1f10 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe74b41700 10 monclient(hunting): authenticate will time out at 2557.375317s 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe7009fa80 0x7efe70135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 0x7efe700a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe700a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe7009fa80 0x7efe70135df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36888/0 (socket says 172.21.15.47:36888) 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 0x7efe700a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55596/0 (socket says 172.21.15.47:55596) 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 1 -- 172.21.15.47:0/4225526570 learned_addr learned my addr 172.21.15.47:0/4225526570 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): get_auth_request con 0x7efe7009fa80 auth_method 0 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): get_auth_request con 0x7efe700a1f10 auth_method 0 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): get_auth_request con 0x7efe7009fe60 auth_method 0 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_done global_id 55194 payload 274 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 -- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 msgr2=0x7efe700a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 0x7efe700a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 -- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe7009fa80 msgr2=0x7efe70135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe7009fa80 0x7efe70135df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 monclient: found mon.noname-d 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6ed9d700 1 -- 172.21.15.47:0/4225526570 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700751e0 con 0x7efe700a1f10 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.974+0000 7efe6dd9b700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 0x7efe700a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6ed9d700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe700a2350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe68009dc0 tx=0x7efe6800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d6ef86d5333fa72d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 1 -- 172.21.15.47:0/4225526570 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe6800e070 con 0x7efe700a1f10 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: dump: 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.989 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 1 -- 172.21.15.47:0/4225526570 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe680098f0 con 0x7efe700a1f10 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 5 monclient: authenticate success, global_id 55194 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 1 -- 172.21.15.47:0/4225526570 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe680128a0 con 0x7efe700a1f10 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 4 set_mon_vals no callback set 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe6d59a700 10 monclient: dump: 2024-04-20T02:31:17.990 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 10 monclient: shutdown 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 -- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 msgr2=0x7efe700a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe700a2350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe68009dc0 tx=0x7efe6800b330 comp rx=0 tx=0).stop 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 -- 172.21.15.47:0/4225526570 shutdown_connections 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe700a2350 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7efe7009fa80 0x7efe70135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 --2- 172.21.15.47:0/4225526570 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7efe7009fe60 0x7efe700a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 -- 172.21.15.47:0/4225526570 >> 172.21.15.47:0/4225526570 conn(0x7efe700a3000 msgr2=0x7efe700a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 -- 172.21.15.47:0/4225526570 shutdown_connections 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.975+0000 7efe74b41700 1 -- 172.21.15.47:0/4225526570 wait complete. 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.976+0000 7efe74b41700 10 monclient: build_initial_monmap 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.976+0000 7efe74b41700 10 monclient: monmap: 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.976+0000 7efe74b41700 1 librados: starting msgr at 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 librados: starting objecter 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 Processor -- start 2024-04-20T02:31:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 -- start start 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 librados: setting wanted keys 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 librados: calling monclient init 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient: init 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient: _add_conns ranks=[5,3,6,0,1,2,4,8,7] 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efe7009fa80 0x7efe70071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): picked mon.noname-h con 0x7efe7009fa80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efe7009fe60 0x7efe70132aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): picked mon.noname-g con 0x7efe7009fe60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe70132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): picked mon.noname-d con 0x7efe700a1f10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7009d080 con 0x7efe7009fe60 2024-04-20T02:31:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe701ca610 con 0x7efe7009fa80 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efe7009cb50 con 0x7efe700a1f10 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe74b41700 10 monclient(hunting): authenticate will time out at 2557.378687s 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe70132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe70132fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45748/0 (socket says 172.21.15.47:45748) 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 1 -- 172.21.15.47:0/2746284712 learned_addr learned my addr 172.21.15.47:0/2746284712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): get_auth_request con 0x7efe700a1f10 auth_method 0 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6e59c700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efe7009fa80 0x7efe70071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6dd9b700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efe7009fe60 0x7efe70132aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6e59c700 10 monclient(hunting): get_auth_request con 0x7efe7009fa80 auth_method 0 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6dd9b700 10 monclient(hunting): get_auth_request con 0x7efe7009fe60 auth_method 0 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6dd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6dd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.977+0000 7efe6dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 10 monclient(hunting): handle_auth_done global_id 55203 payload 824 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efe7009fa80 msgr2=0x7efe70071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7efe7009fa80 0x7efe70071fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efe7009fe60 msgr2=0x7efe70132aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efe7009fe60 0x7efe70132aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 monclient: found mon.noname-d 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580094a0 con 0x7efe700a1f10 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6dd9b700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efe7009fe60 0x7efe70132aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6ed9d700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe70132fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe68014040 tx=0x7efe68004ad0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 client.?.objecter ms_handle_connect 0x7efe700a1f10 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe680047c0 con 0x7efe700a1f10 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: dump: 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: _finish_auth 0 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe68004960 con 0x7efe700a1f10 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 5 monclient: authenticate success, global_id 55203 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe680128a0 con 0x7efe700a1f10 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 10 monclient: _renew_subs 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe68009a70 con 0x7efe700a1f10 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe6cd99700 4 set_mon_vals no callback set 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 10 client.55203.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe66ffd700 10 monclient: dump: 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.000 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 10 monclient: _renew_subs 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7012dca0 con 0x7efe700a1f10 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.978+0000 7efe74b41700 1 librados: init done 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe74b41700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe74b41700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 0x7efe7012b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efe7012e040 con 0x7efe7006aac0 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7efe68012430 con 0x7efe700a1f10 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe5003a2f0 0x7efe5003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 0x7efe7012b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient: get_auth_request con 0x7efe7006aac0 auth_method 0 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7efe6804c620 con 0x7efe700a1f10 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 10 client.55203.objecter ms_dispatch 0x7efe70001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 3 client.55203.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 3 client.55203.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe66ffd700 20 client.55203.objecter dump_active .. 0 homeless 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6dd9b700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe5003a2f0 0x7efe5003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.979+0000 7efe6dd9b700 10 monclient: get_auth_request con 0x7efe5003a2f0 auth_method 0 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe6e59c700 10 monclient(hunting): handle_auth_done global_id 84400 payload 824 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe6e59c700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe7009e550 con 0x7efe7006aac0 2024-04-20T02:31:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 client.55203.objecter ms_handle_connect 0x7efe7006aac0 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 client.55203.objecter resend_mon_ops 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe6e59c700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 0x7efe7012b5c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe600119d0 tx=0x7efe60011d90 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 client.55203.objecter ms_handle_connect 0x7efe5003a2f0 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe6dd9b700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe5003a2f0 0x7efe5003c7b0 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7efe70072700 tx=0x7efe58008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe6000fe40 con 0x7efe7006aac0 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 monclient: dump: 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efe6000f9d0 con 0x7efe7006aac0 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 msgr2=0x7efe7012b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 0x7efe7012b5c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe600119d0 tx=0x7efe60011d90 comp rx=0 tx=0).stop 2024-04-20T02:31:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 msgr2=0x7efe7012b5c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.980+0000 7efe66ffd700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7006aac0 0x7efe7012b5c0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe600119d0 tx=0x7efe60011d90 comp rx=0 tx=0).stop 2024-04-20T02:31:18.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe74b41700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:18.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe74b41700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 0x7efe7012bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe74b41700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efe70068180 con 0x7efe7012bb00 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe6e59c700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 0x7efe7012bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe6e59c700 10 monclient: get_auth_request con 0x7efe7012bb00 auth_method 0 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe6e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe6e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.988+0000 7efe6e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe6e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe6e59c700 10 monclient(hunting): handle_auth_done global_id 84409 payload 824 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe6e59c700 1 -- 172.21.15.47:0/2746284712 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efe700668e0 con 0x7efe7012bb00 2024-04-20T02:31:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe6e59c700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 0x7efe7012bf10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe600119d0 tx=0x7efe60004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe66ffd700 10 client.55203.objecter ms_handle_connect 0x7efe7012bb00 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe66ffd700 10 client.55203.objecter resend_mon_ops 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe60003d80 con 0x7efe7012bb00 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe66ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe66ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.989+0000 7efe66ffd700 10 monclient: dump: 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9353 (secure 0 0 0) 0x7efe600273a0 con 0x7efe7012bb00 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 msgr2=0x7efe7012bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 0x7efe7012bf10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe600119d0 tx=0x7efe60004750 comp rx=0 tx=0).stop 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 msgr2=0x7efe7012bf10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe66ffd700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7efe7012bb00 0x7efe7012bf10 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efe600119d0 tx=0x7efe60004750 comp rx=0 tx=0).stop 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 98, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:18.010 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:18.011 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:18.012 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:18.013 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:18.014 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:18.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:18.017 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:18.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:18.020 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:18.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.026 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:18.027 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.028 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:18.029 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:18.030 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:18.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 10 librados: watch_flush enter 2024-04-20T02:31:18.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 10 librados: watch_flush exit 2024-04-20T02:31:18.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 20 client.55203.objecter shutdown clearing up homeless session... 2024-04-20T02:31:18.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 10 client.55203.objecter successfully canceled tick 2024-04-20T02:31:18.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe5003a2f0 msgr2=0x7efe5003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7efe5003a2f0 0x7efe5003c7b0 secure :-1 s=READY pgs=3344 cs=0 l=1 rev1=1 crypto rx=0x7efe70072700 tx=0x7efe58008040 comp rx=0 tx=0).stop 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 10 monclient: shutdown 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 msgr2=0x7efe70132fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 1 --2- 172.21.15.47:0/2746284712 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7efe700a1f10 0x7efe70132fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efe68014040 tx=0x7efe68004ad0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe6ed9d700 1 -- 172.21.15.47:0/2746284712 reap_dead start 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe6ed9d700 1 -- 172.21.15.47:0/2746284712 reap_dead start 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 shutdown_connections 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.990+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 >> 172.21.15.47:0/2746284712 conn(0x7efe700a3000 msgr2=0x7efe7009ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.991+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 shutdown_connections 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.991+0000 7efe74b41700 1 -- 172.21.15.47:0/2746284712 wait complete. 2024-04-20T02:31:18.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:17.991+0000 7efe74b41700 1 librados: shutdown 2024-04-20T02:31:18.034 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:31:18.034 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'b', 'g'] 2024-04-20T02:31:18.034 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-20T02:31:18.034 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-20T02:31:18.034 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-20T02:31:18.034 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T02:31:18.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:31:18.052 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T02:31:18.052 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-20T02:31:18.052 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b store 2024-04-20T02:31:18.052 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b sync_force --yes-i-really-mean-it 2024-04-20T02:31:18.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.234+0000 7f4eb44c1700 10 monclient: get_monmap_and_config 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.234+0000 7f4eb44c1700 10 monclient: build_initial_monmap 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.234+0000 7f4eb44c1700 10 monclient: monmap: 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:18.235823+0000 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:18.235823+0000 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:18.235 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:18.236 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.235+0000 7f4eb44c1700 1 Processor -- start 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.235+0000 7f4eb44c1700 1 -- start start 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.235+0000 7f4eb44c1700 10 monclient: init 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient: _add_conns ranks=[6,2,9,4,1,3,7,10,5,0,8] 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 0x7f4eac0a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): picked mon.noname-d con 0x7f4eac0a27f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a2bd0 0x7f4eac09c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): picked mon.noname-b con 0x7f4eac0a2bd0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:18.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4eac09c8a0 0x7f4eac09cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): picked mon.noname-a con 0x7f4eac09c8a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac066c80 con 0x7f4eac09c8a0 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac09b980 con 0x7f4eac0a2bd0 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac09d260 con 0x7f4eac0a27f0 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb44c1700 10 monclient(hunting): authenticate will time out at 2557.637203s 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb2a5e700 1 -- >> v2:172.21.15.47:3300/0 conn(0x7f4eac09c8a0 msgr2=0x7f4eac09cd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:18.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb2a5e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4eac09c8a0 0x7f4eac09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 0x7f4eac0a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 0x7f4eac0a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45764/0 (socket says 172.21.15.47:45764) 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a2bd0 0x7f4eac09c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 -- 172.21.15.47:0/387050126 learned_addr learned my addr 172.21.15.47:0/387050126 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): get_auth_request con 0x7f4eac0a27f0 auth_method 0 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): get_auth_request con 0x7f4eac0a2bd0 auth_method 0 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient(hunting): handle_auth_done global_id 55212 payload 274 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 -- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a2bd0 msgr2=0x7f4eac09c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 --2- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a2bd0 0x7f4eac09c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 -- 172.21.15.47:0/387050126 >> v2:172.21.15.47:3300/0 conn(0x7f4eac09c8a0 msgr2=0x7f4eac09cd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 --2- 172.21.15.47:0/387050126 >> v2:172.21.15.47:3300/0 conn(0x7f4eac09c8a0 0x7f4eac09cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 monclient: found mon.noname-d 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 -- 172.21.15.47:0/387050126 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0751e0 con 0x7f4eac0a27f0 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb1a5c700 1 --2- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a2bd0 0x7f4eac09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.236+0000 7f4eb225d700 1 --2- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 0x7f4eac0a1ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098f0 tx=0x7f4ea8009c00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2e559e2834aee263 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 1 -- 172.21.15.47:0/387050126 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ea8011aa0 con 0x7f4eac0a27f0 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: dump: 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.250 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: _finish_auth 0 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: _check_auth_tickets 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 5 monclient: authenticate success, global_id 55212 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 1 -- 172.21.15.47:0/387050126 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8011c00 con 0x7f4eac0a27f0 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 4 set_mon_vals no callback set 2024-04-20T02:31:18.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 1 -- 172.21.15.47:0/387050126 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ea8011d70 con 0x7f4eac0a27f0 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb125b700 10 monclient: dump: 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.252 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 10 monclient: shutdown 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 -- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 msgr2=0x7f4eac0a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 0x7f4eac0a1ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80098f0 tx=0x7f4ea8009c00 comp rx=0 tx=0).stop 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 -- 172.21.15.47:0/387050126 shutdown_connections 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/387050126 >> v2:172.21.15.47:3300/0 conn(0x7f4eac09c8a0 0x7f4eac09cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a27f0 0x7f4eac0a1ef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/387050126 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a2bd0 0x7f4eac09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 -- 172.21.15.47:0/387050126 >> 172.21.15.47:0/387050126 conn(0x7f4eac0a2fe0 msgr2=0x7f4eac0a33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 -- 172.21.15.47:0/387050126 shutdown_connections 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.237+0000 7f4eb44c1700 1 -- 172.21.15.47:0/387050126 wait complete. 2024-04-20T02:31:18.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 10 monclient: build_initial_monmap 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 10 monclient: monmap: 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:18.254 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 1 librados: starting msgr at 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 1 librados: starting objecter 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 1 Processor -- start 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 1 -- start start 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 1 librados: setting wanted keys 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 1 librados: calling monclient init 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.238+0000 7f4eb44c1700 10 monclient: init 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient: _add_conns ranks=[4,2,6,7,3,0,5,8,1] 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4eac09c8a0 0x7f4eac1ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): picked mon.noname-c con 0x7f4eac09c8a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a27f0 0x7f4eac1cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): picked mon.noname-b con 0x7f4eac0a27f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a2bd0 0x7f4eac06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): picked mon.noname-d con 0x7f4eac0a2bd0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac09aca0 con 0x7f4eac0a27f0 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac09a8b0 con 0x7f4eac09c8a0 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac090370 con 0x7f4eac0a2bd0 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb44c1700 10 monclient(hunting): authenticate will time out at 2557.640440s 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4eac09c8a0 0x7f4eac1ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a27f0 0x7f4eac1cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4eac09c8a0 0x7f4eac1ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55604/0 (socket says 172.21.15.47:55604) 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a27f0 0x7f4eac1cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36904/0 (socket says 172.21.15.47:36904) 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 1 -- 172.21.15.47:0/4102360964 learned_addr learned my addr 172.21.15.47:0/4102360964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:18.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): get_auth_request con 0x7f4eac09c8a0 auth_method 0 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): get_auth_request con 0x7f4eac0a27f0 auth_method 0 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a2bd0 0x7f4eac06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): get_auth_request con 0x7f4eac0a2bd0 auth_method 0 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb2a5e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient(hunting): handle_auth_done global_id 24833 payload 824 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a2bd0 msgr2=0x7f4eac06f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4eac0a2bd0 0x7f4eac06f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4eac09c8a0 msgr2=0x7f4eac1ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4eac09c8a0 0x7f4eac1ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient: _finish_hunting 0 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 monclient: found mon.noname-b 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:18.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.239+0000 7f4eb1a5c700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0094a0 con 0x7f4eac0a27f0 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb1a5c700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a27f0 0x7f4eac1cd510 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00e900 tx=0x7f4e9c00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 client.?.objecter ms_handle_connect 0x7f4eac0a27f0 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e9c013af0 con 0x7f4eac0a27f0 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: dump: 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.259 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c013c70 con 0x7f4eac0a27f0 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 5 monclient: authenticate success, global_id 24833 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 10 monclient: _renew_subs 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:18.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea8009550 con 0x7f4eac0a27f0 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e9c013de0 con 0x7f4eac0a27f0 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4ea2ffd700 10 monclient: dump: 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.261 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb0a5a700 4 set_mon_vals no callback set 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 10 client.24833.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 10 monclient: _renew_subs 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac1cda50 con 0x7f4eac0a27f0 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.240+0000 7f4eb44c1700 1 librados: init done 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb44c1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4e9c013680 con 0x7f4eac0a27f0 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 0x7f4eac090590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac068230 con 0x7f4eac071350 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 0x7f4eac090590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient: get_auth_request con 0x7f4eac071350 auth_method 0 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e9003a2f0 0x7f4e9003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f4e9c04a910 con 0x7f4eac0a27f0 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 10 client.24833.objecter ms_dispatch 0x7f4eac001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 3 client.24833.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:18.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 3 client.24833.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 20 client.24833.objecter dump_active .. 0 homeless 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 10 monclient(hunting): handle_auth_done global_id 24842 payload 824 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4eac0a3430 con 0x7f4eac071350 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb2a5e700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e9003a2f0 0x7f4e9003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb2a5e700 10 monclient: get_auth_request con 0x7f4e9003a2f0 auth_method 0 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 10 client.24833.objecter ms_handle_connect 0x7f4eac071350 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4ea2ffd700 10 client.24833.objecter resend_mon_ops 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.241+0000 7f4eb225d700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 0x7f4eac090590 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009c40 tx=0x7f4ea8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ea8020070 con 0x7f4eac071350 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 10 monclient: dump: 2024-04-20T02:31:18.264 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.265 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 10 client.24833.objecter ms_handle_connect 0x7f4e9003a2f0 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4eb2a5e700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e9003a2f0 0x7f4e9003c7b0 secure :-1 s=READY pgs=3345 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003d40 tx=0x7f4ea400d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4ea8011a00 con 0x7f4eac071350 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 msgr2=0x7f4eac090590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 0x7f4eac090590 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009c40 tx=0x7f4ea8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 msgr2=0x7f4eac090590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:18.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.242+0000 7f4ea2ffd700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac071350 0x7f4eac090590 secure :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009c40 tx=0x7f4ea8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb44c1700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:31:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 0x7f4eac070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb44c1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac06a400 con 0x7f4eac06fe00 2024-04-20T02:31:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 0x7f4eac070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient: get_auth_request con 0x7f4eac06fe00 auth_method 0 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 10 monclient(hunting): handle_auth_done global_id 24851 payload 824 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.251+0000 7f4eb225d700 1 -- 172.21.15.47:0/4102360964 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f4eac06bdc0 con 0x7f4eac06fe00 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4eb225d700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 0x7f4eac070210 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8023000 tx=0x7f4ea8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 client.24833.objecter ms_handle_connect 0x7f4eac06fe00 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 client.24833.objecter resend_mon_ops 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ea802b310 con 0x7f4eac06fe00 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 monclient: dump: 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.270 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f4ea802b890 con 0x7f4eac06fe00 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 msgr2=0x7f4eac070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 0x7f4eac070210 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8023000 tx=0x7f4ea8011000 comp rx=0 tx=0).stop 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 msgr2=0x7f4eac070210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4ea2ffd700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac06fe00 0x7f4eac070210 secure :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8023000 tx=0x7f4ea8011000 comp rx=0 tx=0).stop 2024-04-20T02:31:18.271 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:18.272 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:31:18.272 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:31:18.272 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4eb44c1700 10 librados: watch_flush enter 2024-04-20T02:31:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4eb44c1700 10 librados: watch_flush exit 2024-04-20T02:31:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4eb44c1700 20 client.24833.objecter shutdown clearing up homeless session... 2024-04-20T02:31:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.252+0000 7f4eb44c1700 10 client.24833.objecter successfully canceled tick 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e9003a2f0 msgr2=0x7f4e9003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4e9003a2f0 0x7f4e9003c7b0 secure :-1 s=READY pgs=3345 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003d40 tx=0x7f4ea400d040 comp rx=0 tx=0).stop 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 10 monclient: shutdown 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a27f0 msgr2=0x7f4eac1cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 --2- 172.21.15.47:0/4102360964 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4eac0a27f0 0x7f4eac1cd510 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00e900 tx=0x7f4e9c00ec10 comp rx=0 tx=0).stop 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb2a5e700 1 -- 172.21.15.47:0/4102360964 reap_dead start 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb2a5e700 1 -- 172.21.15.47:0/4102360964 reap_dead start 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 shutdown_connections 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 >> 172.21.15.47:0/4102360964 conn(0x7f4eac0a2fe0 msgr2=0x7f4eac1bf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 shutdown_connections 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 -- 172.21.15.47:0/4102360964 wait complete. 2024-04-20T02:31:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.253+0000 7f4eb44c1700 1 librados: shutdown 2024-04-20T02:31:18.275 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-20T02:31:18.275 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T02:31:18.275 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:31:18.294 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T02:31:18.294 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-04-20T02:31:18.294 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-04-20T02:31:18.295 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-04-20T02:31:18.295 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:31:18.308 INFO:tasks.ceph.mon.g:Stopped 2024-04-20T02:31:18.308 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-04-20T02:31:18.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:31:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.472+0000 7fa991b9f700 10 monclient: get_monmap_and_config 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.472+0000 7fa991b9f700 10 monclient: build_initial_monmap 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.472+0000 7fa991b9f700 10 monclient: monmap: 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:18.473553+0000 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:18.473553+0000 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:18.474 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 Processor -- start 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 -- start start 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient: init 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient: _add_conns ranks=[9,5,7,2,8,6,4,1,3,0,10] 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa98c137d00 0x7fa98c0a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): picked mon.noname-a con 0x7fa98c137d00 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 0x7fa98c1380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): picked mon.noname-h con 0x7fa98c0a0a20 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa98c0a4bf0 0x7fa98c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): picked mon.noname-i con 0x7fa98c0a4bf0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0a34f0 con 0x7fa98c137d00 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0a2f30 con 0x7fa98c0a0a20 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0a41f0 con 0x7fa98c0a4bf0 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:18.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa991b9f700 10 monclient(hunting): authenticate will time out at 2557.874843s 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa98b7fe700 1 -- >> v2:172.21.15.47:3300/0 conn(0x7fa98c137d00 msgr2=0x7fa98c0a3cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.473+0000 7fa98b7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa98c137d00 0x7fa98c0a3cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 0x7fa98c1380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa98c0a4bf0 0x7fa98c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 0x7fa98c1380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:40624/0 (socket says 172.21.15.47:40624) 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 -- 172.21.15.47:0/3702495222 learned_addr learned my addr 172.21.15.47:0/3702495222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): get_auth_request con 0x7fa98c0a0a20 auth_method 0 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): get_auth_request con 0x7fa98c0a4bf0 auth_method 0 2024-04-20T02:31:18.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient(hunting): handle_auth_done global_id 84416 payload 274 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 -- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa98c0a4bf0 msgr2=0x7fa98c0a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 --2- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa98c0a4bf0 0x7fa98c0a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 -- 172.21.15.47:0/3702495222 >> v2:172.21.15.47:3300/0 conn(0x7fa98c137d00 msgr2=0x7fa98c0a3cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 --2- 172.21.15.47:0/3702495222 >> v2:172.21.15.47:3300/0 conn(0x7fa98c137d00 0x7fa98c0a3cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 monclient: found mon.noname-h 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 -- 172.21.15.47:0/3702495222 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0751e0 con 0x7fa98c0a0a20 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98bfff700 1 --2- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa98c0a4bf0 0x7fa98c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98affd700 1 --2- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 0x7fa98c1380e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9800098f0 tx=0x7fa980009c00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=7886a022baddf6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.474+0000 7fa98a7fc700 1 -- 172.21.15.47:0/3702495222 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa980011aa0 con 0x7fa98c0a0a20 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: dump: 2024-04-20T02:31:18.489 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.490 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 1 -- 172.21.15.47:0/3702495222 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa980011c00 con 0x7fa98c0a0a20 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 5 monclient: authenticate success, global_id 84416 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 1 -- 172.21.15.47:0/3702495222 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa980011d70 con 0x7fa98c0a0a20 2024-04-20T02:31:18.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 4 set_mon_vals no callback set 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa98a7fc700 10 monclient: dump: 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.492 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 10 monclient: shutdown 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 -- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 msgr2=0x7fa98c1380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 --2- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 0x7fa98c1380e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa9800098f0 tx=0x7fa980009c00 comp rx=0 tx=0).stop 2024-04-20T02:31:18.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 -- 172.21.15.47:0/3702495222 shutdown_connections 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 --2- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa98c0a4bf0 0x7fa98c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 --2- 172.21.15.47:0/3702495222 >> v2:172.21.15.47:3300/0 conn(0x7fa98c137d00 0x7fa98c0a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 --2- 172.21.15.47:0/3702495222 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa98c0a0a20 0x7fa98c1380e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 -- 172.21.15.47:0/3702495222 >> 172.21.15.47:0/3702495222 conn(0x7fa98c0aa1f0 msgr2=0x7fa98c0aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 -- 172.21.15.47:0/3702495222 shutdown_connections 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.475+0000 7fa991b9f700 1 -- 172.21.15.47:0/3702495222 wait complete. 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.476+0000 7fa991b9f700 10 monclient: build_initial_monmap 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.476+0000 7fa991b9f700 10 monclient: monmap: 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:18.494 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.476+0000 7fa991b9f700 1 librados: starting msgr at 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.476+0000 7fa991b9f700 1 librados: starting objecter 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 1 Processor -- start 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 1 -- start start 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 1 librados: setting wanted keys 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 1 librados: calling monclient init 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 10 monclient: init 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.477+0000 7fa991b9f700 10 monclient: _add_conns ranks=[2,3,6,0,4,8,1,7,5] 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa98c0a0a20 0x7fa98c1cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): picked mon.noname-b con 0x7fa98c0a0a20 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa98c0a4bf0 0x7fa98c136da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): picked mon.noname-g con 0x7fa98c0a4bf0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:18.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 0x7fa98c1372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98b7fe700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa98c0a0a20 msgr2=0x7fa98c1cc490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): picked mon.noname-d con 0x7fa98c137d00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c124480 con 0x7fa98c0a0a20 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa98c0a0a20 0x7fa98c1cc490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c122d90 con 0x7fa98c0a4bf0 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c137820 con 0x7fa98c137d00 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient(hunting): authenticate will time out at 2557.879121s 2024-04-20T02:31:18.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98affd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa98c0a4bf0 msgr2=0x7fa98c136da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa98c0a4bf0 0x7fa98c136da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 0x7fa98c1372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 0x7fa98c1372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:45786/0 (socket says 172.21.15.47:45786) 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 -- 172.21.15.47:0/4033114114 learned_addr learned my addr 172.21.15.47:0/4033114114 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): get_auth_request con 0x7fa98c137d00 auth_method 0 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:18.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient(hunting): handle_auth_done global_id 55230 payload 824 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 -- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa98c0a4bf0 msgr2=0x7fa98c136da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa98c0a4bf0 0x7fa98c136da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 -- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa98c0a0a20 msgr2=0x7fa98c1cc490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa98c0a0a20 0x7fa98c1cc490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 monclient: found mon.noname-d 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 -- 172.21.15.47:0/4033114114 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009a70 con 0x7fa98c137d00 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa98bfff700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 0x7fa98c1372e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa984009f00 tx=0x7fa98400c3a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 client.?.objecter ms_handle_connect 0x7fa98c137d00 2024-04-20T02:31:18.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa984009d70 con 0x7fa98c137d00 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: dump: 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.500 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: _finish_auth 0 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98400f040 con 0x7fa98c137d00 2024-04-20T02:31:18.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 5 monclient: authenticate success, global_id 55230 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient: _renew_subs 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa980009550 con 0x7fa98c137d00 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa98400cc60 con 0x7fa98c137d00 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa973fff700 10 monclient: dump: 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:18.502 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:18.503 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.478+0000 7fa989ffb700 4 set_mon_vals no callback set 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.479+0000 7fa991b9f700 10 client.55230.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.479+0000 7fa991b9f700 10 monclient: _renew_subs 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.479+0000 7fa991b9f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.479+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c123ec0 con 0x7fa98c137d00 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.479+0000 7fa991b9f700 1 librados: init done 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.479+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa984014480 con 0x7fa98c137d00 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa97803a2f0 0x7fa97803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa991b9f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa991b9f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa991b9f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c1246c0 con 0x7fa98c137d00 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7fa98404b160 con 0x7fa98c137d00 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 10 client.55230.objecter ms_dispatch 0x7fa98c001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 3 client.55230.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 3 client.55230.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 20 client.55230.objecter dump_active .. 0 homeless 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa98b7fe700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa97803a2f0 0x7fa97803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa98b7fe700 10 monclient: get_auth_request con 0x7fa97803a2f0 auth_method 0 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa98b7fe700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa97803a2f0 0x7fa97803c7b0 secure :-1 s=READY pgs=3346 cs=0 l=1 rev1=1 crypto rx=0x7fa97c003a10 tx=0x7fa97c006d70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:18.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.480+0000 7fa973fff700 10 client.55230.objecter ms_handle_connect 0x7fa97803a2f0 2024-04-20T02:31:18.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.485+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa984018030 con 0x7fa98c137d00 2024-04-20T02:31:18.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.485+0000 7fa973fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:18.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.485+0000 7fa973fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.794+0000 7fa991b9f700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.795+0000 7fa991b9f700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.795+0000 7fa991b9f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.795+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa98c0008d0 con 0x7fa98c137d00 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.795+0000 7fa973fff700 1 -- 172.21.15.47:0/4033114114 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3175 (secure 0 0 0) 0x7fa984018030 con 0x7fa98c137d00 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.795+0000 7fa973fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.795+0000 7fa973fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:18.796 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":7,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:31:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 10 librados: watch_flush enter 2024-04-20T02:31:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 10 librados: watch_flush exit 2024-04-20T02:31:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 20 client.55230.objecter shutdown clearing up homeless session... 2024-04-20T02:31:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 10 client.55230.objecter successfully canceled tick 2024-04-20T02:31:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa97803a2f0 msgr2=0x7fa97803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa97803a2f0 0x7fa97803c7b0 secure :-1 s=READY pgs=3346 cs=0 l=1 rev1=1 crypto rx=0x7fa97c003a10 tx=0x7fa97c006d70 comp rx=0 tx=0).stop 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 10 monclient: shutdown 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 msgr2=0x7fa98c1372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 0x7fa98c1372e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa984009f00 tx=0x7fa98400c3a0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 shutdown_connections 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa97803a2f0 0x7fa97803c7b0 unknown :-1 s=CLOSED pgs=3346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fa98c137d00 0x7fa98c1372e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa98c0a0a20 0x7fa98c1cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 --2- 172.21.15.47:0/4033114114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa98c0a4bf0 0x7fa98c136da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 >> 172.21.15.47:0/4033114114 conn(0x7fa98c0aa1f0 msgr2=0x7fa98c0a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.797+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 shutdown_connections 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.798+0000 7fa991b9f700 1 -- 172.21.15.47:0/4033114114 wait complete. 2024-04-20T02:31:18.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:18.798+0000 7fa991b9f700 1 librados: shutdown 2024-04-20T02:31:21.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.957+0000 7f9466e70700 10 monclient: get_monmap_and_config 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.957+0000 7f9466e70700 10 monclient: build_initial_monmap 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.957+0000 7f9466e70700 10 monclient: monmap: 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:21.959099+0000 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:21.959099+0000 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:21.958 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:21.959 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.958+0000 7f9466e70700 1 Processor -- start 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.958+0000 7f9466e70700 1 -- start start 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.958+0000 7f9466e70700 10 monclient: init 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient: _add_conns ranks=[0,9,5,3,1,4,10,8,7,2,6] 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f94600a0ad0 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94600a0eb0 0x7f94600a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): picked mon.noname-a con 0x7f94600a0eb0 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 0x7f946013fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): picked mon.noname-h con 0x7f946013d9f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9464c0c700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f94600a0ad0 legacy=0x7f94600a2800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600751e0 con 0x7f94600a0ad0 2024-04-20T02:31:21.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946009ee40 con 0x7f94600a0eb0 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f945ffff700 1 -- >> v2:172.21.15.47:3300/0 conn(0x7f94600a0eb0 msgr2=0x7f94600a2f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94600a1330 con 0x7f946013d9f0 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f9466e70700 10 monclient(hunting): authenticate will time out at 2561.360545s 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f945ffff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94600a0eb0 0x7f94600a2f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 0x7f946013fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 0x7f946013fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:40626/0 (socket says 172.21.15.47:40626) 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 1 -- 172.21.15.47:0/3835874620 learned_addr learned my addr 172.21.15.47:0/3835874620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 10 monclient(hunting): get_auth_request con 0x7f946013d9f0 auth_method 0 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:21.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.959+0000 7f946540d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 10 monclient(hunting): handle_auth_done global_id 84425 payload 274 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 1 -- 172.21.15.47:0/3835874620 >> v2:172.21.15.47:3300/0 conn(0x7f94600a0eb0 msgr2=0x7f94600a2f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 1 --2- 172.21.15.47:0/3835874620 >> v2:172.21.15.47:3300/0 conn(0x7f94600a0eb0 0x7f94600a2f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 1 -- 172.21.15.47:0/3835874620 >> v1:172.21.15.47:6789/0 conn(0x7f94600a0ad0 legacy=0x7f94600a2800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 1 monclient: found mon.noname-h 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 1 -- 172.21.15.47:0/3835874620 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946009ddb0 con 0x7f946013d9f0 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f946540d700 1 --2- 172.21.15.47:0/3835874620 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 0x7f946013fde0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9458009e50 tx=0x7f945800b330 comp rx=0 tx=0).ready entity=mon.5 client_cookie=aeeb8f789a84e2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 1 -- 172.21.15.47:0/3835874620 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f945800e070 con 0x7f946013d9f0 2024-04-20T02:31:21.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: dump: 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:21.972 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 1 -- 172.21.15.47:0/3835874620 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458003e90 con 0x7f946013d9f0 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 1 -- 172.21.15.47:0/3835874620 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f945800bdf0 con 0x7f946013d9f0 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f9466e70700 5 monclient: authenticate success, global_id 84425 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f9466e70700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f9466e70700 4 set_mon_vals no callback set 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.960+0000 7f945f7fe700 10 monclient: dump: 2024-04-20T02:31:21.973 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:21.974 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 10 monclient: shutdown 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 -- 172.21.15.47:0/3835874620 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 msgr2=0x7f946013fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 --2- 172.21.15.47:0/3835874620 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 0x7f946013fde0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9458009e50 tx=0x7f945800b330 comp rx=0 tx=0).stop 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 -- 172.21.15.47:0/3835874620 shutdown_connections 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 --2- 172.21.15.47:0/3835874620 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f946013d9f0 0x7f946013fde0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 --2- 172.21.15.47:0/3835874620 >> v2:172.21.15.47:3300/0 conn(0x7f94600a0eb0 0x7f94600a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 -- 172.21.15.47:0/3835874620 >> 172.21.15.47:0/3835874620 conn(0x7f94600a7420 msgr2=0x7f94600a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 -- 172.21.15.47:0/3835874620 shutdown_connections 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.961+0000 7f9466e70700 1 -- 172.21.15.47:0/3835874620 wait complete. 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 10 monclient: build_initial_monmap 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 10 monclient: monmap: 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:21.975 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 1 librados: starting msgr at 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 1 librados: starting objecter 2024-04-20T02:31:21.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 1 Processor -- start 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 1 -- start start 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 1 librados: setting wanted keys 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 1 librados: calling monclient init 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.962+0000 7f9466e70700 10 monclient: init 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient: _add_conns ranks=[4,1,2,3,6,8,5,7,0] 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 0x7f94601c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): picked mon.noname-c con 0x7f94600a0ad0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94600a0eb0 0x7f94601380d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): picked mon.noname-f con 0x7f94600a0eb0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:21.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f946013d9f0 0x7f94601cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): picked mon.noname-b con 0x7f946013d9f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94601220c0 con 0x7f94600a0eb0 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600720c0 con 0x7f946013d9f0 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f946009e510 con 0x7f94600a0ad0 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f946540d700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f946013d9f0 msgr2=0x7f94601cc320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9466e70700 10 monclient(hunting): authenticate will time out at 2561.364415s 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f946540d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f946013d9f0 0x7f94601cc320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 0x7f94601c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 0x7f94601c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55614/0 (socket says 172.21.15.47:55614) 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 -- 172.21.15.47:0/2265950651 learned_addr learned my addr 172.21.15.47:0/2265950651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:21.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): get_auth_request con 0x7f94600a0ad0 auth_method 0 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945ffff700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94600a0eb0 0x7f94601380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945ffff700 10 monclient(hunting): get_auth_request con 0x7f94600a0eb0 auth_method 0 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient(hunting): handle_auth_done global_id 84460 payload 824 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 -- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f946013d9f0 msgr2=0x7f94601cc320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:21.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f946013d9f0 0x7f94601cc320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 -- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94600a0eb0 msgr2=0x7f94601380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94600a0eb0 0x7f94601380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient: _finish_hunting 0 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 monclient: found mon.noname-c 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 -- 172.21.15.47:0/2265950651 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458009b30 con 0x7f94600a0ad0 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f9464c0c700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 0x7f94601c8dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f945400b550 tx=0x7f945400b910 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945d7fa700 10 client.?.objecter ms_handle_connect 0x7f94600a0ad0 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.963+0000 7f945d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9454003c50 con 0x7f94600a0ad0 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: dump: 2024-04-20T02:31:21.980 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:21.981 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 5 monclient: authenticate success, global_id 84460 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454003df0 con 0x7f94600a0ad0 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f94540103a0 con 0x7f94600a0ad0 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 10 monclient: _renew_subs 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94601c97f0 con 0x7f94600a0ad0 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945effd700 4 set_mon_vals no callback set 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f945d7fa700 10 monclient: dump: 2024-04-20T02:31:21.982 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:21.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 10 client.84460.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 10 monclient: _renew_subs 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601231d0 con 0x7f94600a0ad0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.964+0000 7f9466e70700 1 librados: init done 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9454005070 con 0x7f94600a0ad0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f9466e70700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f9466e70700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f9466e70700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94600a04b0 con 0x7f94600a0ad0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f944403a2f0 0x7f944403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f94540498d0 con 0x7f94600a0ad0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 10 client.84460.objecter ms_dispatch 0x7f9460001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 3 client.84460.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:21.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 3 client.84460.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945d7fa700 20 client.84460.objecter dump_active .. 0 homeless 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945ffff700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f944403a2f0 0x7f944403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.965+0000 7f945ffff700 10 monclient: get_auth_request con 0x7f944403a2f0 auth_method 0 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.966+0000 7f945ffff700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f944403a2f0 0x7f944403c7b0 secure :-1 s=READY pgs=3347 cs=0 l=1 rev1=1 crypto rx=0x7f944c006fd0 tx=0x7f944c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.966+0000 7f945d7fa700 10 client.84460.objecter ms_handle_connect 0x7f944403a2f0 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.971+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f94600a04b0 con 0x7f94600a0ad0 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.971+0000 7f945d7fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:21.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:21.971+0000 7f945d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.276+0000 7f9466e70700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.276+0000 7f9466e70700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.276+0000 7f9466e70700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.276+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f94600a04b0 con 0x7f94600a0ad0 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.277+0000 7f945d7fa700 1 -- 172.21.15.47:0/2265950651 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f94600a04b0 con 0x7f94600a0ad0 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.277+0000 7f945d7fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.277+0000 7f945d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:22.277 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":10,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 10 librados: watch_flush enter 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 10 librados: watch_flush exit 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 20 client.84460.objecter shutdown clearing up homeless session... 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 10 client.84460.objecter successfully canceled tick 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f944403a2f0 msgr2=0x7f944403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f944403a2f0 0x7f944403c7b0 secure :-1 s=READY pgs=3347 cs=0 l=1 rev1=1 crypto rx=0x7f944c006fd0 tx=0x7f944c008040 comp rx=0 tx=0).stop 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 10 monclient: shutdown 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:22.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 msgr2=0x7f94601c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 0x7f94601c8dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f945400b550 tx=0x7f945400b910 comp rx=0 tx=0).stop 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 shutdown_connections 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f946013d9f0 0x7f94601cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94600a0ad0 0x7f94601c8dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f944403a2f0 0x7f944403c7b0 unknown :-1 s=CLOSED pgs=3347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 --2- 172.21.15.47:0/2265950651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94600a0eb0 0x7f94601380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 >> 172.21.15.47:0/2265950651 conn(0x7f94600a7420 msgr2=0x7f946013abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 shutdown_connections 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 -- 172.21.15.47:0/2265950651 wait complete. 2024-04-20T02:31:22.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:22.279+0000 7f9466e70700 1 librados: shutdown 2024-04-20T02:31:25.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.439+0000 7f3d2c5f1700 10 monclient: get_monmap_and_config 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.439+0000 7f3d2c5f1700 10 monclient: build_initial_monmap 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.439+0000 7f3d2c5f1700 10 monclient: monmap: 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:25.440784+0000 2024-04-20T02:31:25.440 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:25.440784+0000 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:25.441 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.440+0000 7f3d2c5f1700 1 Processor -- start 2024-04-20T02:31:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.440+0000 7f3d2c5f1700 1 -- start start 2024-04-20T02:31:25.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.440+0000 7f3d2c5f1700 10 monclient: init 2024-04-20T02:31:25.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.440+0000 7f3d2c5f1700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.440+0000 7f3d2c5f1700 10 monclient: _add_conns ranks=[9,1,4,10,5,7,8,6,0,3,2] 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3d24071660 0x7f3d24071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): picked mon.noname-a con 0x7f3d24071660 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3d24071f80 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 0x7f3d240a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): picked mon.noname-c con 0x7f3d24137de0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d240751e0 con 0x7f3d24071f80 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2406ca10 con 0x7f3d24071660 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2406e140 con 0x7f3d24137de0 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2a38d700 1 -- >> v2:172.21.15.47:3300/0 conn(0x7f3d24071660 msgr2=0x7f3d24071a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:25.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2c5f1700 10 monclient(hunting): authenticate will time out at 2564.842132s 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2a38d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3d24071660 0x7f3d24071a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 0x7f3d240a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 0x7f3d240a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44138/0 (socket says 172.21.15.47:44138) 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 -- 172.21.15.47:0/535845527 learned_addr learned my addr 172.21.15.47:0/535845527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): get_auth_request con 0x7f3d24137de0 auth_method 0 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:25.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_done global_id 84469 payload 274 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 -- 172.21.15.47:0/535845527 >> v2:172.21.15.47:3300/0 conn(0x7f3d24071660 msgr2=0x7f3d24071a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 --2- 172.21.15.47:0/535845527 >> v2:172.21.15.47:3300/0 conn(0x7f3d24071660 0x7f3d24071a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 -- 172.21.15.47:0/535845527 >> v1:172.21.15.179:6789/0 conn(0x7f3d24071f80 legacy=0x7f3d240a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient: _finish_hunting 0 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 monclient: found mon.noname-c 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 -- 172.21.15.47:0/535845527 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2406e420 con 0x7f3d24137de0 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2ab8e700 1 --2- 172.21.15.47:0/535845527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 0x7f3d240a3f90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009e50 tx=0x7f3d1c00b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=299ba0702a165e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2938b700 1 -- 172.21.15.47:0/535845527 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d1c00e070 con 0x7f3d24137de0 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.441+0000 7f3d2938b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:25.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: dump: 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:25.454 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: _finish_auth 0 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: _check_auth_tickets 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 1 -- 172.21.15.47:0/535845527 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c00be30 con 0x7f3d24137de0 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 5 monclient: authenticate success, global_id 84469 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 1 -- 172.21.15.47:0/535845527 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d1c012860 con 0x7f3d24137de0 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 4 set_mon_vals no callback set 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2938b700 10 monclient: dump: 2024-04-20T02:31:25.455 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:25.456 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 10 monclient: shutdown 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/535845527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 msgr2=0x7f3d240a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/535845527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 0x7f3d240a3f90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009e50 tx=0x7f3d1c00b330 comp rx=0 tx=0).stop 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/535845527 shutdown_connections 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/535845527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3d24137de0 0x7f3d240a3f90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/535845527 >> v2:172.21.15.47:3300/0 conn(0x7f3d24071660 0x7f3d24071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/535845527 >> 172.21.15.47:0/535845527 conn(0x7f3d240a84a0 msgr2=0x7f3d240a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/535845527 shutdown_connections 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.442+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/535845527 wait complete. 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 10 monclient: build_initial_monmap 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 10 monclient: monmap: 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:25.457 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:25.458 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 1 librados: starting msgr at 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 1 librados: starting objecter 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 1 Processor -- start 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 1 -- start start 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 1 librados: setting wanted keys 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 1 librados: calling monclient init 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.443+0000 7f3d2c5f1700 10 monclient: init 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient: _add_conns ranks=[7,2,5,0,8,3,1,6,4] 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3d24071660 0x7f3d241ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): picked mon.noname-i con 0x7f3d24071660 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d24071f80 0x7f3d241ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): picked mon.noname-b con 0x7f3d24071f80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:25.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3d24137de0 0x7f3d24120d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): picked mon.noname-h con 0x7f3d24137de0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24123a20 con 0x7f3d24071f80 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24124c80 con 0x7f3d24137de0 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241234f0 con 0x7f3d24071660 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2c5f1700 10 monclient(hunting): authenticate will time out at 2564.845406s 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d29b8c700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d24071f80 msgr2=0x7f3d241ccfe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d29b8c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d24071f80 0x7f3d241ccfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3d24137de0 0x7f3d24120d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:25.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3d24071660 0x7f3d241ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3d24137de0 0x7f3d24120d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:47888/0 (socket says 172.21.15.47:47888) 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 1 -- 172.21.15.47:0/1736855603 learned_addr learned my addr 172.21.15.47:0/1736855603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 10 monclient(hunting): get_auth_request con 0x7f3d24137de0 auth_method 0 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2ab8e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): get_auth_request con 0x7f3d24071660 auth_method 0 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.444+0000 7f3d2a38d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:25.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2ab8e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 10 monclient(hunting): handle_auth_done global_id 84427 payload 824 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 -- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3d24137de0 msgr2=0x7f3d24120d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3d24137de0 0x7f3d24120d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 -- 172.21.15.47:0/1736855603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d24071f80 msgr2=0x7f3d241ccfe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d24071f80 0x7f3d241ccfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 monclient: found mon.noname-i 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:31:25.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 -- 172.21.15.47:0/1736855603 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c009b30 con 0x7f3d24071660 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2a38d700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3d24071660 0x7f3d241ccaa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000b520 tx=0x7f3d2000b830 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 client.?.objecter ms_handle_connect 0x7f3d24071660 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d20003c50 con 0x7f3d24071660 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: dump: 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:25.463 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:25.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: _finish_auth 0 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 5 monclient: authenticate success, global_id 84427 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d20003df0 con 0x7f3d24071660 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d200103a0 con 0x7f3d24071660 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 10 monclient: _renew_subs 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2406d780 con 0x7f3d24071660 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d28b8a700 4 set_mon_vals no callback set 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d1affd700 10 monclient: dump: 2024-04-20T02:31:25.465 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:25.466 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 10 client.84427.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 10 monclient: _renew_subs 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241249a0 con 0x7f3d24071660 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.445+0000 7f3d2c5f1700 1 librados: init done 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d2c5f1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d2c5f1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d2c5f1700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3d20005070 con 0x7f3d24071660 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24134500 con 0x7f3d24071660 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d1affd700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3d0803a350 0x7f3d0803c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(604..604 src has 266..604) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f3d200497d0 con 0x7f3d24071660 2024-04-20T02:31:25.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d1affd700 10 client.84427.objecter ms_dispatch 0x7f3d24001550 osd_map(604..604 src has 266..604) v4 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d1affd700 3 client.84427.objecter handle_osd_map got epochs [604,604] > 0 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.446+0000 7f3d1affd700 3 client.84427.objecter handle_osd_map decoding full epoch 604 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.447+0000 7f3d1affd700 20 client.84427.objecter dump_active .. 0 homeless 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.447+0000 7f3d29b8c700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3d0803a350 0x7f3d0803c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.447+0000 7f3d29b8c700 10 monclient: get_auth_request con 0x7f3d0803a350 auth_method 0 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.447+0000 7f3d1affd700 10 client.84427.objecter ms_handle_connect 0x7f3d0803a350 2024-04-20T02:31:25.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.447+0000 7f3d29b8c700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3d0803a350 0x7f3d0803c810 secure :-1 s=READY pgs=3348 cs=0 l=1 rev1=1 crypto rx=0x7f3d2409efb0 tx=0x7f3d14007a80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.452+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f3d20017860 con 0x7f3d24071660 2024-04-20T02:31:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.452+0000 7f3d1affd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:25.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.452+0000 7f3d1affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:25.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.759+0000 7f3d2c5f1700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:31:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.759+0000 7f3d2c5f1700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.759+0000 7f3d2c5f1700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:31:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.759+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3d24124550 con 0x7f3d24071660 2024-04-20T02:31:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.760+0000 7f3d1affd700 1 -- 172.21.15.47:0/1736855603 <== mon.7 v2:172.21.15.179:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f3d20016c80 con 0x7f3d24071660 2024-04-20T02:31:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.760+0000 7f3d1affd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:25.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.760+0000 7f3d1affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:25.761 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 10 librados: watch_flush enter 2024-04-20T02:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 10 librados: watch_flush exit 2024-04-20T02:31:25.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 20 client.84427.objecter shutdown clearing up homeless session... 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 10 client.84427.objecter successfully canceled tick 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3d0803a350 msgr2=0x7f3d0803c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3d0803a350 0x7f3d0803c810 secure :-1 s=READY pgs=3348 cs=0 l=1 rev1=1 crypto rx=0x7f3d2409efb0 tx=0x7f3d14007a80 comp rx=0 tx=0).stop 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.762+0000 7f3d2c5f1700 10 monclient: shutdown 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3d24071660 msgr2=0x7f3d241ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3d24071660 0x7f3d241ccaa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000b520 tx=0x7f3d2000b830 comp rx=0 tx=0).stop 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 shutdown_connections 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3d0803a350 0x7f3d0803c810 unknown :-1 s=CLOSED pgs=3348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3d24071660 0x7f3d241ccaa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3d24137de0 0x7f3d24120d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 --2- 172.21.15.47:0/1736855603 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3d24071f80 0x7f3d241ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 >> 172.21.15.47:0/1736855603 conn(0x7f3d240a84a0 msgr2=0x7f3d24070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 shutdown_connections 2024-04-20T02:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 -- 172.21.15.47:0/1736855603 wait complete. 2024-04-20T02:31:25.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:25.763+0000 7f3d2c5f1700 1 librados: shutdown 2024-04-20T02:31:28.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:31:28.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.920+0000 7f7423400700 10 monclient: get_monmap_and_config 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.920+0000 7f7423400700 10 monclient: build_initial_monmap 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.920+0000 7f7423400700 10 monclient: monmap: 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:28.921938+0000 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:28.921938+0000 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.921+0000 7f7423400700 1 Processor -- start 2024-04-20T02:31:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.921+0000 7f7423400700 1 -- start start 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.921+0000 7f7423400700 10 monclient: init 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient: _add_conns ranks=[4,10,0,1,5,3,6,8,7,9,2] 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): picked mon.noname-c con 0x7f741c0a27c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): picked mon.noname-f con 0x7f741c0a30e0 addr v2:172.21.15.179:3300/0 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f741c0720e0 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c0751e0 con 0x7f741c0720e0 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c0a0180 con 0x7f741c0a30e0 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f741c09ff10 con 0x7f741c0a27c0 2024-04-20T02:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f7423400700 10 monclient(hunting): authenticate will time out at 2568.323244s 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742199d700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f741c0720e0 legacy=0x7f741c134b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:44148/0 (socket says 172.21.15.47:44148) 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 learned_addr learned my addr 172.21.15.47:0/1583649514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f741c0a27c0 auth_method 0 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f741c0a30e0 auth_method 0 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:28.922+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v1:172.21.15.47:6789/0 conn(0x7f741c0720e0 legacy=0x7f741c134b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f741c0a27c0 auth_method 0 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:29.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f741c0a30e0 auth_method 0 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:31:29.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:29.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:29.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.123+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v1:172.21.15.47:6789/0 conn(0x7f741c0720e0 legacy=0x7f741c134b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f741c0a27c0 auth_method 0 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f741c0a30e0 auth_method 0 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:29.524+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v1:172.21.15.47:6789/0 conn(0x7f741c0720e0 legacy=0x7f741c134b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f741c0a27c0 auth_method 0 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:30.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f741c0a30e0 auth_method 0 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:30.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:30.325+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:31.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): tick 2024-04-20T02:31:31.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 msgr2=0x7f741c0a2ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f741c0a27c0 0x7f741c0a2ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 msgr2=0x7f741c134620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.179:3300/0 conn(0x7f741c0a30e0 0x7f741c134620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> v1:172.21.15.47:6789/0 conn(0x7f741c0720e0 legacy=0x7f741c134b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient: _add_conns ranks=[1,3,7,8,5,9,2,6] 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7400003880 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 0x7f7400009900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-g con 0x7f7400007470 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-i con 0x7f7400009fb0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c09f080 con 0x7f7400003880 2024-04-20T02:31:31.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0a0180 con 0x7f7400007470 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f741c09ff10 con 0x7f7400009fb0 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f7412ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 msgr2=0x7f7400009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 0x7f7400009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 10 monclient(hunting): get_auth_request con 0x7f7400009fb0 auth_method 0 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.922+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:31.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:31.923+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.123+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 msgr2=0x7f7400009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.123+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 0x7f7400009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 10 monclient(hunting): get_auth_request con 0x7f7400009fb0 auth_method 0 2024-04-20T02:31:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.124+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.524+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 msgr2=0x7f7400009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:32.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.524+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 0x7f7400009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:32.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 10 monclient(hunting): get_auth_request con 0x7f7400009fb0 auth_method 0 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:32.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:32.525+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.325+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 msgr2=0x7f7400009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:33.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.325+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 0x7f7400009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.326+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.326+0000 7f742199d700 10 monclient(hunting): get_auth_request con 0x7f7400009fb0 auth_method 0 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.326+0000 7f742199d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.326+0000 7f742199d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.326+0000 7f742199d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.327+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.327+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.327+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:33.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:33.327+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:34.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): tick 2024-04-20T02:31:34.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:31:34.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:31:34.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 msgr2=0x7f740000c470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7400009fb0 0x7f740000c470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 msgr2=0x7f7400009900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7400007470 0x7f7400009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> v1:172.21.15.179:6789/0 conn(0x7f7400003880 legacy=0x7f7400005c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient: _add_conns ranks=[5,8,9,6,2] 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 reap_dead start 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-h con 0x7f740000ce70 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-e con 0x7f740000fe20 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 0x7f7400014b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-a con 0x7f7400012750 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:34.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c0a0180 con 0x7f7400012750 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f741c09ff10 con 0x7f740000ce70 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7400015290 con 0x7f740000fe20 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f7412ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 msgr2=0x7f7400014b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 0x7f7400014b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.922+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f740000fe20 auth_method 0 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:34.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:34.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f740000ce70 auth_method 0 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:34.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:34.923+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.123+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 msgr2=0x7f7400014b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.123+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 0x7f7400014b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f740000fe20 auth_method 0 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f740000ce70 auth_method 0 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.124+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:35.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.524+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 msgr2=0x7f7400014b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:35.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.524+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 0x7f7400014b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f740000fe20 auth_method 0 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:35.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f740000ce70 auth_method 0 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.525+0000 7f742119c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.526+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.526+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.526+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:35.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:35.526+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.325+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 msgr2=0x7f7400014b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.325+0000 7f742199d700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 0x7f7400014b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f740000fe20 auth_method 0 2024-04-20T02:31:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.326+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 10 monclient(hunting): get_auth_request con 0x7f740000ce70 auth_method 0 2024-04-20T02:31:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:31:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:31:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:31:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:31:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:36.327+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): tick 2024-04-20T02:31:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:31:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:31:37.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 msgr2=0x7f7400012210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f740000fe20 0x7f7400012210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 msgr2=0x7f740000d2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f740000ce70 0x7f740000d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 msgr2=0x7f7400014b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> v2:172.21.15.47:3300/0 conn(0x7f7400012750 0x7f7400014b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient: _add_conns ranks=[2,6] 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7400006560 0x7f7400006940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:37.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-b con 0x7f7400006560 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f740000c4f0 0x7f7400009900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): picked mon.noname-d con 0x7f740000c4f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0a0180 con 0x7f7400006560 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f741c09ff10 con 0x7f740000c4f0 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f7412ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742119c700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7400006560 msgr2=0x7f7400006940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742119c700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7400006560 0x7f7400006940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f740000c4f0 0x7f7400009900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f740000c4f0 auth_method 0 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:37.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient(hunting): handle_auth_done global_id 55239 payload 274 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7400006560 msgr2=0x7f7400006940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7400006560 0x7f7400006940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient: _finish_hunting 0 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 1 monclient: found mon.noname-d 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.923+0000 7f742099b700 1 -- 172.21.15.47:0/1583649514 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0751e0 con 0x7f740000c4f0 2024-04-20T02:31:37.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f742099b700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f740000c4f0 0x7f7400009900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f740c009100 tx=0x7f740c003980 comp rx=0 tx=0).ready entity=mon.6 client_cookie=43e4f86dcb512464 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 1 -- 172.21.15.47:0/1583649514 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f740c01e070 con 0x7f740000c4f0 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: dump: 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:37.930 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: _finish_auth 0 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 1 -- 172.21.15.47:0/1583649514 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00c040 con 0x7f740000c4f0 2024-04-20T02:31:37.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7423400700 5 monclient: authenticate success, global_id 55239 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7423400700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7423400700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 1 -- 172.21.15.47:0/1583649514 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f740c0173e0 con 0x7f740000c4f0 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7423400700 4 set_mon_vals no callback set 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.924+0000 7f7413fff700 10 monclient: dump: 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.932 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:37.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 10 monclient: shutdown 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 1 -- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f740000c4f0 msgr2=0x7f7400009900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 1 --2- 172.21.15.47:0/1583649514 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f740000c4f0 0x7f7400009900 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f740c009100 tx=0x7f740c003980 comp rx=0 tx=0).stop 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f742199d700 1 -- 172.21.15.47:0/1583649514 reap_dead start 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 1 -- 172.21.15.47:0/1583649514 shutdown_connections 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 1 -- 172.21.15.47:0/1583649514 >> 172.21.15.47:0/1583649514 conn(0x7f741c06cbf0 msgr2=0x7f741c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 1 -- 172.21.15.47:0/1583649514 shutdown_connections 2024-04-20T02:31:37.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.925+0000 7f7423400700 1 -- 172.21.15.47:0/1583649514 wait complete. 2024-04-20T02:31:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.926+0000 7f7423400700 10 monclient: build_initial_monmap 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.926+0000 7f7423400700 10 monclient: monmap: 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 1 librados: starting msgr at 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 1 librados: starting objecter 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 1 Processor -- start 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 1 -- start start 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 1 librados: setting wanted keys 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 1 librados: calling monclient init 2024-04-20T02:31:37.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.927+0000 7f7423400700 10 monclient: init 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient: _add_conns ranks=[2,5,6,1,4,8,0,3,7] 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f741c09c810 0x7f741c1b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): picked mon.noname-b con 0x7f741c09c810 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f741c1b7b10 0x7f741c1c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): picked mon.noname-h con 0x7f741c1b7b10 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 0x7f741c1c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): picked mon.noname-d con 0x7f741c1c8570 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c1a3190 con 0x7f741c09c810 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:37.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f741c1a4cb0 con 0x7f741c1b7b10 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f741c09d500 con 0x7f741c1c8570 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f7423400700 10 monclient(hunting): authenticate will time out at 2577.329556s 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742119c700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f741c09c810 msgr2=0x7f741c1b75d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742119c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f741c09c810 0x7f741c1b75d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 0x7f741c1c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 0x7f741c1c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57658/0 (socket says 172.21.15.47:57658) 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 1 -- 172.21.15.47:0/1533141335 learned_addr learned my addr 172.21.15.47:0/1533141335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): get_auth_request con 0x7f741c1c8570 auth_method 0 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:37.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742099b700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f741c1b7b10 0x7f741c1c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742199d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742099b700 10 monclient(hunting): get_auth_request con 0x7f741c1b7b10 auth_method 0 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742099b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742099b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.928+0000 7f742099b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742099b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742099b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742099b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 10 monclient(hunting): handle_auth_done global_id 55248 payload 824 2024-04-20T02:31:37.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 -- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f741c1b7b10 msgr2=0x7f741c1c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f741c1b7b10 0x7f741c1c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 -- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f741c09c810 msgr2=0x7f741c1b75d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f741c09c810 0x7f741c1b75d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 10 monclient: _finish_hunting 0 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 monclient: found mon.noname-d 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 -- 172.21.15.47:0/1533141335 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c00e040 con 0x7f741c1c8570 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f742199d700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 0x7f741c1c89d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74000037b0 tx=0x7f741400f9d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 client.?.objecter ms_handle_connect 0x7f741c1c8570 2024-04-20T02:31:37.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f741400f030 con 0x7f741c1c8570 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: dump: 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:37.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:37.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741400f1b0 con 0x7f741c1c8570 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f741400d420 con 0x7f741c1c8570 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7411ffb700 10 monclient: dump: 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:37.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f74137fe700 4 set_mon_vals no callback set 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7423400700 5 monclient: authenticate success, global_id 55248 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7423400700 10 monclient: _renew_subs 2024-04-20T02:31:37.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7423400700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.929+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c09cbf0 con 0x7f741c1c8570 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.930+0000 7f7423400700 10 client.55248.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.930+0000 7f7423400700 10 monclient: _renew_subs 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.930+0000 7f7423400700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.930+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c1a33d0 con 0x7f741c1c8570 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.930+0000 7f7423400700 1 librados: init done 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f741400c650 con 0x7f741c1c8570 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f740003bc60 0x7f740003e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(605..605 src has 266..605) v4 ==== 6641+0+0 (secure 0 0 0) 0x7f741404a600 con 0x7f741c1c8570 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 10 client.55248.objecter ms_dispatch 0x7f741c001550 osd_map(605..605 src has 266..605) v4 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 3 client.55248.objecter handle_osd_map got epochs [605,605] > 0 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 3 client.55248.objecter handle_osd_map decoding full epoch 605 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7423400700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:37.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7423400700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7423400700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7411ffb700 20 client.55248.objecter dump_active .. 0 homeless 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c0a4700 con 0x7f741c1c8570 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.931+0000 7f742119c700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f740003bc60 0x7f740003e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.932+0000 7f742119c700 10 monclient: get_auth_request con 0x7f740003bc60 auth_method 0 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.933+0000 7f7411ffb700 10 client.55248.objecter ms_handle_connect 0x7f740003bc60 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.933+0000 7f742119c700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f740003bc60 0x7f740003e120 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7f7418000c00 tx=0x7f7418006c60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.937+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7414017bd0 con 0x7f741c1c8570 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.937+0000 7f7411ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:37.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:37.937+0000 7f7411ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.259+0000 7f7423400700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.259+0000 7f7423400700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.259+0000 7f7423400700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.259+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f741c066610 con 0x7f741c1c8570 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.260+0000 7f7411ffb700 1 -- 172.21.15.47:0/1533141335 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3158 (secure 0 0 0) 0x7f74140177c0 con 0x7f741c1c8570 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.260+0000 7f7411ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.260+0000 7f7411ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:38.260 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":104,"quorum":[1,4,5,6,7,8],"quorum_names":["f","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:31:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 10 librados: watch_flush enter 2024-04-20T02:31:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 10 librados: watch_flush exit 2024-04-20T02:31:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 20 client.55248.objecter shutdown clearing up homeless session... 2024-04-20T02:31:38.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 10 client.55248.objecter successfully canceled tick 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f740003bc60 msgr2=0x7f740003e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f740003bc60 0x7f740003e120 secure :-1 s=READY pgs=3349 cs=0 l=1 rev1=1 crypto rx=0x7f7418000c00 tx=0x7f7418006c60 comp rx=0 tx=0).stop 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 10 monclient: shutdown 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 msgr2=0x7f741c1c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 0x7f741c1c89d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74000037b0 tx=0x7f741400f9d0 comp rx=0 tx=0).stop 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 shutdown_connections 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f740003bc60 0x7f740003e120 unknown :-1 s=CLOSED pgs=3349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f741c1c8570 0x7f741c1c89d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:38.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f741c09c810 0x7f741c1b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 --2- 172.21.15.47:0/1533141335 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f741c1b7b10 0x7f741c1c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.262+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 >> 172.21.15.47:0/1533141335 conn(0x7f741c06cbf0 msgr2=0x7f741c1bb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.263+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 shutdown_connections 2024-04-20T02:31:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.263+0000 7f7423400700 1 -- 172.21.15.47:0/1533141335 wait complete. 2024-04-20T02:31:38.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.263+0000 7f7423400700 1 librados: shutdown 2024-04-20T02:31:38.278 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-04-20T02:31:38.278 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-04-20T02:31:38.278 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:31:38.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.420+0000 7f91d1676700 10 monclient: get_monmap_and_config 2024-04-20T02:31:38.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.420+0000 7f91d1676700 10 monclient: build_initial_monmap 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.420+0000 7f91d1676700 10 monclient: monmap: 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:38.421862+0000 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:38.421862+0000 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:38.421 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:38.422 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.421+0000 7f91d1676700 1 Processor -- start 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.421+0000 7f91d1676700 1 -- start start 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.421+0000 7f91d1676700 10 monclient: init 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient: _add_conns ranks=[0,2,3,4,1,10,5,7,9,8,6] 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f91cc071610 addr v1:172.21.15.47:6789/0 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 0x7f91cc139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): picked mon.noname-b con 0x7f91cc137930 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 0x7f91cc13c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:38.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): picked mon.noname-g con 0x7f91cc13a260 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc0751e0 con 0x7f91cc071610 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0a0880 con 0x7f91cc137930 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc09e8f0 con 0x7f91cc13a260 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91d1676700 10 monclient(hunting): authenticate will time out at 2577.823153s 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91caffd700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f91cc071610 legacy=0x7f91cc0719f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91ca7fc700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 msgr2=0x7f91cc139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 0x7f91cc139d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91cb7fe700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 msgr2=0x7f91cc13c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:38.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.422+0000 7f91cb7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 0x7f91cc13c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:38.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.623+0000 7f91ca7fc700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 msgr2=0x7f91cc139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:38.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.623+0000 7f91caffd700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f91cc071610 legacy=0x7f91cc0719f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:38.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.623+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 0x7f91cc139d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:38.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.623+0000 7f91cb7fe700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 msgr2=0x7f91cc13c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:38.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:38.623+0000 7f91cb7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 0x7f91cc13c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:31:39.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.024+0000 7f91caffd700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f91cc071610 legacy=0x7f91cc0719f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:39.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.024+0000 7f91ca7fc700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 msgr2=0x7f91cc139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:39.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.024+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 0x7f91cc139d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:39.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.024+0000 7f91cb7fe700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 msgr2=0x7f91cc13c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:39.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.024+0000 7f91cb7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 0x7f91cc13c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:31:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.825+0000 7f91ca7fc700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 msgr2=0x7f91cc139d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.825+0000 7f91caffd700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f91cc071610 legacy=0x7f91cc0719f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.47:6789/0 2024-04-20T02:31:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.825+0000 7f91cb7fe700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 msgr2=0x7f91cc13c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.825+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 0x7f91cc139d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:39.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:39.825+0000 7f91cb7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 0x7f91cc13c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:31:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): tick 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 monclient(hunting): continuing hunt 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 msgr2=0x7f91cc13c650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f91cc13a260 0x7f91cc13c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 msgr2=0x7f91cc139d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f91cc137930 0x7f91cc139d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 -- >> v1:172.21.15.47:6789/0 conn(0x7f91cc071610 legacy=0x7f91cc0719f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient: _add_conns ranks=[1,8,7,4,10,9,6,5] 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f91b80038b0 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 0x7f91b8009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): picked mon.noname-e con 0x7f91b80074e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b8009fd0 0x7f91b800c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): picked mon.noname-i con 0x7f91b8009fd0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc09ed00 con 0x7f91b80038b0 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0a0880 con 0x7f91b8009fd0 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc09e8f0 con 0x7f91b80074e0 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91c8ff9700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 0x7f91b8009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 0x7f91b8009970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55760/0 (socket says 172.21.15.47:55760) 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 -- 172.21.15.47:0/816739804 learned_addr learned my addr 172.21.15.47:0/816739804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:41.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): get_auth_request con 0x7f91b80074e0 auth_method 0 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91caffd700 1 --1- 172.21.15.47:0/816739804 >> v1:172.21.15.179:6789/0 conn(0x7f91b80038b0 0x7f91b8005d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:54068/0 (socket says 172.21.15.47:54068) 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 1 --2- 172.21.15.47:0/816739804 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b8009fd0 0x7f91b800c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): get_auth_request con 0x7f91b8009fd0 auth_method 0 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_done global_id 36026 payload 274 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 -- 172.21.15.47:0/816739804 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b8009fd0 msgr2=0x7f91b800c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 --2- 172.21.15.47:0/816739804 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f91b8009fd0 0x7f91b800c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 -- 172.21.15.47:0/816739804 >> v1:172.21.15.179:6789/0 conn(0x7f91b80038b0 legacy=0x7f91b8005d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 1 monclient: found mon.noname-e 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.422+0000 7f91ca7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.423+0000 7f91ca7fc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.423+0000 7f91ca7fc700 1 -- 172.21.15.47:0/816739804 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0751e0 con 0x7f91b80074e0 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.423+0000 7f91cb7fe700 1 -- 172.21.15.47:0/816739804 reap_dead start 2024-04-20T02:31:41.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.423+0000 7f91ca7fc700 1 --2- 172.21.15.47:0/816739804 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 0x7f91b8009970 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f91c000bfa0 tx=0x7f91c000d330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1d035ec61c79584b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 1 -- 172.21.15.47:0/816739804 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91c0010070 con 0x7f91b80074e0 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: dump: 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.430 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91d1676700 5 monclient: authenticate success, global_id 36026 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91d1676700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 1 -- 172.21.15.47:0/816739804 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c000bad0 con 0x7f91b80074e0 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91d1676700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91d1676700 4 set_mon_vals no callback set 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 1 -- 172.21.15.47:0/816739804 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91c000de70 con 0x7f91b80074e0 2024-04-20T02:31:41.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.426+0000 7f91c9ffb700 10 monclient: dump: 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.432 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.427+0000 7f91d1676700 10 monclient: shutdown 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.427+0000 7f91d1676700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.427+0000 7f91d1676700 1 -- 172.21.15.47:0/816739804 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 msgr2=0x7f91b8009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.427+0000 7f91d1676700 1 --2- 172.21.15.47:0/816739804 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 0x7f91b8009970 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f91c000bfa0 tx=0x7f91c000d330 comp rx=0 tx=0).stop 2024-04-20T02:31:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.433+0000 7f91d1676700 1 -- 172.21.15.47:0/816739804 shutdown_connections 2024-04-20T02:31:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.433+0000 7f91d1676700 1 --2- 172.21.15.47:0/816739804 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f91b80074e0 0x7f91b8009970 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.433+0000 7f91d1676700 1 -- 172.21.15.47:0/816739804 >> 172.21.15.47:0/816739804 conn(0x7f91cc0a7570 msgr2=0x7f91cc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.433+0000 7f91d1676700 1 -- 172.21.15.47:0/816739804 shutdown_connections 2024-04-20T02:31:41.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.433+0000 7f91d1676700 1 -- 172.21.15.47:0/816739804 wait complete. 2024-04-20T02:31:41.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.434+0000 7f91d1676700 10 monclient: build_initial_monmap 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.434+0000 7f91d1676700 10 monclient: monmap: 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:41.438 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.435+0000 7f91d1676700 1 librados: starting msgr at 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.435+0000 7f91d1676700 1 librados: starting objecter 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.435+0000 7f91d1676700 1 Processor -- start 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 -- start start 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 librados: setting wanted keys 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 librados: calling monclient init 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient: init 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient: _add_conns ranks=[0,1,5,6,7,2,3,8,4] 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b80074e0 0x7f91cc1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): picked mon.noname-a con 0x7f91b80074e0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:41.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc1cfdd0 0x7f91cc0a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): picked mon.noname-f con 0x7f91cc1cfdd0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91cc0a1d80 0x7f91cc0a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): picked mon.noname-h con 0x7f91cc0a1d80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc135850 con 0x7f91b80074e0 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1d01f0 con 0x7f91cc1cfdd0 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc135da0 con 0x7f91cc0a1d80 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91d1676700 10 monclient(hunting): authenticate will time out at 2580.837957s 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.436+0000 7f91caffd700 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b80074e0 msgr2=0x7f91cc1cf890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:41.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91caffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b80074e0 0x7f91cc1cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc1cfdd0 0x7f91cc0a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc1cfdd0 0x7f91cc0a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:39362/0 (socket says 172.21.15.47:39362) 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 1 -- 172.21.15.47:0/2517825320 learned_addr learned my addr 172.21.15.47:0/2517825320 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): get_auth_request con 0x7f91cc1cfdd0 auth_method 0 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91cc0a1d80 0x7f91cc0a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): get_auth_request con 0x7f91cc0a1d80 auth_method 0 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient(hunting): handle_auth_done global_id 84452 payload 824 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc1cfdd0 msgr2=0x7f91cc0a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc1cfdd0 0x7f91cc0a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b80074e0 msgr2=0x7f91cc1cf890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91b80074e0 0x7f91cc1cf890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:41.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 monclient: found mon.noname-h 2024-04-20T02:31:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:31:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.437+0000 7f91cb7fe700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c000bc50 con 0x7f91cc0a1d80 2024-04-20T02:31:41.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91ca7fc700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91cc1cfdd0 0x7f91cc0a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:41.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91cb7fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91cc0a1d80 0x7f91cc0a2200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0686d0 tx=0x7f91c400d5c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 client.?.objecter ms_handle_connect 0x7f91cc0a1d80 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91c4011440 con 0x7f91cc0a1d80 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: dump: 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.446 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:41.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4011600 con 0x7f91cc0a1d80 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91c4011be0 con 0x7f91cc0a1d80 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91b37fe700 10 monclient: dump: 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.448 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91c97fa700 4 set_mon_vals no callback set 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91d1676700 5 monclient: authenticate success, global_id 84452 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91d1676700 10 monclient: _renew_subs 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91d1676700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:31:41.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.438+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc136080 con 0x7f91cc0a1d80 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.439+0000 7f91d1676700 10 client.84452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.439+0000 7f91d1676700 10 monclient: _renew_subs 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.439+0000 7f91d1676700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.439+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc0723f0 con 0x7f91cc0a1d80 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.439+0000 7f91d1676700 1 librados: init done 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91d1676700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91d1676700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 0x7f91cc12d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc068310 con 0x7f91cc06aac0 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 0x7f91cc12d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient: get_auth_request con 0x7f91cc06aac0 auth_method 0 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.440+0000 7f91caffd700 10 monclient(hunting): handle_auth_done global_id 84478 payload 824 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91caffd700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91cc0a02c0 con 0x7f91cc06aac0 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91caffd700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 0x7f91cc12d660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91bc003a10 tx=0x7f91bc00bc80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91b37fe700 10 client.84452.objecter ms_handle_connect 0x7f91cc06aac0 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91b37fe700 10 client.84452.objecter resend_mon_ops 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91bc00cd20 con 0x7f91cc06aac0 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91b37fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.441+0000 7f91b37fe700 10 monclient: dump: 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.452 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f91bc00ce80 con 0x7f91cc06aac0 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 msgr2=0x7f91cc12d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 0x7f91cc12d660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91bc003a10 tx=0x7f91bc00bc80 comp rx=0 tx=0).stop 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 msgr2=0x7f91cc12d660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc06aac0 0x7f91cc12d660 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91bc003a10 tx=0x7f91bc00bc80 comp rx=0 tx=0).stop 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f91c4011d40 con 0x7f91cc0a1d80 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.442+0000 7f91b37fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91b803b650 0x7f91b803db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.443+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(610..610 src has 266..610) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f91c400c6b0 con 0x7f91cc0a1d80 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.443+0000 7f91b37fe700 10 client.84452.objecter ms_dispatch 0x7f91cc001550 osd_map(610..610 src has 266..610) v4 2024-04-20T02:31:41.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.443+0000 7f91b37fe700 3 client.84452.objecter handle_osd_map got epochs [610,610] > 0 2024-04-20T02:31:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.443+0000 7f91b37fe700 3 client.84452.objecter handle_osd_map decoding full epoch 610 2024-04-20T02:31:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.443+0000 7f91b37fe700 20 client.84452.objecter dump_active .. 0 homeless 2024-04-20T02:31:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.443+0000 7f91caffd700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91b803b650 0x7f91b803db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.444+0000 7f91caffd700 10 monclient: get_auth_request con 0x7f91b803b650 auth_method 0 2024-04-20T02:31:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.444+0000 7f91b37fe700 10 client.84452.objecter ms_handle_connect 0x7f91b803b650 2024-04-20T02:31:41.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.444+0000 7f91caffd700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91b803b650 0x7f91b803db10 secure :-1 s=READY pgs=3350 cs=0 l=1 rev1=1 crypto rx=0x7f91bc003a10 tx=0x7f91bc005000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91d1676700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91d1676700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 0x7f91cc12dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91d1676700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc06a400 con 0x7f91cc12dba0 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 0x7f91cc12dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient: get_auth_request con 0x7f91cc12dba0 auth_method 0 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 10 monclient(hunting): handle_auth_done global_id 84487 payload 824 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 1 -- 172.21.15.47:0/2517825320 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91cc0a4480 con 0x7f91cc12dba0 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91b37fe700 10 client.84452.objecter ms_handle_connect 0x7f91cc12dba0 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91ca7fc700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 0x7f91cc12dfb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91c0000ea0 tx=0x7f91c0010000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91b37fe700 10 client.84452.objecter resend_mon_ops 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91c0023070 con 0x7f91cc12dba0 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.452+0000 7f91b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 10 monclient: dump: 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.457 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.458 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9321 (secure 0 0 0) 0x7f91c001b040 con 0x7f91cc12dba0 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 msgr2=0x7f91cc12dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 0x7f91cc12dfb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91c0000ea0 tx=0x7f91c0010000 comp rx=0 tx=0).stop 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 msgr2=0x7f91cc12dfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91b37fe700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f91cc12dba0 0x7f91cc12dfb0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f91c0000ea0 tx=0x7f91c0010000 comp rx=0 tx=0).stop 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:41.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 104, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:31:41.460 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.461 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:41.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.464 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:41.466 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.467 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:41.474 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:41.475 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.477 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:41.479 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.480 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:41.481 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 10 librados: watch_flush enter 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 10 librados: watch_flush exit 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 20 client.84452.objecter shutdown clearing up homeless session... 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 10 client.84452.objecter successfully canceled tick 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91b803b650 msgr2=0x7f91b803db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f91b803b650 0x7f91b803db10 secure :-1 s=READY pgs=3350 cs=0 l=1 rev1=1 crypto rx=0x7f91bc003a10 tx=0x7f91bc005000 comp rx=0 tx=0).stop 2024-04-20T02:31:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 10 monclient: shutdown 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91cc0a1d80 msgr2=0x7f91cc0a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 --2- 172.21.15.47:0/2517825320 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f91cc0a1d80 0x7f91cc0a2200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0686d0 tx=0x7f91c400d5c0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91cb7fe700 1 -- 172.21.15.47:0/2517825320 reap_dead start 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91cb7fe700 1 -- 172.21.15.47:0/2517825320 reap_dead start 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 shutdown_connections 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 >> 172.21.15.47:0/2517825320 conn(0x7f91cc0a7570 msgr2=0x7f91cc06fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.453+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 shutdown_connections 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.454+0000 7f91d1676700 1 -- 172.21.15.47:0/2517825320 wait complete. 2024-04-20T02:31:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.454+0000 7f91d1676700 1 librados: shutdown 2024-04-20T02:31:41.484 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:31:41.485 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-04-20T02:31:41.485 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.626+0000 7f58126a4700 10 monclient: get_monmap_and_config 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.626+0000 7f58126a4700 10 monclient: build_initial_monmap 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.626+0000 7f58126a4700 10 monclient: monmap: 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:41.627452+0000 2024-04-20T02:31:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:41.627452+0000 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:41.627 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:41.628 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:41.628 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 Processor -- start 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 -- start start 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient: init 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient: _add_conns ranks=[3,6,1,2,10,7,9,0,5,8,4] 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c0a1510 0x7f580c0a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): picked mon.noname-g con 0x7f580c0a1510 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 0x7f580c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): picked mon.noname-d con 0x7f580c09f9a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f580c09fe00 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c0751e0 con 0x7f580c09fe00 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0720a0 con 0x7f580c0a1510 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f580c070640 con 0x7f580c09f9a0 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f58126a4700 10 monclient(hunting): authenticate will time out at 2581.028695s 2024-04-20T02:31:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 0x7f580c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580bfff700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c0a1510 msgr2=0x7f580c0a18f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c0a1510 0x7f580c0a18f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 0x7f580c0a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57670/0 (socket says 172.21.15.47:57670) 2024-04-20T02:31:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 1 -- 172.21.15.47:0/766576081 learned_addr learned my addr 172.21.15.47:0/766576081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 10 monclient(hunting): get_auth_request con 0x7f580c09f9a0 auth_method 0 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.627+0000 7f580b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 10 monclient(hunting): handle_auth_done global_id 55257 payload 274 2024-04-20T02:31:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 1 -- 172.21.15.47:0/766576081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c0a1510 msgr2=0x7f580c0a18f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 1 --2- 172.21.15.47:0/766576081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c0a1510 0x7f580c0a18f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 1 -- 172.21.15.47:0/766576081 >> v1:172.21.15.179:6789/0 conn(0x7f580c09fe00 legacy=0x7f580c0a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:31:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 1 monclient: found mon.noname-d 2024-04-20T02:31:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:31:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 1 -- 172.21.15.47:0/766576081 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c071d30 con 0x7f580c09f9a0 2024-04-20T02:31:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580b7fe700 1 --2- 172.21.15.47:0/766576081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 0x7f580c0a38f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5800009920 tx=0x7f5800009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c39d57c70f698daa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 1 -- 172.21.15.47:0/766576081 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5800009e70 con 0x7f580c09f9a0 2024-04-20T02:31:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:31:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: dump: 2024-04-20T02:31:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.651 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: _finish_auth 0 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f58126a4700 5 monclient: authenticate success, global_id 55257 2024-04-20T02:31:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f58126a4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 1 -- 172.21.15.47:0/766576081 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580000bcd0 con 0x7f580c09f9a0 2024-04-20T02:31:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f58126a4700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:41.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 1 -- 172.21.15.47:0/766576081 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f580000be40 con 0x7f580c09f9a0 2024-04-20T02:31:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f58126a4700 4 set_mon_vals no callback set 2024-04-20T02:31:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.628+0000 7f580affd700 10 monclient: dump: 2024-04-20T02:31:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.654 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.655 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 10 monclient: shutdown 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 -- 172.21.15.47:0/766576081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 msgr2=0x7f580c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 --2- 172.21.15.47:0/766576081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 0x7f580c0a38f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5800009920 tx=0x7f5800009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 -- 172.21.15.47:0/766576081 shutdown_connections 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 --2- 172.21.15.47:0/766576081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c0a1510 0x7f580c0a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 --2- 172.21.15.47:0/766576081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c09f9a0 0x7f580c0a38f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 -- 172.21.15.47:0/766576081 >> 172.21.15.47:0/766576081 conn(0x7f580c0a8510 msgr2=0x7f580c0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:41.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 -- 172.21.15.47:0/766576081 shutdown_connections 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 1 -- 172.21.15.47:0/766576081 wait complete. 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 10 monclient: build_initial_monmap 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.629+0000 7f58126a4700 10 monclient: monmap: 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:41.657 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 1 librados: starting msgr at 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 1 librados: starting objecter 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 1 Processor -- start 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 1 -- start start 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 1 librados: setting wanted keys 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 1 librados: calling monclient init 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 10 monclient: init 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.630+0000 7f58126a4700 10 monclient: _add_conns ranks=[3,7,8,5,1,6,2,0,4] 2024-04-20T02:31:41.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c09f9a0 0x7f580c1cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): picked mon.noname-g con 0x7f580c09f9a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f580c09fe00 0x7f580c1ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): picked mon.noname-i con 0x7f580c09fe00 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 0x7f580c124130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): picked mon.noname-e con 0x7f580c0a1510 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1277a0 con 0x7f580c09f9a0 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f580c126670 con 0x7f580c09fe00 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0a2130 con 0x7f580c0a1510 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient(hunting): authenticate will time out at 2581.032066s 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580bfff700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c09f9a0 msgr2=0x7f580c1cc820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:41.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 0x7f580c124130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c09f9a0 0x7f580c1cc820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 0x7f580c124130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55776/0 (socket says 172.21.15.47:55776) 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 -- 172.21.15.47:0/4026709636 learned_addr learned my addr 172.21.15.47:0/4026709636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): get_auth_request con 0x7f580c0a1510 auth_method 0 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580b7fe700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f580c09fe00 0x7f580c1ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580b7fe700 10 monclient(hunting): get_auth_request con 0x7f580c09fe00 auth_method 0 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient(hunting): handle_auth_done global_id 36035 payload 824 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f580c09fe00 msgr2=0x7f580c1ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f580c09fe00 0x7f580c1ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c09f9a0 msgr2=0x7f580c1cc820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f580c09f9a0 0x7f580c1cc820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient: _finish_hunting 0 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 monclient: found mon.noname-e 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800009550 con 0x7f580c0a1510 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580b7fe700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f580c09fe00 0x7f580c1ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5810c41700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 0x7f580c124130 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f580400cc80 tx=0x7f580400a340 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 client.?.objecter ms_handle_connect 0x7f580c0a1510 2024-04-20T02:31:41.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f580400ce80 con 0x7f580c0a1510 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: dump: 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.662 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: _finish_auth 0 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 5 monclient: authenticate success, global_id 36035 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804003a00 con 0x7f580c0a1510 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient: _renew_subs 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c071040 con 0x7f580c0a1510 2024-04-20T02:31:41.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f58040050d0 con 0x7f580c0a1510 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f58126a4700 10 client.36035.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f580a7fc700 4 set_mon_vals no callback set 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.631+0000 7f5808ff9700 10 monclient: dump: 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.664 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 10 monclient: _renew_subs 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c125c70 con 0x7f580c0a1510 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 1 librados: init done 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 0x7f580c124670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f580c0682f0 con 0x7f580c06aac0 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5804003b70 con 0x7f580c0a1510 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 0x7f580c124670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient: get_auth_request con 0x7f580c06aac0 auth_method 0 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.632+0000 7f580bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f57f803a2f0 0x7f57f803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f580bfff700 10 monclient(hunting): handle_auth_done global_id 55266 payload 824 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f580bfff700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f580c0a1e30 con 0x7f580c06aac0 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(610..610 src has 266..610) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f580404a980 con 0x7f580c0a1510 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 client.36035.objecter ms_dispatch 0x7f580c001550 osd_map(610..610 src has 266..610) v4 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 3 client.36035.objecter handle_osd_map got epochs [610,610] > 0 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 3 client.36035.objecter handle_osd_map decoding full epoch 610 2024-04-20T02:31:41.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f580bfff700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 0x7f580c124670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b450 tx=0x7f57fc00b810 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 20 client.36035.objecter dump_active .. 0 homeless 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 client.36035.objecter ms_handle_connect 0x7f580c06aac0 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 client.36035.objecter resend_mon_ops 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f580b7fe700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f57f803a2f0 0x7f57f803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f580b7fe700 10 monclient: get_auth_request con 0x7f57f803a2f0 auth_method 0 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57fc003c50 con 0x7f580c06aac0 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 monclient: dump: 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.669 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f57fc003db0 con 0x7f580c06aac0 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:41.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 msgr2=0x7f580c124670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 0x7f580c124670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b450 tx=0x7f57fc00b810 comp rx=0 tx=0).stop 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 msgr2=0x7f580c124670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c06aac0 0x7f580c124670 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b450 tx=0x7f57fc00b810 comp rx=0 tx=0).stop 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f580b7fe700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f57f803a2f0 0x7f57f803c7b0 secure :-1 s=READY pgs=3351 cs=0 l=1 rev1=1 crypto rx=0x7f580c0a8fa0 tx=0x7f5800008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.633+0000 7f5808ff9700 10 client.36035.objecter ms_handle_connect 0x7f57f803a2f0 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f58126a4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f58126a4700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 0x7f580c124bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f58126a4700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f580c06a400 con 0x7f580c128460 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 0x7f580c124bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient: get_auth_request con 0x7f580c128460 auth_method 0 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.641+0000 7f580bfff700 10 monclient(hunting): handle_auth_done global_id 55275 payload 824 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f580bfff700 1 -- 172.21.15.47:0/4026709636 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f580c070960 con 0x7f580c128460 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f580bfff700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 0x7f580c124bb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b450 tx=0x7f57fc005a60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 client.36035.objecter ms_handle_connect 0x7f580c128460 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 client.36035.objecter resend_mon_ops 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57fc01e070 con 0x7f580c128460 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 monclient: dump: 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9321 (secure 0 0 0) 0x7f57fc019e90 con 0x7f580c128460 2024-04-20T02:31:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 msgr2=0x7f580c124bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 0x7f580c124bb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b450 tx=0x7f57fc005a60 comp rx=0 tx=0).stop 2024-04-20T02:31:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 msgr2=0x7f580c124bb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:41.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f5808ff9700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f580c128460 0x7f580c124bb0 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00b450 tx=0x7f57fc005a60 comp rx=0 tx=0).stop 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 104, 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:41.676 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.677 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.678 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:41.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.680 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:41.681 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.683 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.684 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:41.686 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.687 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.689 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:41.689 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.689 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.689 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:41.690 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:41.691 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.692 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:41.693 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:41.694 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.694 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.694 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.694 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.694 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:41.695 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.696 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.697 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:41.697 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:41.697 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.697 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.697 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.698 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:41.699 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.700 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.701 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.702 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.703 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:41.704 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.705 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.706 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.707 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.707 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.707 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.707 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:41.707 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 10 librados: watch_flush enter 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 10 librados: watch_flush exit 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 20 client.36035.objecter shutdown clearing up homeless session... 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 10 client.36035.objecter successfully canceled tick 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f57f803a2f0 msgr2=0x7f57f803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f57f803a2f0 0x7f57f803c7b0 secure :-1 s=READY pgs=3351 cs=0 l=1 rev1=1 crypto rx=0x7f580c0a8fa0 tx=0x7f5800008030 comp rx=0 tx=0).stop 2024-04-20T02:31:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.642+0000 7f58126a4700 10 monclient: shutdown 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 msgr2=0x7f580c124130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f5810c41700 1 -- 172.21.15.47:0/4026709636 reap_dead start 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 0x7f580c124130 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f580400cc80 tx=0x7f580400a340 comp rx=0 tx=0).stop 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 shutdown_connections 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 --2- 172.21.15.47:0/4026709636 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f580c0a1510 0x7f580c124130 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 >> 172.21.15.47:0/4026709636 conn(0x7f580c0a8510 msgr2=0x7f580c135e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 shutdown_connections 2024-04-20T02:31:41.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 -- 172.21.15.47:0/4026709636 wait complete. 2024-04-20T02:31:41.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.643+0000 7f58126a4700 1 librados: shutdown 2024-04-20T02:31:41.713 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:31:41.713 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-04-20T02:31:41.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:31:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.854+0000 7fedd30f8700 10 monclient: get_monmap_and_config 2024-04-20T02:31:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.854+0000 7fedd30f8700 10 monclient: build_initial_monmap 2024-04-20T02:31:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.854+0000 7fedd30f8700 10 monclient: monmap: 2024-04-20T02:31:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:41.855603+0000 2024-04-20T02:31:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:41.855603+0000 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:41.857 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:41.858 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 Processor -- start 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 -- start start 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient: init 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient: _add_conns ranks=[5,4,1,3,2,9,8,7,6,10,0] 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc09cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): picked mon.noname-h con 0x7fedcc09c750 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 0x7fedcc0a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): picked mon.noname-c con 0x7fedcc09d2a0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fedcc134c80 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0751e0 con 0x7fedcc134c80 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc135770 con 0x7fedcc09d2a0 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc06cd70 con 0x7fedcc09c750 2024-04-20T02:31:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.855+0000 7fedd30f8700 10 monclient(hunting): authenticate will time out at 2581.256992s 2024-04-20T02:31:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 0x7fedcc0a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 0x7fedcc0a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:56024/0 (socket says 172.21.15.47:56024) 2024-04-20T02:31:41.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 -- 172.21.15.47:0/1393833554 learned_addr learned my addr 172.21.15.47:0/1393833554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): get_auth_request con 0x7fedcc09d2a0 auth_method 0 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc09cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): get_auth_request con 0x7fedcc09c750 auth_method 0 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient(hunting): handle_auth_done global_id 84496 payload 274 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 -- 172.21.15.47:0/1393833554 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 msgr2=0x7fedcc09cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc09cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 -- 172.21.15.47:0/1393833554 >> v1:172.21.15.179:6789/0 conn(0x7fedcc134c80 legacy=0x7fedcc0a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient: _finish_hunting 0 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 monclient: found mon.noname-c 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:41.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 -- 172.21.15.47:0/1393833554 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0a0250 con 0x7fedcc09d2a0 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 1 -- 172.21.15.47:0/1393833554 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253701880 0 0) 0x7fedcc0751e0 con 0x7fedcc134c80 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd0e94700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc09cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcbfff700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 0x7fedcc0a1f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009920 tx=0x7fedb8009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6b64d6ce251337f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 1 -- 172.21.15.47:0/1393833554 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb8011aa0 con 0x7fedcc09d2a0 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: dump: 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.885 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:41.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 1 -- 172.21.15.47:0/1393833554 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8011c00 con 0x7fedcc09d2a0 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd30f8700 5 monclient: authenticate success, global_id 84496 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd30f8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd30f8700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 1 -- 172.21.15.47:0/1393833554 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb8011d70 con 0x7fedcc09d2a0 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedd30f8700 4 set_mon_vals no callback set 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.856+0000 7fedcb7fe700 10 monclient: dump: 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.887 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 10 monclient: shutdown 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 -- 172.21.15.47:0/1393833554 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 msgr2=0x7fedcc0a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 0x7fedcc0a1f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009920 tx=0x7fedb8009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:41.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 -- 172.21.15.47:0/1393833554 shutdown_connections 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc09cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 --2- 172.21.15.47:0/1393833554 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fedcc09d2a0 0x7fedcc0a1f10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 -- 172.21.15.47:0/1393833554 >> 172.21.15.47:0/1393833554 conn(0x7fedcc0a3000 msgr2=0x7fedcc0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 -- 172.21.15.47:0/1393833554 shutdown_connections 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.857+0000 7fedd30f8700 1 -- 172.21.15.47:0/1393833554 wait complete. 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.858+0000 7fedd30f8700 10 monclient: build_initial_monmap 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.858+0000 7fedd30f8700 10 monclient: monmap: 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.858+0000 7fedd30f8700 1 librados: starting msgr at 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.858+0000 7fedd30f8700 1 librados: starting objecter 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.858+0000 7fedd30f8700 1 Processor -- start 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 1 -- start start 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 1 librados: setting wanted keys 2024-04-20T02:31:41.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 1 librados: calling monclient init 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 10 monclient: init 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 10 monclient: _add_conns ranks=[5,3,7,6,2,8,0,4,1] 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc072450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.859+0000 7fedd30f8700 10 monclient(hunting): picked mon.noname-h con 0x7fedcc09c750 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedcc09d2a0 0x7fedcc1c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): picked mon.noname-g con 0x7fedcc09d2a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedcc134c80 0x7fedcc1c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): picked mon.noname-i con 0x7fedcc134c80 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc12e7d0 con 0x7fedcc09d2a0 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1c9100 con 0x7fedcc09c750 2024-04-20T02:31:41.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc12df30 con 0x7fedcc134c80 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd30f8700 10 monclient(hunting): authenticate will time out at 2581.261060s 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedcbfff700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedcc09d2a0 msgr2=0x7fedcc1c93a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedcc09d2a0 0x7fedcc1c93a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedcc134c80 0x7fedcc1c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedcc134c80 0x7fedcc1c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:40318/0 (socket says 172.21.15.47:40318) 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc072450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:43048/0 (socket says 172.21.15.47:43048) 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 1 -- 172.21.15.47:0/740365833 learned_addr learned my addr 172.21.15.47:0/740365833 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): get_auth_request con 0x7fedcc134c80 auth_method 0 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): get_auth_request con 0x7fedcc09c750 auth_method 0 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient(hunting): handle_auth_done global_id 84470 payload 824 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedcc134c80 msgr2=0x7fedcc1c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedcc134c80 0x7fedcc1c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedcc09d2a0 msgr2=0x7fedcc1c93a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fedcc09d2a0 0x7fedcc1c93a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient: _finish_hunting 0 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 monclient: found mon.noname-h 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00094a0 con 0x7fedcc09c750 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd1695700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedcc134c80 0x7fedcc1c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedd0e94700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc072450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fedc000e9b0 tx=0x7fedc000ecc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedc97fa700 10 client.?.objecter ms_handle_connect 0x7fedcc09c750 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.860+0000 7fedc97fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedc000ccc0 con 0x7fedcc09c750 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: dump: 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.894 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: _finish_auth 0 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: _check_auth_tickets 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc000ce60 con 0x7fedcc09c750 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 5 monclient: authenticate success, global_id 84470 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 monclient: _renew_subs 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb8009550 con 0x7fedcc09c750 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedc00138e0 con 0x7fedcc09c750 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedc97fa700 10 monclient: dump: 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedcaffd700 4 set_mon_vals no callback set 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 client.84470.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 monclient: _renew_subs 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc12e4f0 con 0x7fedcc09c750 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 1 librados: init done 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 0x7fedcc12b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.861+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc068230 con 0x7fedcc06aac0 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 0x7fedcc12b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fedc0013470 con 0x7fedcc09c750 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient: get_auth_request con 0x7fedcc06aac0 auth_method 0 2024-04-20T02:31:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fedb003a320 0x7fedb003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 10 monclient(hunting): handle_auth_done global_id 36044 payload 824 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedcc09efa0 con 0x7fedcc06aac0 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(610..610 src has 266..610) v4 ==== 6689+0+0 (secure 0 0 0) 0x7fedc004a880 con 0x7fedcc09c750 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 10 client.84470.objecter ms_dispatch 0x7fedcc001550 osd_map(610..610 src has 266..610) v4 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 3 client.84470.objecter handle_osd_map got epochs [610,610] > 0 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 3 client.84470.objecter handle_osd_map decoding full epoch 610 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 20 client.84470.objecter dump_active .. 0 homeless 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 10 client.84470.objecter ms_handle_connect 0x7fedcc06aac0 2024-04-20T02:31:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedcbfff700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 0x7fedcc12b5c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009920 tx=0x7fedb8004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 10 client.84470.objecter resend_mon_ops 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedd1695700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fedb003a320 0x7fedb003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedd1695700 10 monclient: get_auth_request con 0x7fedb003a320 auth_method 0 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb800b890 con 0x7fedcc06aac0 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.862+0000 7fedc97fa700 10 monclient: dump: 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.900 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 10 client.84470.objecter ms_handle_connect 0x7fedb003a320 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedd1695700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fedb003a320 0x7fedb003c7e0 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7fedcc12ce40 tx=0x7fedc4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fedb800b420 con 0x7fedcc06aac0 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 msgr2=0x7fedcc12b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 0x7fedcc12b5c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009920 tx=0x7fedb8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 msgr2=0x7fedcc12b5c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.863+0000 7fedc97fa700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc06aac0 0x7fedcc12b5c0 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009920 tx=0x7fedb8004750 comp rx=0 tx=0).stop 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.870+0000 7fedd30f8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.870+0000 7fedd30f8700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 0x7fedcc12bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.870+0000 7fedd30f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.870+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc06a960 con 0x7fedcc12bb00 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 0x7fedcc12bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient: get_auth_request con 0x7fedcc12bb00 auth_method 0 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:41.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 10 monclient(hunting): handle_auth_done global_id 36053 payload 824 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 1 -- 172.21.15.47:0/740365833 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fedcc09fc90 con 0x7fedcc12bb00 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedcbfff700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 0x7fedcc12bf10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098f0 tx=0x7fedb80092c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedc97fa700 10 client.84470.objecter ms_handle_connect 0x7fedcc12bb00 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedc97fa700 10 client.84470.objecter resend_mon_ops 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedb8029250 con 0x7fedcc12bb00 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedc97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedc97fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.871+0000 7fedc97fa700 10 monclient: dump: 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:41.903 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9321 (secure 0 0 0) 0x7fedb801ab40 con 0x7fedcc12bb00 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 msgr2=0x7fedcc12bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 0x7fedcc12bf10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098f0 tx=0x7fedb80092c0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 msgr2=0x7fedcc12bf10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedc97fa700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedcc12bb00 0x7fedcc12bf10 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098f0 tx=0x7fedb80092c0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 104, 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:41.905 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.906 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.907 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:41.908 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:41.909 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:41.910 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:41.912 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.913 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:41.914 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.915 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.916 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.918 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.919 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.921 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.923 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.925 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.926 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:41.927 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 10 librados: watch_flush enter 2024-04-20T02:31:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 10 librados: watch_flush exit 2024-04-20T02:31:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 20 client.84470.objecter shutdown clearing up homeless session... 2024-04-20T02:31:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 10 client.84470.objecter successfully canceled tick 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fedb003a320 msgr2=0x7fedb003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fedb003a320 0x7fedb003c7e0 secure :-1 s=READY pgs=3352 cs=0 l=1 rev1=1 crypto rx=0x7fedcc12ce40 tx=0x7fedc4009040 comp rx=0 tx=0).stop 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 10 monclient: shutdown 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 msgr2=0x7fedcc072450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc072450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fedc000e9b0 tx=0x7fedc000ecc0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd1695700 1 -- 172.21.15.47:0/740365833 reap_dead start 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 shutdown_connections 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 --2- 172.21.15.47:0/740365833 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedcc09c750 0x7fedcc072450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 >> 172.21.15.47:0/740365833 conn(0x7fedcc0a3000 msgr2=0x7fedcc1bc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 shutdown_connections 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 -- 172.21.15.47:0/740365833 wait complete. 2024-04-20T02:31:41.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:41.872+0000 7fedd30f8700 1 librados: shutdown 2024-04-20T02:31:41.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:31:41.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-04-20T02:31:41.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:31:42.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.071+0000 7f3214e9c700 10 monclient: get_monmap_and_config 2024-04-20T02:31:42.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.071+0000 7f3214e9c700 10 monclient: build_initial_monmap 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.071+0000 7f3214e9c700 10 monclient: monmap: 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:42.072642+0000 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:42.072642+0000 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:42.072 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:42.073 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:42.073 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:42.073 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:42.073 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:42.073 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 1 Processor -- start 2024-04-20T02:31:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 1 -- start start 2024-04-20T02:31:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 10 monclient: init 2024-04-20T02:31:42.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 10 monclient: _add_conns ranks=[6,4,3,0,10,8,2,7,9,5,1] 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 0x7f3210071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 10 monclient(hunting): picked mon.noname-d con 0x7f3210071630 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 0x7f32100a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 10 monclient(hunting): picked mon.noname-c con 0x7f3210071f50 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.072+0000 7f3214e9c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32100a2e90 0x7f32100a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient(hunting): picked mon.noname-g con 0x7f32100a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210072720 con 0x7f32100a2e90 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32100a0820 con 0x7f3210071f50 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f321009e890 con 0x7f3210071630 2024-04-20T02:31:42.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient(hunting): authenticate will time out at 2581.474052s 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 0x7f3210071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 0x7f32100a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320ed9d700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32100a2e90 msgr2=0x7f32100a3310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 0x7f32100a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:56038/0 (socket says 172.21.15.47:56038) 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320ed9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32100a2e90 0x7f32100a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 0x7f3210071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57698/0 (socket says 172.21.15.47:57698) 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 -- 172.21.15.47:0/608035789 learned_addr learned my addr 172.21.15.47:0/608035789 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): get_auth_request con 0x7f3210071f50 auth_method 0 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): get_auth_request con 0x7f3210071630 auth_method 0 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient(hunting): handle_auth_done global_id 84505 payload 274 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 -- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 msgr2=0x7f3210071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 0x7f3210071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 -- 172.21.15.47:0/608035789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32100a2e90 msgr2=0x7f32100a3310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32100a2e90 0x7f32100a3310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 monclient: found mon.noname-c 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 -- 172.21.15.47:0/608035789 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100751e0 con 0x7f3210071f50 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320e59c700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 0x7f3210071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320dd9b700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 0x7f32100a2950 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f320400bbf0 tx=0x7f320400bf00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=552041001c9a3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 1 -- 172.21.15.47:0/608035789 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3204010470 con 0x7f3210071f50 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: dump: 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.089 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: _finish_auth 0 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: _check_auth_tickets 2024-04-20T02:31:42.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 1 -- 172.21.15.47:0/608035789 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32040105d0 con 0x7f3210071f50 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 1 -- 172.21.15.47:0/608035789 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3204010bb0 con 0x7f3210071f50 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 5 monclient: authenticate success, global_id 84505 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f3214e9c700 4 set_mon_vals no callback set 2024-04-20T02:31:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.073+0000 7f320d59a700 10 monclient: dump: 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 10 monclient: shutdown 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 -- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 msgr2=0x7f32100a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 0x7f32100a2950 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f320400bbf0 tx=0x7f320400bf00 comp rx=0 tx=0).stop 2024-04-20T02:31:42.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 -- 172.21.15.47:0/608035789 shutdown_connections 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f32100a2e90 0x7f32100a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3210071630 0x7f3210071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 --2- 172.21.15.47:0/608035789 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3210071f50 0x7f32100a2950 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 -- 172.21.15.47:0/608035789 >> 172.21.15.47:0/608035789 conn(0x7f32100a7570 msgr2=0x7f32100a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 -- 172.21.15.47:0/608035789 shutdown_connections 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.074+0000 7f3214e9c700 1 -- 172.21.15.47:0/608035789 wait complete. 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.075+0000 7f3214e9c700 10 monclient: build_initial_monmap 2024-04-20T02:31:42.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.075+0000 7f3214e9c700 10 monclient: monmap: 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:42.095 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.075+0000 7f3214e9c700 1 librados: starting msgr at 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.075+0000 7f3214e9c700 1 librados: starting objecter 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.075+0000 7f3214e9c700 1 Processor -- start 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 -- start start 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 librados: setting wanted keys 2024-04-20T02:31:42.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 librados: calling monclient init 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient: init 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient: _add_conns ranks=[0,5,7,2,1,4,6,8,3] 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3210071630 0x7f32101c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): picked mon.noname-a con 0x7f3210071630 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3210071f50 0x7f32101b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): picked mon.noname-h con 0x7f3210071f50 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f32100a2e90 0x7f32101b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): picked mon.noname-i con 0x7f32100a2e90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320e59c700 1 -- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3210071630 msgr2=0x7f32101c8130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321009f200 con 0x7f3210071630 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3210072380 con 0x7f3210071f50 2024-04-20T02:31:42.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f32101a60d0 con 0x7f32100a2e90 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3210071630 0x7f32101c8130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f3214e9c700 10 monclient(hunting): authenticate will time out at 2581.477606s 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3210071f50 0x7f32101b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320ed9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f32100a2e90 0x7f32101b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3210071f50 0x7f32101b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:43058/0 (socket says 172.21.15.47:43058) 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 1 -- 172.21.15.47:0/3741443932 learned_addr learned my addr 172.21.15.47:0/3741443932 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320ed9d700 10 monclient(hunting): get_auth_request con 0x7f32100a2e90 auth_method 0 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320ed9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320ed9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320ed9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 10 monclient(hunting): get_auth_request con 0x7f3210071f50 auth_method 0 2024-04-20T02:31:42.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.076+0000 7f320dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320ed9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320ed9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320ed9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 10 monclient(hunting): handle_auth_done global_id 84479 payload 824 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f32100a2e90 msgr2=0x7f32101b8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f32100a2e90 0x7f32101b8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3210071630 msgr2=0x7f32101c8130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3210071630 0x7f32101c8130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:31:42.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 monclient: found mon.noname-h 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000094a0 con 0x7f3210071f50 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320ed9d700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f32100a2e90 0x7f32101b8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320dd9b700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3210071f50 0x7f32101b8710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3204015000 tx=0x7f3204004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 client.?.objecter ms_handle_connect 0x7f3210071f50 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f320401f070 con 0x7f3210071f50 2024-04-20T02:31:42.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: dump: 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.101 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: _finish_auth 0 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: _check_auth_tickets 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3204010dc0 con 0x7f3210071f50 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 5 monclient: authenticate success, global_id 84479 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3204010470 con 0x7f3210071f50 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 10 monclient: _renew_subs 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320400b7f0 con 0x7f3210071f50 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f320cd99700 4 set_mon_vals no callback set 2024-04-20T02:31:42.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f31feffd700 10 monclient: dump: 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.103 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 10 client.84479.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 10 monclient: _renew_subs 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32101a5830 con 0x7f3210071f50 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.077+0000 7f3214e9c700 1 librados: init done 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f3214e9c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f3214e9c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 0x7f32101a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3210068310 con 0x7f321006aac0 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3204010950 con 0x7f3210071f50 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 0x7f32101a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 10 monclient: get_auth_request con 0x7f321006aac0 auth_method 0 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f31feffd700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f31f003a2f0 0x7f31f003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(610..610 src has 266..610) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f320404dab0 con 0x7f3210071f50 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.078+0000 7f31feffd700 10 client.84479.objecter ms_dispatch 0x7f3210001550 osd_map(610..610 src has 266..610) v4 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 3 client.84479.objecter handle_osd_map got epochs [610,610] > 0 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 3 client.84479.objecter handle_osd_map decoding full epoch 610 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 20 client.84479.objecter dump_active .. 0 homeless 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320ed9d700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f31f003a2f0 0x7f31f003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320e59c700 10 monclient(hunting): handle_auth_done global_id 55666 payload 824 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320e59c700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32101a6240 con 0x7f321006aac0 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320ed9d700 10 monclient: get_auth_request con 0x7f31f003a2f0 auth_method 0 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 client.84479.objecter ms_handle_connect 0x7f321006aac0 2024-04-20T02:31:42.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320e59c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 0x7f32101a22a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f32000096f0 tx=0x7f320000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 client.84479.objecter resend_mon_ops 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32000044a0 con 0x7f321006aac0 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 monclient: dump: 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.106 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 client.84479.objecter ms_handle_connect 0x7f31f003a2f0 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f320ed9d700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f31f003a2f0 0x7f31f003c7b0 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 crypto rx=0x7f3208003d40 tx=0x7f32080095f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3200004030 con 0x7f321006aac0 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 msgr2=0x7f32101a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 0x7f32101a22a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f32000096f0 tx=0x7f320000ee70 comp rx=0 tx=0).stop 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 msgr2=0x7f32101a22a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:42.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.079+0000 7f31feffd700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f321006aac0 0x7f32101a22a0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f32000096f0 tx=0x7f320000ee70 comp rx=0 tx=0).stop 2024-04-20T02:31:42.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.087+0000 7f3214e9c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.087+0000 7f3214e9c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 0x7f32101a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.087+0000 7f3214e9c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.087+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321006a400 con 0x7f32101a27e0 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 0x7f32101a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient: get_auth_request con 0x7f32101a27e0 auth_method 0 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 10 monclient(hunting): handle_auth_done global_id 55675 payload 824 2024-04-20T02:31:42.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 1 -- 172.21.15.47:0/3741443932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f32100a4480 con 0x7f32101a27e0 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f320e59c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 0x7f32101a2bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32000096f0 tx=0x7f32000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f31feffd700 10 client.84479.objecter ms_handle_connect 0x7f32101a27e0 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f31feffd700 10 client.84479.objecter resend_mon_ops 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.088+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3200003d00 con 0x7f32101a27e0 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 10 monclient: dump: 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.111 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9323 (secure 0 0 0) 0x7f3200003e60 con 0x7f32101a27e0 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 msgr2=0x7f32101a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 0x7f32101a2bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32000096f0 tx=0x7f32000047c0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 msgr2=0x7f32101a2bf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:42.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.089+0000 7f31feffd700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32101a27e0 0x7f32101a2bf0 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32000096f0 tx=0x7f32000047c0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 104, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.113 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.114 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.115 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:42.116 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.117 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.118 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:42.119 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:42.120 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:42.122 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:42.123 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.124 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:42.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:42.126 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.127 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:42.129 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.130 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.134 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:42.135 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.136 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:42.137 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 10 librados: watch_flush enter 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 10 librados: watch_flush exit 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 20 client.84479.objecter shutdown clearing up homeless session... 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 10 client.84479.objecter successfully canceled tick 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f31f003a2f0 msgr2=0x7f31f003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f31f003a2f0 0x7f31f003c7b0 secure :-1 s=READY pgs=3353 cs=0 l=1 rev1=1 crypto rx=0x7f3208003d40 tx=0x7f32080095f0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 10 monclient: shutdown 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:42.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3210071f50 msgr2=0x7f32101b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 --2- 172.21.15.47:0/3741443932 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3210071f50 0x7f32101b8710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3204015000 tx=0x7f3204004750 comp rx=0 tx=0).stop 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f320ed9d700 1 -- 172.21.15.47:0/3741443932 reap_dead start 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f320ed9d700 1 -- 172.21.15.47:0/3741443932 reap_dead start 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 shutdown_connections 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 >> 172.21.15.47:0/3741443932 conn(0x7f32100a7570 msgr2=0x7f32101bb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 shutdown_connections 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 -- 172.21.15.47:0/3741443932 wait complete. 2024-04-20T02:31:42.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.090+0000 7f3214e9c700 1 librados: shutdown 2024-04-20T02:31:42.140 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:31:42.140 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-04-20T02:31:42.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.279+0000 7ff3e1e37700 10 monclient: get_monmap_and_config 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.279+0000 7ff3e1e37700 10 monclient: build_initial_monmap 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.279+0000 7ff3e1e37700 10 monclient: monmap: 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:42.281058+0000 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:42.281058+0000 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:42.280 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:42.281 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.280+0000 7ff3e1e37700 1 Processor -- start 2024-04-20T02:31:42.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.280+0000 7ff3e1e37700 1 -- start start 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.280+0000 7ff3e1e37700 10 monclient: init 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient: _add_conns ranks=[2,8,1,4,9,5,6,7,0,3,10] 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3dc0a0830 0x7ff3dc0a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): picked mon.noname-b con 0x7ff3dc0a0830 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 0x7ff3dc0a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): picked mon.noname-e con 0x7ff3dc0a01b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff3dc0a1150 addr v1:172.21.15.179:6789/0 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc0751e0 con 0x7ff3dc0a1150 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc09dee0 con 0x7ff3dc0a0830 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc067820 con 0x7ff3dc0a01b0 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:42.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3e1e37700 10 monclient(hunting): authenticate will time out at 2581.682343s 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3db7fe700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3dc0a0830 msgr2=0x7ff3dc0a0c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3db7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3dc0a0830 0x7ff3dc0a0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 0x7ff3dc0a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 0x7ff3dc0a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55800/0 (socket says 172.21.15.47:55800) 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 -- 172.21.15.47:0/2560264060 learned_addr learned my addr 172.21.15.47:0/2560264060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): get_auth_request con 0x7ff3dc0a01b0 auth_method 0 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient(hunting): handle_auth_done global_id 36062 payload 274 2024-04-20T02:31:42.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 -- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3dc0a0830 msgr2=0x7ff3dc0a0c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 --2- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3dc0a0830 0x7ff3dc0a0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 -- 172.21.15.47:0/2560264060 >> v1:172.21.15.179:6789/0 conn(0x7ff3dc0a1150 legacy=0x7ff3dc0a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient: _finish_hunting 0 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 monclient: found mon.noname-e 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 -- 172.21.15.47:0/2560264060 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc09eee0 con 0x7ff3dc0a01b0 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.281+0000 7ff3daffd700 1 --2- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 0x7ff3dc0a2950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009920 tx=0x7ff3d0009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5118d702a735c801 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 1 -- 172.21.15.47:0/2560264060 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3d0009e70 con 0x7ff3dc0a01b0 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: dump: 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.295 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: _finish_auth 0 2024-04-20T02:31:42.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 5 monclient: authenticate success, global_id 36062 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 1 -- 172.21.15.47:0/2560264060 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000bcd0 con 0x7ff3dc0a01b0 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 4 set_mon_vals no callback set 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 1 -- 172.21.15.47:0/2560264060 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3d000be40 con 0x7ff3dc0a01b0 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3da7fc700 10 monclient: dump: 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.297 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 10 monclient: shutdown 2024-04-20T02:31:42.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 -- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 msgr2=0x7ff3dc0a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 0x7ff3dc0a2950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009920 tx=0x7ff3d0009c30 comp rx=0 tx=0).stop 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 -- 172.21.15.47:0/2560264060 shutdown_connections 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff3dc0a0830 0x7ff3dc0a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/2560264060 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a01b0 0x7ff3dc0a2950 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 -- 172.21.15.47:0/2560264060 >> 172.21.15.47:0/2560264060 conn(0x7ff3dc0a7570 msgr2=0x7ff3dc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 -- 172.21.15.47:0/2560264060 shutdown_connections 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.282+0000 7ff3e1e37700 1 -- 172.21.15.47:0/2560264060 wait complete. 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.283+0000 7ff3e1e37700 10 monclient: build_initial_monmap 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.283+0000 7ff3e1e37700 10 monclient: monmap: 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:42.299 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.283+0000 7ff3e1e37700 1 librados: starting msgr at 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.283+0000 7ff3e1e37700 1 librados: starting objecter 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.283+0000 7ff3e1e37700 1 Processor -- start 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 -- start start 2024-04-20T02:31:42.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 librados: setting wanted keys 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 librados: calling monclient init 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient: init 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient: _add_conns ranks=[6,1,8,3,0,7,4,5,2] 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3dc0a01b0 0x7ff3dc1cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): picked mon.noname-d con 0x7ff3dc0a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3dc0a0830 0x7ff3dc1ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): picked mon.noname-f con 0x7ff3dc0a0830 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a1150 0x7ff3dc1baf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): picked mon.noname-e con 0x7ff3dc0a1150 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1a9e10 con 0x7ff3dc0a0830 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1cc910 con 0x7ff3dc0a01b0 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc09e440 con 0x7ff3dc0a1150 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3e1e37700 10 monclient(hunting): authenticate will time out at 2581.685599s 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3dc0a01b0 0x7ff3dc1cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a1150 0x7ff3dc1baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3dc0a01b0 0x7ff3dc1cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57714/0 (socket says 172.21.15.47:57714) 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 1 -- 172.21.15.47:0/3637510714 learned_addr learned my addr 172.21.15.47:0/3637510714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): get_auth_request con 0x7ff3dc0a01b0 auth_method 0 2024-04-20T02:31:42.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3dc0a0830 0x7ff3dc1ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): get_auth_request con 0x7ff3dc0a1150 auth_method 0 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): get_auth_request con 0x7ff3dc0a0830 auth_method 0 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3dbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.284+0000 7ff3daffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 10 monclient(hunting): handle_auth_done global_id 36071 payload 824 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3dc0a01b0 msgr2=0x7ff3dc1cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3dc0a01b0 0x7ff3dc1cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3dc0a0830 msgr2=0x7ff3dc1ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3dc0a0830 0x7ff3dc1ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 10 monclient: _finish_hunting 0 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 monclient: found mon.noname-e 2024-04-20T02:31:42.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0009550 con 0x7ff3dc0a1150 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3db7fe700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff3dc0a01b0 0x7ff3dc1cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3daffd700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3dc0a0830 0x7ff3dc1ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3dbfff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a1150 0x7ff3dc1baf30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400cae0 tx=0x7ff3d400cdf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 client.?.objecter ms_handle_connect 0x7ff3dc0a1150 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3d4003e70 con 0x7ff3dc0a1150 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: dump: 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.305 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: _finish_auth 0 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:31:42.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4001b80 con 0x7ff3dc0a1150 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 5 monclient: authenticate success, global_id 36071 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 10 monclient: _renew_subs 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc1a8bd0 con 0x7ff3dc0a1150 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3d400ac50 con 0x7ff3dc0a1150 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3c3fff700 10 monclient: dump: 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 10 client.36071.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3d9ffb700 4 set_mon_vals no callback set 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 10 monclient: _renew_subs 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc1a9b30 con 0x7ff3dc0a1150 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.285+0000 7ff3e1e37700 1 librados: init done 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff3d400a7e0 con 0x7ff3dc0a1150 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3e1e37700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 0x7ff3dc1a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1a9cc0 con 0x7ff3dc06aac0 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff3c803a2f0 0x7ff3c803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(610..610 src has 266..610) v4 ==== 6689+0+0 (secure 0 0 0) 0x7ff3d404a840 con 0x7ff3dc0a1150 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 10 client.36071.objecter ms_dispatch 0x7ff3dc001550 osd_map(610..610 src has 266..610) v4 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 3 client.36071.objecter handle_osd_map got epochs [610,610] > 0 2024-04-20T02:31:42.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 3 client.36071.objecter handle_osd_map decoding full epoch 610 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.286+0000 7ff3c3fff700 20 client.36071.objecter dump_active .. 0 homeless 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 0x7ff3dc1a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient: get_auth_request con 0x7ff3dc06aac0 auth_method 0 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3daffd700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff3c803a2f0 0x7ff3c803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3daffd700 10 monclient: get_auth_request con 0x7ff3c803a2f0 auth_method 0 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_done global_id 84488 payload 824 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3dc0672f0 con 0x7ff3dc06aac0 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3db7fe700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 0x7ff3dc1a66b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005620 tx=0x7ff3cc0059e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 10 client.36071.objecter ms_handle_connect 0x7ff3dc06aac0 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 10 client.36071.objecter resend_mon_ops 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 10 client.36071.objecter ms_handle_connect 0x7ff3c803a2f0 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3daffd700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff3c803a2f0 0x7ff3c803c7b0 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0a05d0 tx=0x7ff3d0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3cc00ed80 con 0x7ff3dc06aac0 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.287+0000 7ff3c3fff700 10 monclient: dump: 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.311 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.312 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff3cc00e910 con 0x7ff3dc06aac0 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 msgr2=0x7ff3dc1a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 0x7ff3dc1a66b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005620 tx=0x7ff3cc0059e0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 msgr2=0x7ff3dc1a66b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:42.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.288+0000 7ff3c3fff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc06aac0 0x7ff3dc1a66b0 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc005620 tx=0x7ff3cc0059e0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3e1e37700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 0x7ff3dc1a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3e1e37700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc068310 con 0x7ff3dc1a6bf0 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 0x7ff3dc1a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient: get_auth_request con 0x7ff3dc1a6bf0 auth_method 0 2024-04-20T02:31:42.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 10 monclient(hunting): handle_auth_done global_id 84497 payload 824 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.296+0000 7ff3db7fe700 1 -- 172.21.15.47:0/3637510714 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3dc09f1c0 con 0x7ff3dc1a6bf0 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3db7fe700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 0x7ff3dc1a7000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc018000 tx=0x7ff3cc005a60 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 client.36071.objecter ms_handle_connect 0x7ff3dc1a6bf0 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 client.36071.objecter resend_mon_ops 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3cc00ebc0 con 0x7ff3dc1a6bf0 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 monclient: dump: 2024-04-20T02:31:42.316 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.317 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9321 (secure 0 0 0) 0x7ff3cc012600 con 0x7ff3dc1a6bf0 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 msgr2=0x7ff3dc1a7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 0x7ff3dc1a7000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc018000 tx=0x7ff3cc005a60 comp rx=0 tx=0).stop 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 msgr2=0x7ff3dc1a7000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.297+0000 7ff3c3fff700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ff3dc1a6bf0 0x7ff3dc1a7000 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc018000 tx=0x7ff3cc005a60 comp rx=0 tx=0).stop 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:42.318 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 104, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:42.319 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.320 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:42.321 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.322 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.323 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:42.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.329 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.330 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.331 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:42.333 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:42.334 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.336 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.337 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.338 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:42.339 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.340 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.340 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:42.340 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:42.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 10 librados: watch_flush enter 2024-04-20T02:31:42.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 10 librados: watch_flush exit 2024-04-20T02:31:42.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 20 client.36071.objecter shutdown clearing up homeless session... 2024-04-20T02:31:42.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 10 client.36071.objecter successfully canceled tick 2024-04-20T02:31:42.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff3c803a2f0 msgr2=0x7ff3c803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff3c803a2f0 0x7ff3c803c7b0 secure :-1 s=READY pgs=3354 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0a05d0 tx=0x7ff3d0008030 comp rx=0 tx=0).stop 2024-04-20T02:31:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 10 monclient: shutdown 2024-04-20T02:31:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a1150 msgr2=0x7ff3dc1baf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 --2- 172.21.15.47:0/3637510714 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff3dc0a1150 0x7ff3dc1baf30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400cae0 tx=0x7ff3d400cdf0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3dbfff700 1 -- 172.21.15.47:0/3637510714 reap_dead start 2024-04-20T02:31:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3dbfff700 1 -- 172.21.15.47:0/3637510714 reap_dead start 2024-04-20T02:31:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 shutdown_connections 2024-04-20T02:31:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 >> 172.21.15.47:0/3637510714 conn(0x7ff3dc0a7570 msgr2=0x7ff3dc1be2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 shutdown_connections 2024-04-20T02:31:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 -- 172.21.15.47:0/3637510714 wait complete. 2024-04-20T02:31:42.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.298+0000 7ff3e1e37700 1 librados: shutdown 2024-04-20T02:31:42.344 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:31:42.344 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-04-20T02:31:42.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.493+0000 7fc6ead2f700 10 monclient: get_monmap_and_config 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.493+0000 7fc6ead2f700 10 monclient: build_initial_monmap 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.493+0000 7fc6ead2f700 10 monclient: monmap: 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:31:42.494261+0000 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:31:42.494261+0000 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.493 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:31:42.494 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.493+0000 7fc6ead2f700 1 Processor -- start 2024-04-20T02:31:42.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 -- start start 2024-04-20T02:31:42.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient: init 2024-04-20T02:31:42.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient: _add_conns ranks=[9,4,6,10,1,5,2,0,3,7,8] 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e409fa80 0x7fc6e4135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): picked mon.noname-a con 0x7fc6e409fa80 addr v2:172.21.15.47:3300/0 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 0x7fc6e40a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): picked mon.noname-c con 0x7fc6e409fe60 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 0x7fc6e40a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): picked mon.noname-d con 0x7fc6e40a1f10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e40a0780 con 0x7fc6e409fa80 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e409df90 con 0x7fc6e409fe60 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e409dcb0 con 0x7fc6e40a1f10 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:42.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6ead2f700 10 monclient(hunting): authenticate will time out at 2581.895650s 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e8acb700 1 -- >> v2:172.21.15.47:3300/0 conn(0x7fc6e409fa80 msgr2=0x7fc6e4135df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3300/0 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e8acb700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc6e409fa80 0x7fc6e4135df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 0x7fc6e40a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 0x7fc6e40a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 0x7fc6e40a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:57722/0 (socket says 172.21.15.47:57722) 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 0x7fc6e40a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:56046/0 (socket says 172.21.15.47:56046) 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 1 -- 172.21.15.47:0/4094928317 learned_addr learned my addr 172.21.15.47:0/4094928317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): get_auth_request con 0x7fc6e40a1f10 auth_method 0 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): get_auth_request con 0x7fc6e409fe60 auth_method 0 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.494+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_done global_id 84514 payload 274 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 -- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 msgr2=0x7fc6e40a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 --2- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 0x7fc6e40a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 -- 172.21.15.47:0/4094928317 >> v2:172.21.15.47:3300/0 conn(0x7fc6e409fa80 msgr2=0x7fc6e4135df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 --2- 172.21.15.47:0/4094928317 >> v2:172.21.15.47:3300/0 conn(0x7fc6e409fa80 0x7fc6e4135df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 monclient: found mon.noname-c 2024-04-20T02:31:42.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 -- 172.21.15.47:0/4094928317 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e40751e0 con 0x7fc6e409fe60 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e92cc700 1 --2- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 0x7fc6e40a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e3fff700 1 --2- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 0x7fc6e40a0240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009870 tx=0x7fc6d0009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e01978edd048e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 1 -- 172.21.15.47:0/4094928317 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d0011aa0 con 0x7fc6e409fe60 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: dump: 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.510 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: _finish_auth 0 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 1 -- 172.21.15.47:0/4094928317 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d0011c00 con 0x7fc6e409fe60 2024-04-20T02:31:42.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6ead2f700 5 monclient: authenticate success, global_id 84514 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6ead2f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6ead2f700 10 monclient: get_monmap_and_config success 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 1 -- 172.21.15.47:0/4094928317 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d0011d70 con 0x7fc6e409fe60 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6ead2f700 4 set_mon_vals no callback set 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.495+0000 7fc6e37fe700 10 monclient: dump: 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.512 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 10 monclient: shutdown 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 -- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 msgr2=0x7fc6e40a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 0x7fc6e40a0240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009870 tx=0x7fc6d0009b80 comp rx=0 tx=0).stop 2024-04-20T02:31:42.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 -- 172.21.15.47:0/4094928317 shutdown_connections 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a1f10 0x7fc6e40a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/4094928317 >> v2:172.21.15.47:3300/0 conn(0x7fc6e409fa80 0x7fc6e4135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/4094928317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc6e409fe60 0x7fc6e40a0240 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 -- 172.21.15.47:0/4094928317 >> 172.21.15.47:0/4094928317 conn(0x7fc6e40a3000 msgr2=0x7fc6e40a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 -- 172.21.15.47:0/4094928317 shutdown_connections 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 1 -- 172.21.15.47:0/4094928317 wait complete. 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.496+0000 7fc6ead2f700 10 monclient: build_initial_monmap 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 10 monclient: monmap: 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:31:42.514 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 1 librados: starting msgr at 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 1 librados: starting objecter 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 1 Processor -- start 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 1 -- start start 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 1 librados: setting wanted keys 2024-04-20T02:31:42.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 1 librados: calling monclient init 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.497+0000 7fc6ead2f700 10 monclient: init 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient: _reopen_session rank -1 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient: _add_conns ranks=[5,7,8,0,3,2,1,4,6] 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc6e409fa80 0x7fc6e4132b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): picked mon.noname-h con 0x7fc6e409fa80 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e409fe60 0x7fc6e41330c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): picked mon.noname-i con 0x7fc6e409fe60 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 0x7fc6e41ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): picked mon.noname-e con 0x7fc6e40a1f10 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e409d010 con 0x7fc6e409fa80 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e409cb50 con 0x7fc6e409fe60 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4133600 con 0x7fc6e40a1f10 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): _renew_subs 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6ead2f700 10 monclient(hunting): authenticate will time out at 2581.899785s 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 0x7fc6e41ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 0x7fc6e41ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:55822/0 (socket says 172.21.15.47:55822) 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 1 -- 172.21.15.47:0/2716752136 learned_addr learned my addr 172.21.15.47:0/2716752136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 10 monclient(hunting): get_auth_request con 0x7fc6e40a1f10 auth_method 0 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e92cc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e8acb700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc6e409fa80 0x7fc6e4132b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e3fff700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e409fe60 0x7fc6e41330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e3fff700 10 monclient(hunting): get_auth_request con 0x7fc6e409fe60 auth_method 0 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.498+0000 7fc6e3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): get_auth_request con 0x7fc6e409fa80 auth_method 0 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 10 monclient(hunting): handle_auth_done global_id 36080 payload 824 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e409fe60 msgr2=0x7fc6e41330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e409fe60 0x7fc6e41330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc6e409fa80 msgr2=0x7fc6e4132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fc6e409fa80 0x7fc6e4132b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 10 monclient: _finish_hunting 0 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 monclient: found mon.noname-e 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc009b30 con 0x7fc6e40a1f10 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e92cc700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 0x7fc6e41ca010 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00efc0 tx=0x7fc6dc00c3a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 client.?.objecter ms_handle_connect 0x7fc6e40a1f10 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6dc00ccf0 con 0x7fc6e40a1f10 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: dump: 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.520 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: _finish_auth 0 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: _check_auth_tickets 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc00f040 con 0x7fc6e40a1f10 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 5 monclient: authenticate success, global_id 36080 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 10 monclient: _renew_subs 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d00094a0 con 0x7fc6e40a1f10 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6dc00ce90 con 0x7fc6e40a1f10 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:31:42.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e17fa700 10 monclient: dump: 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.522 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6e2ffd700 4 set_mon_vals no callback set 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 10 client.36080.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 10 monclient: _renew_subs 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e412e340 con 0x7fc6e40a1f10 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.499+0000 7fc6ead2f700 1 librados: init done 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.500+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc6dc00c880 con 0x7fc6e40a1f10 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.500+0000 7fc6ead2f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.500+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 0x7fc6e41ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.500+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.500+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4068180 con 0x7fc6e406aac0 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.500+0000 7fc6e17fa700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c803a320 0x7fc6c803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(611..611 src has 266..611) v4 ==== 6702+0+0 (secure 0 0 0) 0x7fc6dc04ad10 con 0x7fc6e40a1f10 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 client.36080.objecter ms_dispatch 0x7fc6e4001550 osd_map(611..611 src has 266..611) v4 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 3 client.36080.objecter handle_osd_map got epochs [611,611] > 0 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 3 client.36080.objecter handle_osd_map decoding full epoch 611 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 0x7fc6e41ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient: get_auth_request con 0x7fc6e406aac0 auth_method 0 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 20 client.36080.objecter dump_active .. 0 homeless 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e3fff700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c803a320 0x7fc6c803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e3fff700 10 monclient: get_auth_request con 0x7fc6c803a320 auth_method 0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_done global_id 84481 payload 824 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6e409e550 con 0x7fc6e406aac0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 client.36080.objecter ms_handle_connect 0x7fc6e406aac0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 client.36080.objecter resend_mon_ops 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e8acb700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 0x7fc6e41ca550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009670 tx=0x7fc6d8009a30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 client.36080.objecter ms_handle_connect 0x7fc6c803a320 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e3fff700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c803a320 0x7fc6c803c7e0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00090b0 tx=0x7fc6d0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d8012a70 con 0x7fc6e406aac0 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.501+0000 7fc6e17fa700 10 monclient: dump: 2024-04-20T02:31:42.525 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.526 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc6d8012bd0 con 0x7fc6e406aac0 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 msgr2=0x7fc6e41ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 0x7fc6e41ca550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009670 tx=0x7fc6d8009a30 comp rx=0 tx=0).stop 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 msgr2=0x7fc6e41ca550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:42.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.502+0000 7fc6e17fa700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e406aac0 0x7fc6e41ca550 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009670 tx=0x7fc6d8009a30 comp rx=0 tx=0).stop 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.509+0000 7fc6ead2f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.509+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 0x7fc6e41caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6ead2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e406a400 con 0x7fc6e412bad0 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 0x7fc6e41caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient: get_auth_request con 0x7fc6e412bad0 auth_method 0 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:31:42.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 10 monclient(hunting): handle_auth_done global_id 84490 payload 824 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 1 -- 172.21.15.47:0/2716752136 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc6e40a0930 con 0x7fc6e412bad0 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e17fa700 10 client.36080.objecter ms_handle_connect 0x7fc6e412bad0 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e17fa700 10 client.36080.objecter resend_mon_ops 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e8acb700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 0x7fc6e41caa90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8016000 tx=0x7fc6d8005a60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d8022070 con 0x7fc6e412bad0 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e17fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.510+0000 7fc6e17fa700 10 monclient: dump: 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:31:42.530 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9151 (secure 0 0 0) 0x7fc6d801c280 con 0x7fc6e412bad0 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 msgr2=0x7fc6e41caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 0x7fc6e41caa90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8016000 tx=0x7fc6d8005a60 comp rx=0 tx=0).stop 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 msgr2=0x7fc6e41caa90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e17fa700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e412bad0 0x7fc6e41caa90 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8016000 tx=0x7fc6d8005a60 comp rx=0 tx=0).stop 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 104, 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:31:42.532 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.533 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.534 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:31:42.535 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:31:42.536 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.537 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.538 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.543 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.544 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:31:42.544 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:31:42.544 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.544 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.544 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.545 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.546 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.548 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:31:42.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:31:42.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.551 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:31:42.552 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:31:42.553 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:31:42.554 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:31:42.555 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:31:42.555 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:31:42.555 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:31:42.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:31:42.555 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:31:42.555 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 10 librados: watch_flush enter 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 10 librados: watch_flush exit 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 20 client.36080.objecter shutdown clearing up homeless session... 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 10 client.36080.objecter successfully canceled tick 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c803a320 msgr2=0x7fc6c803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c803a320 0x7fc6c803c7e0 secure :-1 s=READY pgs=3355 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00090b0 tx=0x7fc6d0008030 comp rx=0 tx=0).stop 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 10 monclient: shutdown 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:31:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 msgr2=0x7fc6e41ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6e92cc700 1 -- 172.21.15.47:0/2716752136 reap_dead start 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 0x7fc6e41ca010 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00efc0 tx=0x7fc6dc00c3a0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 shutdown_connections 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 --2- 172.21.15.47:0/2716752136 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc6e40a1f10 0x7fc6e41ca010 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 >> 172.21.15.47:0/2716752136 conn(0x7fc6e40a3000 msgr2=0x7fc6e4133e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 shutdown_connections 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 -- 172.21.15.47:0/2716752136 wait complete. 2024-04-20T02:31:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:31:42.511+0000 7fc6ead2f700 1 librados: shutdown 2024-04-20T02:31:42.558 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-04-20T02:31:42.558 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:33:12.559 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-20T02:33:12.560 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-20T02:33:12.560 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T02:33:12.560 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T02:33:12.564 INFO:tasks.ceph.mon.a:Started 2024-04-20T02:33:12.564 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-20T02:33:12.564 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-20T02:33:12.564 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T02:33:12.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T02:33:12.566 INFO:tasks.ceph.mon.b:Started 2024-04-20T02:33:12.566 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-04-20T02:33:12.566 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-04-20T02:33:12.566 INFO:tasks.ceph.mon.g:Restarting daemon 2024-04-20T02:33:12.567 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-04-20T02:33:12.569 INFO:tasks.ceph.mon.g:Started 2024-04-20T02:33:12.569 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:33:12.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:33:12.716 INFO:tasks.ceph.mon.b.smithi047.stderr:2024-04-20T02:33:12.716+0000 7f9bd5aebcc0 -1 mon.b@-1(???).log v0 log_external_backlog rewinding external_log_to from 630 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:33:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.720+0000 7f1fbab8e700 10 monclient: get_monmap_and_config 2024-04-20T02:33:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.720+0000 7f1fbab8e700 10 monclient: build_initial_monmap 2024-04-20T02:33:12.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.720+0000 7f1fbab8e700 10 monclient: monmap: 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:12.722045+0000 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:12.722045+0000 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:12.721 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:12.722 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:12.722 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:12.722 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.721+0000 7f1fbab8e700 1 Processor -- start 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.721+0000 7f1fbab8e700 1 -- start start 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.721+0000 7f1fbab8e700 10 monclient: init 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient: _add_conns ranks=[3,8,0,1,2,6,4,7,9,5,10] 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 0x7f1fb40a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): picked mon.noname-g con 0x7f1fb40a2950 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:12.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1fb409eac0 0x7f1fb40a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): picked mon.noname-e con 0x7f1fb409eac0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1fb40a11f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40751e0 con 0x7f1fb40a11f0 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40a02e0 con 0x7f1fb40a2950 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40a37b0 con 0x7f1fb409eac0 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fbab8e700 10 monclient(hunting): authenticate will time out at 2672.123422s 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb912b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1fb40a11f0 0x7f1fb4134700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:48582/0 (socket says 172.21.15.47:48582) 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb912b700 1 -- 172.21.15.47:0/2088054782 learned_addr learned my addr 172.21.15.47:0/2088054782 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1fb409eac0 0x7f1fb40a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:12.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): get_auth_request con 0x7f1fb409eac0 auth_method 0 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 0x7f1fb40a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 10 monclient(hunting): get_auth_request con 0x7f1fb40a2950 auth_method 0 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 1 -- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 msgr2=0x7f1fb40a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 1 -- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 msgr2=0x7f1fb40a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 0x7f1fb40a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:12.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb892a700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 0x7f1fb40a2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_done global_id 36089 payload 274 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 1 -- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 msgr2=0x7f1fb40a2d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 0x7f1fb40a2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 1 -- 172.21.15.47:0/2088054782 >> v1:172.21.15.47:6789/0 conn(0x7f1fb40a11f0 legacy=0x7f1fb4134700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 1 monclient: found mon.noname-e 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.722+0000 7f1fb3fff700 1 -- 172.21.15.47:0/2088054782 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40a0810 con 0x7f1fb409eac0 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1fb409eac0 0x7f1fb40a3270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009870 tx=0x7f1fa0009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=23367b078eb50e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 1 -- 172.21.15.47:0/2088054782 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fa0011aa0 con 0x7f1fb409eac0 2024-04-20T02:33:12.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: dump: 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:12.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fbab8e700 5 monclient: authenticate success, global_id 36089 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fbab8e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:12.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 1 -- 172.21.15.47:0/2088054782 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0011c00 con 0x7f1fb409eac0 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fbab8e700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fbab8e700 4 set_mon_vals no callback set 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 1 -- 172.21.15.47:0/2088054782 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fa0011d70 con 0x7f1fb409eac0 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.723+0000 7f1fb37fe700 10 monclient: dump: 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:12.739 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 10 monclient: shutdown 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 -- 172.21.15.47:0/2088054782 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1fb409eac0 msgr2=0x7f1fb40a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1fb409eac0 0x7f1fb40a3270 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009870 tx=0x7f1fa0009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 -- 172.21.15.47:0/2088054782 shutdown_connections 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb40a2950 0x7f1fb40a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:12.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/2088054782 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f1fb409eac0 0x7f1fb40a3270 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 -- 172.21.15.47:0/2088054782 >> 172.21.15.47:0/2088054782 conn(0x7f1fb40a7570 msgr2=0x7f1fb40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 -- 172.21.15.47:0/2088054782 shutdown_connections 2024-04-20T02:33:12.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 1 -- 172.21.15.47:0/2088054782 wait complete. 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 10 monclient: build_initial_monmap 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.724+0000 7f1fbab8e700 10 monclient: monmap: 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:12.742 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 librados: starting msgr at 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 librados: starting objecter 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 Processor -- start 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 -- start start 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 librados: setting wanted keys 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 librados: calling monclient init 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 10 monclient: init 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 10 monclient: _add_conns ranks=[3,1,2,4,5,8,0,6,7] 2024-04-20T02:33:12.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.725+0000 7f1fbab8e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 0x7f1fb41b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): picked mon.noname-g con 0x7f1fb409eac0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fb40a11f0 0x7f1fb41b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): picked mon.noname-f con 0x7f1fb40a11f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): picked mon.noname-b con 0x7f1fb40a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb409f260 con 0x7f1fb40a11f0 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41a6cd0 con 0x7f1fb40a2950 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4072410 con 0x7f1fb409eac0 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fbab8e700 10 monclient(hunting): authenticate will time out at 2672.127107s 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:12.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36092/0 (socket says 172.21.15.47:36092) 2024-04-20T02:33:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 -- 172.21.15.47:0/1579507205 learned_addr learned my addr 172.21.15.47:0/1579507205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 10 monclient(hunting): get_auth_request con 0x7f1fb40a2950 auth_method 0 2024-04-20T02:33:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:12.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 0x7f1fb41b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fb40a11f0 0x7f1fb41b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 msgr2=0x7f1fb41c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 msgr2=0x7f1fb41c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): get_auth_request con 0x7f1fb40a11f0 auth_method 0 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:12.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb912b700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 10 monclient(hunting): get_auth_request con 0x7f1fb409eac0 auth_method 0 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 msgr2=0x7f1fb41b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:33:12.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 msgr2=0x7f1fb41b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 0x7f1fb41b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb892a700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 0x7f1fb41b78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient(hunting): handle_auth_done global_id 55702 payload 824 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 msgr2=0x7f1fb41b78b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 0x7f1fb41b78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 msgr2=0x7f1fb41c5a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 monclient: found mon.noname-f 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:12.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:33:12.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.726+0000 7f1fb3fff700 1 -- 172.21.15.47:0/1579507205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00094a0 con 0x7f1fb40a11f0 2024-04-20T02:33:12.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb3fff700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fb40a11f0 0x7f1fb41b7df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1fb409f150 tx=0x7f1fa0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:12.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 client.?.objecter ms_handle_connect 0x7f1fb40a11f0 2024-04-20T02:33:12.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:12.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fa00043e0 con 0x7f1fb40a11f0 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: dump: 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.750 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:12.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: _finish_auth 0 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: _check_auth_tickets 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 5 monclient: authenticate success, global_id 55702 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa001f070 con 0x7f1fb40a11f0 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 10 monclient: _renew_subs 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb41a5d20 con 0x7f1fb40a11f0 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fa00045a0 con 0x7f1fb40a11f0 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb2ffd700 4 set_mon_vals no callback set 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 10 client.55702.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fb17fa700 10 monclient: dump: 2024-04-20T02:33:12.752 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:12.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 10 monclient: _renew_subs 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb41a69f0 con 0x7f1fb40a11f0 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.727+0000 7f1fbab8e700 1 librados: init done 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fbab8e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fbab8e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fbab8e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb41a6e10 con 0x7f1fb40a11f0 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1fa0021030 con 0x7f1fb40a11f0 2024-04-20T02:33:12.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1f9803a2f0 0x7f1f9803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(677..677 src has 266..677) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f1fa004bb30 con 0x7f1fb40a11f0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 10 client.55702.objecter ms_dispatch 0x7f1fb4001550 osd_map(677..677 src has 266..677) v4 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 3 client.55702.objecter handle_osd_map got epochs [677,677] > 0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 3 client.55702.objecter handle_osd_map decoding full epoch 677 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb892a700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1f9803a2f0 0x7f1f9803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb17fa700 20 client.55702.objecter dump_active .. 0 homeless 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.728+0000 7f1fb892a700 10 monclient: get_auth_request con 0x7f1f9803a2f0 auth_method 0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.729+0000 7f1fb17fa700 10 client.55702.objecter ms_handle_connect 0x7f1f9803a2f0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.729+0000 7f1fb892a700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1f9803a2f0 0x7f1f9803c7b0 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80091e0 tx=0x7f1fa8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.733+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f1fa000b830 con 0x7f1fb40a11f0 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.733+0000 7f1fb17fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:12.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:12.733+0000 7f1fb17fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:13.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.048+0000 7f1fbab8e700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:33:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.048+0000 7f1fbab8e700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.048+0000 7f1fbab8e700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.048+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1fb41b8330 con 0x7f1fb40a11f0 2024-04-20T02:33:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.049+0000 7f1fb17fa700 1 -- 172.21.15.47:0/1579507205 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3159 (secure 0 0 0) 0x7f1fa0015080 con 0x7f1fb40a11f0 2024-04-20T02:33:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.049+0000 7f1fb17fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:13.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.049+0000 7f1fb17fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:13.050 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":104,"quorum":[1,4,5,6,7,8],"quorum_names":["f","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":95,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 10 librados: watch_flush enter 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 10 librados: watch_flush exit 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 20 client.55702.objecter shutdown clearing up homeless session... 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 10 client.55702.objecter successfully canceled tick 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1f9803a2f0 msgr2=0x7f1f9803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1f9803a2f0 0x7f1f9803c7b0 secure :-1 s=READY pgs=3361 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80091e0 tx=0x7f1fa8008040 comp rx=0 tx=0).stop 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 10 monclient: shutdown 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fb40a11f0 msgr2=0x7f1fb41b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:13.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fb40a11f0 0x7f1fb41b7df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1fb409f150 tx=0x7f1fa0004750 comp rx=0 tx=0).stop 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 shutdown_connections 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1fb40a2950 0x7f1fb41c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1fb409eac0 0x7f1fb41b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f1f9803a2f0 0x7f1f9803c7b0 unknown :-1 s=CLOSED pgs=3361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 --2- 172.21.15.47:0/1579507205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fb40a11f0 0x7f1fb41b7df0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 >> 172.21.15.47:0/1579507205 conn(0x7f1fb40a7570 msgr2=0x7f1fb41ba600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.051+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 shutdown_connections 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.052+0000 7f1fbab8e700 1 -- 172.21.15.47:0/1579507205 wait complete. 2024-04-20T02:33:13.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:13.052+0000 7f1fbab8e700 1 librados: shutdown 2024-04-20T02:33:16.066 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:33:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.211+0000 7f92377a7700 10 monclient: get_monmap_and_config 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.211+0000 7f92377a7700 10 monclient: build_initial_monmap 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.211+0000 7f92377a7700 10 monclient: monmap: 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:16.212016+0000 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:16.212016+0000 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:16.211 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:16.212 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:16.212 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:16.212 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:16.212 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:16.212 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:16.212 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:16.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.212+0000 7f92377a7700 1 Processor -- start 2024-04-20T02:33:16.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.212+0000 7f92377a7700 1 -- start start 2024-04-20T02:33:16.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.212+0000 7f92377a7700 10 monclient: init 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient: _add_conns ranks=[10,5,9,7,6,4,3,0,8,1,2] 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): picked mon.noname-f con 0x7f9230066610 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): picked mon.noname-h con 0x7f92300a00e0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): picked mon.noname-a con 0x7f92300a0560 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:16.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9230139760 con 0x7f92300a0560 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923009dc30 con 0x7f9230066610 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92300669f0 con 0x7f92300a00e0 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f92377a7700 10 monclient(hunting): authenticate will time out at 2675.613263s 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42870/0 (socket says 172.21.15.47:42870) 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 learned_addr learned my addr 172.21.15.47:0/277944121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 10 monclient(hunting): get_auth_request con 0x7f92300a0560 auth_method 0 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:16.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 10 monclient(hunting): get_auth_request con 0x7f9230066610 auth_method 0 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f92300a00e0 auth_method 0 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:16.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.213+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 10 monclient(hunting): get_auth_request con 0x7f92300a0560 auth_method 0 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:16.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f92300a00e0 auth_method 0 2024-04-20T02:33:16.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 10 monclient(hunting): get_auth_request con 0x7f9230066610 auth_method 0 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:16.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:16.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.414+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 10 monclient(hunting): get_auth_request con 0x7f92300a0560 auth_method 0 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:16.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 10 monclient(hunting): get_auth_request con 0x7f9230066610 auth_method 0 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f92300a00e0 auth_method 0 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:16.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:16.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:16.815+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 10 monclient(hunting): get_auth_request con 0x7f92300a0560 auth_method 0 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:17.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f92300a00e0 auth_method 0 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 10 monclient(hunting): get_auth_request con 0x7f9230066610 auth_method 0 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:17.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:17.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:17.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:17.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:17.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.616+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:17.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:17.617+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:19.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): tick 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 monclient(hunting): continuing hunt 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f92300a00e0 0x7f9230098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 msgr2=0x7f92301389e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.179:3300/0 conn(0x7f9230066610 0x7f92301389e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 -- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 msgr2=0x7f9230099020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 --2- 172.21.15.47:0/277944121 >> v2:172.21.15.47:3300/0 conn(0x7f92300a0560 0x7f9230099020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient: _add_conns ranks=[1,8,0,3,6,7,4,2] 2024-04-20T02:33:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f921c003850 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:19.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): picked mon.noname-e con 0x7f921c007480 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:19.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f921c009f90 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 -- 172.21.15.47:0/277944121 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92300751e0 con 0x7f921c009f90 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 -- 172.21.15.47:0/277944121 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c00ed70 con 0x7f921c003850 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 1 -- 172.21.15.47:0/277944121 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9230139760 con 0x7f921c007480 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9226ffd700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f921c007480 auth_method 0 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 msgr2=0x7f921c0098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 msgr2=0x7f921c0098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:19.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.213+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:19.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f921c007480 auth_method 0 2024-04-20T02:33:19.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 msgr2=0x7f921c0098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 msgr2=0x7f921c0098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:19.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.414+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298151271 0 0) 0x7f921c00ed70 con 0x7f921c003850 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient(hunting): _negotiate 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient(hunting): my global_id is 55711 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92140035b0 con 0x7f921c003850 2024-04-20T02:33:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529549580 0 0) 0x7f92140035b0 con 0x7f921c003850 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 msgr2=0x7f921c0098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f921c007480 0x7f921c0098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 >> v1:172.21.15.47:6789/0 conn(0x7f921c009f90 legacy=0x7f921c00c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c010070 con 0x7f921c003850 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: _finish_auth 0 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9235d44700 1 -- 172.21.15.47:0/277944121 reap_dead start 2024-04-20T02:33:19.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7f922c00e800 con 0x7f921c003850 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.503+0000 7f9227fff700 10 monclient: dump: 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:19.507 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 >> v1:172.21.15.179:6789/0 conn(0x7f921c003850 legacy=0x7f921c005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient: _add_conns ranks=[8,2,0,4,6,3,1,7,5] 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214004250 0x7f9214004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): picked mon.e con 0x7f9214004250 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9214004e60 0x7f92140052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): picked mon.b con 0x7f9214004e60 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214009d20 0x7f921400c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): picked mon.a con 0x7f9214009d20 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9230139760 con 0x7f9214009d20 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921400ca90 con 0x7f9214004e60 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f921400cc00 con 0x7f9214004250 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9227fff700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9214004e60 0x7f92140052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214004250 0x7f9214004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f9214004e60 auth_method 0 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): get_auth_request con 0x7f9214004250 auth_method 0 2024-04-20T02:33:19.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214009d20 0x7f921400c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): get_auth_request con 0x7f9214009d20 auth_method 0 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:19.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient(hunting): handle_auth_done global_id 55711 payload 293 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214004250 msgr2=0x7f9214004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214004250 0x7f9214004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214009d20 msgr2=0x7f921400c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214009d20 0x7f921400c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient: _finish_hunting 0 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 monclient: found mon.b 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:19.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 1 -- 172.21.15.47:0/277944121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140035b0 con 0x7f9214004e60 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient: _finish_auth 0 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 10 monclient: _check_auth_tickets 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9234d42700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f92377a7700 5 monclient: authenticate success, global_id 55711 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f92377a7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235d44700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214009d20 0x7f921400c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.504+0000 7f9235543700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214004250 0x7f9214004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:19.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.505+0000 7f9234d42700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9214004e60 0x7f92140052e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f92200046e0 tx=0x7f9214005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c290123ff721bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:19.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92200044a0 con 0x7f9214004e60 2024-04-20T02:33:19.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:19.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:19.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: dump: 2024-04-20T02:33:19.519 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:19.519 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:19.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922000c040 con 0x7f9214004e60 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 1 -- 172.21.15.47:0/277944121 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9220018df0 con 0x7f9214004e60 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 4 set_mon_vals no callback set 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f9227fff700 10 monclient: dump: 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 10 monclient: shutdown 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 1 -- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9214004e60 msgr2=0x7f92140052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.512+0000 7f92377a7700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9214004e60 0x7f92140052e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f92200046e0 tx=0x7f9214005880 comp rx=0 tx=0).stop 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 -- 172.21.15.47:0/277944121 shutdown_connections 2024-04-20T02:33:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9214004e60 0x7f92140052e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214004250 0x7f9214004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 --2- 172.21.15.47:0/277944121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214009d20 0x7f921400c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 -- 172.21.15.47:0/277944121 >> 172.21.15.47:0/277944121 conn(0x7f92300a6720 msgr2=0x7f92300a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 -- 172.21.15.47:0/277944121 shutdown_connections 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.513+0000 7f92377a7700 1 -- 172.21.15.47:0/277944121 wait complete. 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.514+0000 7f92377a7700 10 monclient: build_initial_monmap 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.514+0000 7f92377a7700 10 monclient: monmap: 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.514+0000 7f92377a7700 1 librados: starting msgr at 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.514+0000 7f92377a7700 1 librados: starting objecter 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.515+0000 7f92377a7700 1 Processor -- start 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.515+0000 7f92377a7700 1 -- start start 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.515+0000 7f92377a7700 1 librados: setting wanted keys 2024-04-20T02:33:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.515+0000 7f92377a7700 1 librados: calling monclient init 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.515+0000 7f92377a7700 10 monclient: init 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient: _add_conns ranks=[0,3,8,4,7,2,1,6,5] 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): picked mon.noname-a con 0x7f9214004250 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 0x7f9230072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): picked mon.noname-g con 0x7f9214004e60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214009d20 0x7f9230072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): picked mon.noname-e con 0x7f9214009d20 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9230072e80 con 0x7f9214004250 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92301d0990 con 0x7f9214004e60 2024-04-20T02:33:19.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f92301c37f0 con 0x7f9214009d20 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f92377a7700 10 monclient(hunting): authenticate will time out at 2678.916303s 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42930/0 (socket says 172.21.15.47:42930) 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 -- 172.21.15.47:0/740484539 learned_addr learned my addr 172.21.15.47:0/740484539 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 10 monclient(hunting): get_auth_request con 0x7f9214004250 auth_method 0 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 msgr2=0x7f92301d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 msgr2=0x7f92301d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235d44700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214009d20 0x7f9230072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235d44700 10 monclient(hunting): get_auth_request con 0x7f9214009d20 auth_method 0 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235d44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235d44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235d44700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9235543700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9234d42700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 0x7f9230072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.516+0000 7f9234d42700 10 monclient(hunting): get_auth_request con 0x7f9214004e60 auth_method 0 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:19.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 msgr2=0x7f9230072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 msgr2=0x7f9230072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 0x7f9230072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9234d42700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 0x7f9230072400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 10 monclient(hunting): handle_auth_done global_id 36098 payload 824 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 msgr2=0x7f9230072400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 0x7f9230072400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 msgr2=0x7f92301d00e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 10 monclient: _finish_hunting 0 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 monclient: found mon.noname-e 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 -- 172.21.15.47:0/740484539 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92301c4050 con 0x7f9214009d20 2024-04-20T02:33:19.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9235d44700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214009d20 0x7f9230072940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9230066c40 tx=0x7f9228014f60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 client.?.objecter ms_handle_connect 0x7f9214009d20 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92280127a0 con 0x7f9214009d20 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 monclient: dump: 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:19.529 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922800f070 con 0x7f9214009d20 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f92377a7700 5 monclient: authenticate success, global_id 36098 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f92377a7700 10 monclient: _renew_subs 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f92377a7700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:19.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92301c4330 con 0x7f9214009d20 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.517+0000 7f9225ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9228012df0 con 0x7f9214009d20 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f9225ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f9225ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f9225ffb700 10 monclient: dump: 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:19.531 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:19.532 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f92277fe700 4 set_mon_vals no callback set 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f92377a7700 10 client.36098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:19.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f92377a7700 10 monclient: _renew_subs 2024-04-20T02:33:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f92377a7700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92301c30f0 con 0x7f9214009d20 2024-04-20T02:33:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.518+0000 7f92377a7700 1 librados: init done 2024-04-20T02:33:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f92377a7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9228012980 con 0x7f9214009d20 2024-04-20T02:33:19.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f92377a7700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f92377a7700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92301c4730 con 0x7f9214009d20 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f9225ffb700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f921c03b070 0x7f921c03d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(677..677 src has 266..677) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f922804aa10 con 0x7f9214009d20 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f9225ffb700 10 client.36098.objecter ms_dispatch 0x7f9230001550 osd_map(677..677 src has 266..677) v4 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f9225ffb700 3 client.36098.objecter handle_osd_map got epochs [677,677] > 0 2024-04-20T02:33:19.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.519+0000 7f9225ffb700 3 client.36098.objecter handle_osd_map decoding full epoch 677 2024-04-20T02:33:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.520+0000 7f9225ffb700 20 client.36098.objecter dump_active .. 0 homeless 2024-04-20T02:33:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.520+0000 7f9235543700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f921c03b070 0x7f921c03d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.520+0000 7f9235543700 10 monclient: get_auth_request con 0x7f921c03b070 auth_method 0 2024-04-20T02:33:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.520+0000 7f9235543700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f921c03b070 0x7f921c03d530 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 crypto rx=0x7f922c00eae0 tx=0x7f922c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:19.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.520+0000 7f9225ffb700 10 client.36098.objecter ms_handle_connect 0x7f921c03b070 2024-04-20T02:33:19.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.525+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f9228012c30 con 0x7f9214009d20 2024-04-20T02:33:19.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.525+0000 7f9225ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:19.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.525+0000 7f9225ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.837+0000 7f92377a7700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.837+0000 7f92377a7700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.837+0000 7f92377a7700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.837+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f92301c2630 con 0x7f9214009d20 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.838+0000 7f9225ffb700 1 -- 172.21.15.47:0/740484539 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3164 (secure 0 0 0) 0x7f9228016e60 con 0x7f9214009d20 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.838+0000 7f9225ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.838+0000 7f9225ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:19.837 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":106,"quorum":[1,2,4,5,6,7,8],"quorum_names":["f","b","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 10 librados: watch_flush enter 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 10 librados: watch_flush exit 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 20 client.36098.objecter shutdown clearing up homeless session... 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 10 client.36098.objecter successfully canceled tick 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f921c03b070 msgr2=0x7f921c03d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f921c03b070 0x7f921c03d530 secure :-1 s=READY pgs=3363 cs=0 l=1 rev1=1 crypto rx=0x7f922c00eae0 tx=0x7f922c008040 comp rx=0 tx=0).stop 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 10 monclient: shutdown 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214009d20 msgr2=0x7f9230072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214009d20 0x7f9230072940 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9230066c40 tx=0x7f9228014f60 comp rx=0 tx=0).stop 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 shutdown_connections 2024-04-20T02:33:19.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f921c03b070 0x7f921c03d530 unknown :-1 s=CLOSED pgs=3363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9214009d20 0x7f9230072940 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9214004250 0x7f92301d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 --2- 172.21.15.47:0/740484539 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9214004e60 0x7f9230072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 >> 172.21.15.47:0/740484539 conn(0x7f92300a6720 msgr2=0x7f9230098ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 shutdown_connections 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 -- 172.21.15.47:0/740484539 wait complete. 2024-04-20T02:33:19.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:19.840+0000 7f92377a7700 1 librados: shutdown 2024-04-20T02:33:22.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.000+0000 7ff9f0fee700 10 monclient: get_monmap_and_config 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.000+0000 7ff9f0fee700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.000+0000 7ff9f0fee700 10 monclient: monmap: 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:23.001436+0000 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:23.001436+0000 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:23.001 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.000+0000 7ff9f0fee700 1 Processor -- start 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 -- start start 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient: init 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient: _add_conns ranks=[8,2,7,6,9,4,1,10,0,3,5] 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 0x7ff9ec1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): picked mon.noname-e con 0x7ff9ec066610 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff9ec135bf0 0x7ff9ec135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): picked mon.noname-b con 0x7ff9ec135bf0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff9ec0a11b0 0x7ff9ec0a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): picked mon.noname-i con 0x7ff9ec0a11b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec136510 con 0x7ff9ec135bf0 2024-04-20T02:33:23.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0669f0 con 0x7ff9ec0a11b0 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec09a7f0 con 0x7ff9ec066610 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9f0fee700 10 monclient(hunting): authenticate will time out at 2682.402759s 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 0x7ff9ec1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9e9d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff9ec135bf0 0x7ff9ec135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 0x7ff9ec1356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:50216/0 (socket says 172.21.15.47:50216) 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 1 -- 172.21.15.47:0/3524404866 learned_addr learned my addr 172.21.15.47:0/3524404866 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 10 monclient(hunting): get_auth_request con 0x7ff9ec066610 auth_method 0 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9ea59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9e9d9b700 10 monclient(hunting): get_auth_request con 0x7ff9ec135bf0 auth_method 0 2024-04-20T02:33:23.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9e9d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9e9d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.001+0000 7ff9e9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e9d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e9d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e9d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ead9d700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff9ec0a11b0 0x7ff9ec0a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ead9d700 10 monclient(hunting): get_auth_request con 0x7ff9ec0a11b0 auth_method 0 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ead9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ead9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ead9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_done global_id 36107 payload 274 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 -- 172.21.15.47:0/3524404866 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff9ec0a11b0 msgr2=0x7ff9ec0a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff9ec0a11b0 0x7ff9ec0a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 -- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff9ec135bf0 msgr2=0x7ff9ec135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff9ec135bf0 0x7ff9ec135fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 monclient: found mon.noname-e 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 -- 172.21.15.47:0/3524404866 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0751e0 con 0x7ff9ec066610 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e9d9b700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff9ec135bf0 0x7ff9ec135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9ea59c700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 0x7ff9ec1356b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009920 tx=0x7ff9dc009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a05e8a71bc2ecaff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 1 -- 172.21.15.47:0/3524404866 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9dc011aa0 con 0x7ff9ec066610 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: dump: 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.015 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9f0fee700 5 monclient: authenticate success, global_id 36107 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 1 -- 172.21.15.47:0/3524404866 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc011c00 con 0x7ff9ec066610 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9f0fee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9f0fee700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 1 -- 172.21.15.47:0/3524404866 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9dc011d70 con 0x7ff9ec066610 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9f0fee700 4 set_mon_vals no callback set 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.002+0000 7ff9e959a700 10 monclient: dump: 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.017 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 10 monclient: shutdown 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 -- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 msgr2=0x7ff9ec1356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 0x7ff9ec1356b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009920 tx=0x7ff9dc009c30 comp rx=0 tx=0).stop 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 -- 172.21.15.47:0/3524404866 shutdown_connections 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff9ec0a11b0 0x7ff9ec0a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff9ec066610 0x7ff9ec1356b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/3524404866 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff9ec135bf0 0x7ff9ec135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 -- 172.21.15.47:0/3524404866 >> 172.21.15.47:0/3524404866 conn(0x7ff9ec0a6650 msgr2=0x7ff9ec0a6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 -- 172.21.15.47:0/3524404866 shutdown_connections 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.003+0000 7ff9f0fee700 1 -- 172.21.15.47:0/3524404866 wait complete. 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 10 monclient: monmap: 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:23.019 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 1 librados: starting msgr at 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 1 librados: starting objecter 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 1 Processor -- start 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 1 -- start start 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 1 librados: setting wanted keys 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 1 librados: calling monclient init 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.004+0000 7ff9f0fee700 10 monclient: init 2024-04-20T02:33:23.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient: _add_conns ranks=[6,3,1,8,4,0,2,7,5] 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 0x7ff9ec1ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): picked mon.noname-d con 0x7ff9ec066610 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9ec0a11b0 0x7ff9ec1caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): picked mon.noname-g con 0x7ff9ec0a11b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9ec135bf0 0x7ff9ec12eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): picked mon.noname-f con 0x7ff9ec135bf0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec1347d0 con 0x7ff9ec135bf0 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec132570 con 0x7ff9ec0a11b0 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0a0020 con 0x7ff9ec066610 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient(hunting): authenticate will time out at 2682.406107s 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 0x7ff9ec1ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 0x7ff9ec1ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:39898/0 (socket says 172.21.15.47:39898) 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 -- 172.21.15.47:0/113509416 learned_addr learned my addr 172.21.15.47:0/113509416 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): get_auth_request con 0x7ff9ec066610 auth_method 0 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9ec0a11b0 0x7ff9ec1caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9ec135bf0 0x7ff9ec12eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): get_auth_request con 0x7ff9ec135bf0 auth_method 0 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): get_auth_request con 0x7ff9ec0a11b0 auth_method 0 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ead9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e9d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient(hunting): handle_auth_done global_id 55311 payload 824 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 -- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9ec0a11b0 msgr2=0x7ff9ec1caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9ec0a11b0 0x7ff9ec1caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 -- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9ec135bf0 msgr2=0x7ff9ec12eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9ec135bf0 0x7ff9ec12eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 monclient: found mon.noname-d 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 -- 172.21.15.47:0/113509416 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00094a0 con 0x7ff9ec066610 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9ea59c700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 0x7ff9ec1ca760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 client.?.objecter ms_handle_connect 0x7ff9ec066610 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9dc0036a0 con 0x7ff9ec066610 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: dump: 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.024 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc01f070 con 0x7ff9ec066610 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff9dc003c70 con 0x7ff9ec066610 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9e8d99700 4 set_mon_vals no callback set 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9daffd700 10 monclient: dump: 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.026 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 5 monclient: authenticate success, global_id 55311 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.005+0000 7ff9f0fee700 10 monclient: _renew_subs 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc009550 con 0x7ff9ec066610 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 10 client.55311.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 10 monclient: _renew_subs 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec09fdf0 con 0x7ff9ec066610 2024-04-20T02:33:23.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9f0fee700 1 librados: init done 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.006+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff9dc021030 con 0x7ff9ec066610 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9cc03a2f0 0x7ff9cc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(681..681 src has 266..681) v4 ==== 6661+0+0 (secure 0 0 0) 0x7ff9dc04bb50 con 0x7ff9ec066610 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 10 client.55311.objecter ms_dispatch 0x7ff9ec001550 osd_map(681..681 src has 266..681) v4 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 3 client.55311.objecter handle_osd_map got epochs [681,681] > 0 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 3 client.55311.objecter handle_osd_map decoding full epoch 681 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9f0fee700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9f0fee700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9f0fee700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec130760 con 0x7ff9ec066610 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 20 client.55311.objecter dump_active .. 0 homeless 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9e9d9b700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9cc03a2f0 0x7ff9cc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9e9d9b700 10 monclient: get_auth_request con 0x7ff9cc03a2f0 auth_method 0 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9e9d9b700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9cc03a2f0 0x7ff9cc03c7b0 secure :-1 s=READY pgs=3364 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec130380 tx=0x7ff9e0009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.007+0000 7ff9daffd700 10 client.55311.objecter ms_handle_connect 0x7ff9cc03a2f0 2024-04-20T02:33:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.012+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ff9dc003800 con 0x7ff9ec066610 2024-04-20T02:33:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.012+0000 7ff9daffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.012+0000 7ff9daffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.321+0000 7ff9f0fee700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.321+0000 7ff9f0fee700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.321+0000 7ff9f0fee700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.321+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff9ec1327b0 con 0x7ff9ec066610 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.322+0000 7ff9daffd700 1 -- 172.21.15.47:0/113509416 <== mon.6 v2:172.21.15.47:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7ff9dc00bc10 con 0x7ff9ec066610 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.322+0000 7ff9daffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.322+0000 7ff9daffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:23.322 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":108,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:33:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 10 librados: watch_flush enter 2024-04-20T02:33:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 10 librados: watch_flush exit 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 20 client.55311.objecter shutdown clearing up homeless session... 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 10 client.55311.objecter successfully canceled tick 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9cc03a2f0 msgr2=0x7ff9cc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9cc03a2f0 0x7ff9cc03c7b0 secure :-1 s=READY pgs=3364 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec130380 tx=0x7ff9e0009210 comp rx=0 tx=0).stop 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 10 monclient: shutdown 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 msgr2=0x7ff9ec1ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 0x7ff9ec1ca760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 shutdown_connections 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff9cc03a2f0 0x7ff9cc03c7b0 unknown :-1 s=CLOSED pgs=3364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9ec135bf0 0x7ff9ec12eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff9ec066610 0x7ff9ec1ca760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 --2- 172.21.15.47:0/113509416 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9ec0a11b0 0x7ff9ec1caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 >> 172.21.15.47:0/113509416 conn(0x7ff9ec0a6650 msgr2=0x7ff9ec099810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 shutdown_connections 2024-04-20T02:33:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 -- 172.21.15.47:0/113509416 wait complete. 2024-04-20T02:33:23.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.324+0000 7ff9f0fee700 1 librados: shutdown 2024-04-20T02:33:23.339 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:33:23.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:33:23.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:33:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.499+0000 7fb882a33700 10 monclient: get_monmap_and_config 2024-04-20T02:33:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.499+0000 7fb882a33700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.499+0000 7fb882a33700 10 monclient: monmap: 2024-04-20T02:33:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:23.500723+0000 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:23.500723+0000 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:23.500 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:23.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.500+0000 7fb882a33700 1 Processor -- start 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.500+0000 7fb882a33700 1 -- start start 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.500+0000 7fb882a33700 10 monclient: init 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient: _add_conns ranks=[7,3,2,9,6,5,1,0,10,4,8] 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c071730 0x7fb87c071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): picked mon.noname-i con 0x7fb87c071730 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c06f020 0x7fb87c0a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): picked mon.noname-g con 0x7fb87c06f020 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb87c0a2db0 0x7fb87c0a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): picked mon.noname-b con 0x7fb87c0a2db0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c09fea0 con 0x7fb87c0a2db0 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c09e7b0 con 0x7fb87c06f020 2024-04-20T02:33:23.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c06d190 con 0x7fb87c071730 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb882a33700 10 monclient(hunting): authenticate will time out at 2682.902243s 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb87c0a2db0 0x7fb87c0a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb87c0a2db0 0x7fb87c0a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43216/0 (socket says 172.21.15.47:43216) 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 1 -- 172.21.15.47:0/3565869869 learned_addr learned my addr 172.21.15.47:0/3565869869 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 10 monclient(hunting): get_auth_request con 0x7fb87c0a2db0 auth_method 0 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb880fd0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c06f020 0x7fb87c0a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c071730 0x7fb87c071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): get_auth_request con 0x7fb87c06f020 auth_method 0 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): get_auth_request con 0x7fb87c071730 auth_method 0 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_done global_id 104115 payload 274 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 -- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c071730 msgr2=0x7fb87c071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c071730 0x7fb87c071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 -- 172.21.15.47:0/3565869869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb87c0a2db0 msgr2=0x7fb87c0a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb87c0a2db0 0x7fb87c0a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 monclient: found mon.noname-g 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.501+0000 7fb87b7fe700 1 -- 172.21.15.47:0/3565869869 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0751e0 con 0x7fb87c06f020 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c06f020 0x7fb87c0a2870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb870009870 tx=0x7fb870009b80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=dfa4ce5f7b41fbff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 1 -- 172.21.15.47:0/3565869869 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb870011aa0 con 0x7fb87c06f020 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: dump: 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb882a33700 5 monclient: authenticate success, global_id 104115 2024-04-20T02:33:23.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb882a33700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 1 -- 172.21.15.47:0/3565869869 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870011c40 con 0x7fb87c06f020 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb882a33700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 1 -- 172.21.15.47:0/3565869869 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb870011db0 con 0x7fb87c06f020 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb882a33700 4 set_mon_vals no callback set 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.502+0000 7fb87affd700 10 monclient: dump: 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.510 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.503+0000 7fb882a33700 10 monclient: shutdown 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.503+0000 7fb882a33700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.503+0000 7fb882a33700 1 -- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c06f020 msgr2=0x7fb87c0a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.503+0000 7fb882a33700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c06f020 0x7fb87c0a2870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb870009870 tx=0x7fb870009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.504+0000 7fb882a33700 1 -- 172.21.15.47:0/3565869869 shutdown_connections 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.504+0000 7fb882a33700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c071730 0x7fb87c071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.504+0000 7fb882a33700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb87c0a2db0 0x7fb87c0a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.504+0000 7fb882a33700 1 --2- 172.21.15.47:0/3565869869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c06f020 0x7fb87c0a2870 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb870009870 tx=0x7fb870009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:23.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.504+0000 7fb882a33700 1 -- 172.21.15.47:0/3565869869 >> 172.21.15.47:0/3565869869 conn(0x7fb87c0a3f90 msgr2=0x7fb87c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.504+0000 7fb882a33700 1 -- 172.21.15.47:0/3565869869 shutdown_connections 2024-04-20T02:33:23.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.505+0000 7fb882a33700 1 -- 172.21.15.47:0/3565869869 wait complete. 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.505+0000 7fb882a33700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.505+0000 7fb882a33700 10 monclient: monmap: 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:23.516 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 1 librados: starting msgr at 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 1 librados: starting objecter 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 1 Processor -- start 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 1 -- start start 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 1 librados: setting wanted keys 2024-04-20T02:33:23.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 1 librados: calling monclient init 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.506+0000 7fb882a33700 10 monclient: init 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient: _add_conns ranks=[3,1,7,5,8,0,2,6,4] 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c071730 0x7fb87c09c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): picked mon.noname-g con 0x7fb87c071730 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb87c0a2db0 0x7fb87c137f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): picked mon.noname-f con 0x7fb87c0a2db0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c1384a0 0x7fb87c138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): picked mon.noname-i con 0x7fb87c1384a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c072410 con 0x7fb87c0a2db0 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c09f1b0 con 0x7fb87c071730 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c139e20 con 0x7fb87c1384a0 2024-04-20T02:33:23.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.507+0000 7fb882a33700 10 monclient(hunting): authenticate will time out at 2682.908884s 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c1384a0 0x7fb87c138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c1384a0 0x7fb87c138900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:50664/0 (socket says 172.21.15.47:50664) 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 -- 172.21.15.47:0/2034428973 learned_addr learned my addr 172.21.15.47:0/2034428973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): get_auth_request con 0x7fb87c1384a0 auth_method 0 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient(hunting): handle_auth_done global_id 84517 payload 824 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c071730 msgr2=0x7fb87c09c900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb87c071730 0x7fb87c09c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb87c0a2db0 msgr2=0x7fb87c137f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb87c0a2db0 0x7fb87c137f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 monclient: found mon.noname-i 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.508+0000 7fb880fd0700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700094a0 con 0x7fb87c1384a0 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb880fd0700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c1384a0 0x7fb87c138900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb874007a70 tx=0x7fb874007e30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 client.?.objecter ms_handle_connect 0x7fb87c1384a0 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb874017070 con 0x7fb87c1384a0 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: dump: 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.520 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400f040 con 0x7fb87c1384a0 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb87400dd00 con 0x7fb87c1384a0 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb878ff9700 10 monclient: dump: 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.522 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 5 monclient: authenticate success, global_id 84517 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 10 monclient: _renew_subs 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c13a8a0 con 0x7fb87c1384a0 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 10 client.84517.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 10 monclient: _renew_subs 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c133290 con 0x7fb87c1384a0 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.509+0000 7fb882a33700 1 librados: init done 2024-04-20T02:33:23.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb87400d890 con 0x7fb87c1384a0 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb86803a320 0x7fb86803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(681..681 src has 266..681) v4 ==== 6661+0+0 (secure 0 0 0) 0x7fb87404d500 con 0x7fb87c1384a0 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 10 client.84517.objecter ms_dispatch 0x7fb87c001550 osd_map(681..681 src has 266..681) v4 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 3 client.84517.objecter handle_osd_map got epochs [681,681] > 0 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 3 client.84517.objecter handle_osd_map decoding full epoch 681 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.511+0000 7fb878ff9700 20 client.84517.objecter dump_active .. 0 homeless 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.512+0000 7fb87bfff700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb86803a320 0x7fb86803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.512+0000 7fb87bfff700 10 monclient: get_auth_request con 0x7fb86803a320 auth_method 0 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.512+0000 7fb878ff9700 10 client.84517.objecter ms_handle_connect 0x7fb86803a320 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.512+0000 7fb87bfff700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb86803a320 0x7fb86803c7e0 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009a70 tx=0x7fb86c009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.515+0000 7fb882a33700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.515+0000 7fb882a33700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 0x7fb87c066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.515+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.515+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c068280 con 0x7fb87c06aac0 2024-04-20T02:33:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.517+0000 7fb87a7fc700 4 set_mon_vals no callback set 2024-04-20T02:33:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.518+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 0x7fb87c066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.518+0000 7fb87b7fe700 10 monclient: get_auth_request con 0x7fb87c06aac0 auth_method 0 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.518+0000 7fb87b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.518+0000 7fb87b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.518+0000 7fb87b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.523+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.523+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.523+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.523+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_done global_id 104103 payload 824 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.523+0000 7fb87b7fe700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb87c09eb60 con 0x7fb87c06aac0 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.524+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 0x7fb87c066ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0a49c0 tx=0x7fb870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.524+0000 7fb878ff9700 10 client.84517.objecter ms_handle_connect 0x7fb87c06aac0 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.524+0000 7fb878ff9700 10 client.84517.objecter resend_mon_ops 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb8700036a0 con 0x7fb87c06aac0 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 10 monclient: dump: 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.527 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb87001f070 con 0x7fb87c06aac0 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 msgr2=0x7fb87c066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 0x7fb87c066ce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0a49c0 tx=0x7fb870004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 msgr2=0x7fb87c066ce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.525+0000 7fb878ff9700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c06aac0 0x7fb87c066ce0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0a49c0 tx=0x7fb870004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb882a33700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb882a33700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 0x7fb87c067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb882a33700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c06a400 con 0x7fb87c067220 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 0x7fb87c067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb87b7fe700 10 monclient: get_auth_request con 0x7fb87c067220 auth_method 0 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb87b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb87b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.542+0000 7fb87b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb87b7fe700 10 monclient(hunting): handle_auth_done global_id 104112 payload 824 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb87b7fe700 1 -- 172.21.15.47:0/2034428973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb87c0a1120 con 0x7fb87c067220 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 10 client.84517.objecter ms_handle_connect 0x7fb87c067220 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 10 client.84517.objecter resend_mon_ops 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb87b7fe700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 0x7fb87c067630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb870009790 tx=0x7fb870009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb870003cd0 con 0x7fb87c067220 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 10 monclient: dump: 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.545 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.546 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.543+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7fb870003e30 con 0x7fb87c067220 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb878ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb878ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 msgr2=0x7fb87c067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb878ff9700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 0x7fb87c067630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb870009790 tx=0x7fb870009210 comp rx=0 tx=0).stop 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb878ff9700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 msgr2=0x7fb87c067630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb878ff9700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb87c067220 0x7fb87c067630 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb870009790 tx=0x7fb870009210 comp rx=0 tx=0).stop 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:23.548 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.549 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.550 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:23.551 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.552 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.553 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:23.554 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.555 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:23.556 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:23.559 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.561 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.563 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.569 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:23.571 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.571 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.571 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:23.571 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 10 librados: watch_flush enter 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 10 librados: watch_flush exit 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 20 client.84517.objecter shutdown clearing up homeless session... 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 10 client.84517.objecter successfully canceled tick 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb86803a320 msgr2=0x7fb86803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb86803a320 0x7fb86803c7e0 secure :-1 s=READY pgs=3365 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009a70 tx=0x7fb86c009210 comp rx=0 tx=0).stop 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 10 monclient: shutdown 2024-04-20T02:33:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c1384a0 msgr2=0x7fb87c138900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 --2- 172.21.15.47:0/2034428973 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb87c1384a0 0x7fb87c138900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb874007a70 tx=0x7fb874007e30 comp rx=0 tx=0).stop 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb880fd0700 1 -- 172.21.15.47:0/2034428973 reap_dead start 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb880fd0700 1 -- 172.21.15.47:0/2034428973 reap_dead start 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 shutdown_connections 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 >> 172.21.15.47:0/2034428973 conn(0x7fb87c0a3f90 msgr2=0x7fb87c0a4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 shutdown_connections 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 -- 172.21.15.47:0/2034428973 wait complete. 2024-04-20T02:33:23.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.544+0000 7fb882a33700 1 librados: shutdown 2024-04-20T02:33:23.574 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:23.575 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:33:23.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:33:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.712+0000 7f4b445e0700 10 monclient: get_monmap_and_config 2024-04-20T02:33:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.712+0000 7f4b445e0700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.712+0000 7f4b445e0700 10 monclient: monmap: 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:23.713803+0000 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:23.713803+0000 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.713 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.714 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.714 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:23.714 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:23.714 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.713+0000 7f4b445e0700 1 Processor -- start 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.713+0000 7f4b445e0700 1 -- start start 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.713+0000 7f4b445e0700 10 monclient: init 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient: _add_conns ranks=[7,2,6,1,5,0,10,9,8,3,4] 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4b3c0a50f0 0x7f4b3c137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): picked mon.noname-i con 0x7f4b3c0a50f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:23.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): picked mon.noname-b con 0x7f4b3c1378a0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 0x7f4b3c0a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): picked mon.noname-d con 0x7f4b3c0a38f0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0a54d0 con 0x7f4b3c1378a0 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c1387b0 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c06da20 con 0x7f4b3c0a50f0 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b445e0700 10 monclient(hunting): authenticate will time out at 2683.115282s 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 0x7f4b3c0a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 0x7f4b3c0a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50772/0 (socket says 172.21.15.47:50772) 2024-04-20T02:33:23.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 -- 172.21.15.47:0/1108018980 learned_addr learned my addr 172.21.15.47:0/1108018980 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): get_auth_request con 0x7f4b3c0a38f0 auth_method 0 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): get_auth_request con 0x7f4b3c1378a0 auth_method 0 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4b3c0a50f0 0x7f4b3c137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): get_auth_request con 0x7f4b3c0a50f0 auth_method 0 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b4237c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_done global_id 55320 payload 274 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 -- 172.21.15.47:0/1108018980 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4b3c0a50f0 msgr2=0x7f4b3c137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4b3c0a50f0 0x7f4b3c137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 -- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 msgr2=0x7f4b3c138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 monclient: found mon.noname-d 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b42b7d700 1 -- 172.21.15.47:0/1108018980 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0751e0 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.714+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b42b7d700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 0x7f4b3c0a3d30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009e80 tx=0x7f4b3400b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b45541d0c155bf95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 1 -- 172.21.15.47:0/1108018980 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b3400e070 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: dump: 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.729 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 1 -- 172.21.15.47:0/1108018980 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b340099b0 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 5 monclient: authenticate success, global_id 55320 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 1 -- 172.21.15.47:0/1108018980 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b3400bd80 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 4 set_mon_vals no callback set 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b4137a700 10 monclient: dump: 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.731 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 10 monclient: shutdown 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 -- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 msgr2=0x7f4b3c0a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 0x7f4b3c0a3d30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009e80 tx=0x7f4b3400b330 comp rx=0 tx=0).stop 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 -- 172.21.15.47:0/1108018980 shutdown_connections 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f4b3c0a50f0 0x7f4b3c137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c0a38f0 0x7f4b3c0a3d30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 --2- 172.21.15.47:0/1108018980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.715+0000 7f4b445e0700 1 -- 172.21.15.47:0/1108018980 >> 172.21.15.47:0/1108018980 conn(0x7f4b3c0a8510 msgr2=0x7f4b3c0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.716+0000 7f4b445e0700 1 -- 172.21.15.47:0/1108018980 shutdown_connections 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.716+0000 7f4b445e0700 1 -- 172.21.15.47:0/1108018980 wait complete. 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.716+0000 7f4b445e0700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.716+0000 7f4b445e0700 10 monclient: monmap: 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.733 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.716+0000 7f4b445e0700 1 librados: starting msgr at 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.716+0000 7f4b445e0700 1 librados: starting objecter 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 Processor -- start 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 -- start start 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 librados: setting wanted keys 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 librados: calling monclient init 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient: init 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient: _add_conns ranks=[0,1,6,3,5,8,4,2,7] 2024-04-20T02:33:23.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b3c0a38f0 0x7f4b3c1cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): picked mon.noname-a con 0x7f4b3c0a38f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b3c0a50f0 0x7f4b3c136a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): picked mon.noname-f con 0x7f4b3c0a50f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c136fb0 0x7f4b3c1205c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): picked mon.noname-d con 0x7f4b3c136fb0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c123b90 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c071e80 con 0x7f4b3c0a50f0 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c072260 con 0x7f4b3c136fb0 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b445e0700 10 monclient(hunting): authenticate will time out at 2683.118584s 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c136fb0 0x7f4b3c1205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b3c0a38f0 0x7f4b3c1cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c136fb0 0x7f4b3c1205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50788/0 (socket says 172.21.15.47:50788) 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 1 -- 172.21.15.47:0/3490749189 learned_addr learned my addr 172.21.15.47:0/3490749189 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): get_auth_request con 0x7f4b3c136fb0 auth_method 0 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): get_auth_request con 0x7f4b3c0a38f0 auth_method 0 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b3c0a50f0 0x7f4b3c136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): get_auth_request con 0x7f4b3c0a50f0 auth_method 0 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b42b7d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b4237c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.717+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 10 monclient(hunting): handle_auth_done global_id 104121 payload 824 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c136fb0 msgr2=0x7f4b3c1205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c136fb0 0x7f4b3c1205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b3c0a50f0 msgr2=0x7f4b3c136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b3c0a50f0 0x7f4b3c136a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 monclient: found mon.noname-a 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b380094a0 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b42b7d700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4b3c136fb0 0x7f4b3c1205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b3c0a50f0 0x7f4b3c136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b4237c700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b3c0a38f0 0x7f4b3c1cc460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00e900 tx=0x7f4b2c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 client.?.objecter ms_handle_connect 0x7f4b3c0a38f0 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b2c013af0 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: dump: 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.738 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c013c70 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 5 monclient: authenticate success, global_id 104121 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 10 monclient: _renew_subs 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b34009b30 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b2c013de0 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b40b79700 4 set_mon_vals no callback set 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 10 client.104121.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b32ffd700 10 monclient: dump: 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.740 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 10 monclient: _renew_subs 2024-04-20T02:33:23.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c124430 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.718+0000 7f4b445e0700 1 librados: init done 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b445e0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b445e0700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c120b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0682f0 con 0x7f4b3c1378a0 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c120b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient: get_auth_request con 0x7f4b3c1378a0 auth_method 0 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_done global_id 104132 payload 824 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b3c06e370 con 0x7f4b3c1378a0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4b2c013680 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.719+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c120b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c074ff0 tx=0x7f4b3800f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4b2003a2f0 0x7f4b2003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 client.104121.objecter ms_handle_connect 0x7f4b3c1378a0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 client.104121.objecter resend_mon_ops 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(681..681 src has 266..681) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f4b2c04aa70 con 0x7f4b3c0a38f0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 client.104121.objecter ms_dispatch 0x7f4b3c001550 osd_map(681..681 src has 266..681) v4 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 3 client.104121.objecter handle_osd_map got epochs [681,681] > 0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 3 client.104121.objecter handle_osd_map decoding full epoch 681 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 20 client.104121.objecter dump_active .. 0 homeless 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b3800fe90 con 0x7f4b3c1378a0 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 monclient: dump: 2024-04-20T02:33:23.743 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4b38014440 con 0x7f4b3c1378a0 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 msgr2=0x7f4b3c120b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c120b00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c074ff0 tx=0x7f4b3800f520 comp rx=0 tx=0).stop 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 msgr2=0x7f4b3c120b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b42b7d700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4b2003a2f0 0x7f4b2003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c1378a0 0x7f4b3c120b00 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c074ff0 tx=0x7f4b3800f520 comp rx=0 tx=0).stop 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b42b7d700 10 monclient: get_auth_request con 0x7f4b2003a2f0 auth_method 0 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b32ffd700 10 client.104121.objecter ms_handle_connect 0x7f4b2003a2f0 2024-04-20T02:33:23.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.720+0000 7f4b42b7d700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4b2003a2f0 0x7f4b2003c7b0 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009e20 tx=0x7f4b34009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b445e0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b445e0700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 0x7f4b3c121040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b445e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c06a400 con 0x7f4b3c06aac0 2024-04-20T02:33:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 0x7f4b3c121040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b41b7b700 10 monclient: get_auth_request con 0x7f4b3c06aac0 auth_method 0 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b41b7b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b41b7b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.732+0000 7f4b41b7b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b41b7b700 10 monclient(hunting): handle_auth_done global_id 104141 payload 824 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b41b7b700 1 -- 172.21.15.47:0/3490749189 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4b3c06e650 con 0x7f4b3c06aac0 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b41b7b700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 0x7f4b3c121040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c121d20 tx=0x7f4b38004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b32ffd700 10 client.104121.objecter ms_handle_connect 0x7f4b3c06aac0 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b32ffd700 10 client.104121.objecter resend_mon_ops 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b38022c00 con 0x7f4b3c06aac0 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b32ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b32ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.733+0000 7f4b32ffd700 10 monclient: dump: 2024-04-20T02:33:23.748 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.749 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f4b38022d60 con 0x7f4b3c06aac0 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 msgr2=0x7f4b3c121040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 0x7f4b3c121040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c121d20 tx=0x7f4b38004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 msgr2=0x7f4b3c121040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.735+0000 7f4b32ffd700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b3c06aac0 0x7f4b3c121040 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c121d20 tx=0x7f4b38004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:23.750 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:23.751 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.754 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:23.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.756 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.758 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.759 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.760 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.764 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.767 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:23.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:23.769 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:23.770 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.737+0000 7f4b445e0700 10 librados: watch_flush enter 2024-04-20T02:33:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.737+0000 7f4b445e0700 10 librados: watch_flush exit 2024-04-20T02:33:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.737+0000 7f4b445e0700 20 client.104121.objecter shutdown clearing up homeless session... 2024-04-20T02:33:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.737+0000 7f4b445e0700 10 client.104121.objecter successfully canceled tick 2024-04-20T02:33:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4b2003a2f0 msgr2=0x7f4b2003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4b2003a2f0 0x7f4b2003c7b0 secure :-1 s=READY pgs=3366 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009e20 tx=0x7f4b34009000 comp rx=0 tx=0).stop 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 10 monclient: shutdown 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b3c0a38f0 msgr2=0x7f4b3c1cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 --2- 172.21.15.47:0/3490749189 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b3c0a38f0 0x7f4b3c1cc460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00e900 tx=0x7f4b2c00ec10 comp rx=0 tx=0).stop 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b42b7d700 1 -- 172.21.15.47:0/3490749189 reap_dead start 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b42b7d700 1 -- 172.21.15.47:0/3490749189 reap_dead start 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 shutdown_connections 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 >> 172.21.15.47:0/3490749189 conn(0x7f4b3c0a8510 msgr2=0x7f4b3c0a76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 shutdown_connections 2024-04-20T02:33:23.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.738+0000 7f4b445e0700 1 -- 172.21.15.47:0/3490749189 wait complete. 2024-04-20T02:33:23.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.741+0000 7f4b445e0700 1 librados: shutdown 2024-04-20T02:33:23.774 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:23.774 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:33:23.774 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:33:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.912+0000 7fde24516700 10 monclient: get_monmap_and_config 2024-04-20T02:33:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.912+0000 7fde24516700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.912+0000 7fde24516700 10 monclient: monmap: 2024-04-20T02:33:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.912 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:23.913276+0000 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:23.913276+0000 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:23.913 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:23.914 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.912+0000 7fde24516700 1 Processor -- start 2024-04-20T02:33:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.912+0000 7fde24516700 1 -- start start 2024-04-20T02:33:23.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.912+0000 7fde24516700 10 monclient: init 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient: _add_conns ranks=[7,3,5,2,4,6,0,9,10,1,8] 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 0x7fde1c0a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): picked mon.noname-i con 0x7fde1c0a5300 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a56e0 0x7fde1c0a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): picked mon.noname-g con 0x7fde1c0a56e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fde1c137ab0 0x7fde1c137f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): picked mon.noname-h con 0x7fde1c137ab0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0a3140 con 0x7fde1c0a56e0 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0a1bf0 con 0x7fde1c137ab0 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0a1fd0 con 0x7fde1c0a5300 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde24516700 10 monclient(hunting): authenticate will time out at 2683.314547s 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 0x7fde1c0a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde21ab1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a56e0 0x7fde1c0a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde22ab3700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fde1c137ab0 0x7fde1c137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 0x7fde1c0a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:50692/0 (socket says 172.21.15.47:50692) 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 1 -- 172.21.15.47:0/3447198414 learned_addr learned my addr 172.21.15.47:0/3447198414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 10 monclient(hunting): get_auth_request con 0x7fde1c0a5300 auth_method 0 2024-04-20T02:33:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde222b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde22ab3700 10 monclient(hunting): get_auth_request con 0x7fde1c137ab0 auth_method 0 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde22ab3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde22ab3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde22ab3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde21ab1700 10 monclient(hunting): get_auth_request con 0x7fde1c0a56e0 auth_method 0 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde21ab1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde21ab1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.913+0000 7fde21ab1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde22ab3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde22ab3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde22ab3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde21ab1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde21ab1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde21ab1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 10 monclient(hunting): handle_auth_done global_id 84535 payload 274 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 -- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fde1c137ab0 msgr2=0x7fde1c137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fde1c137ab0 0x7fde1c137f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 -- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a56e0 msgr2=0x7fde1c0a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a56e0 0x7fde1c0a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 monclient: found mon.noname-i 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 -- 172.21.15.47:0/3447198414 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0751e0 con 0x7fde1c0a5300 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde22ab3700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fde1c137ab0 0x7fde1c137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde21ab1700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a56e0 0x7fde1c0a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde222b2700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 0x7fde1c0a3f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde18009950 tx=0x7fde18009c60 comp rx=0 tx=0).ready entity=mon.7 client_cookie=33e526a8c3066062 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 1 -- 172.21.15.47:0/3447198414 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde18011aa0 con 0x7fde1c0a5300 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: dump: 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde24516700 5 monclient: authenticate success, global_id 84535 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 1 -- 172.21.15.47:0/3447198414 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18011c20 con 0x7fde1c0a5300 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde24516700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde24516700 4 set_mon_vals no callback set 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 1 -- 172.21.15.47:0/3447198414 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde18011d90 con 0x7fde1c0a5300 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.914+0000 7fde212b0700 10 monclient: dump: 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.930 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 10 monclient: shutdown 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 -- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 msgr2=0x7fde1c0a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 0x7fde1c0a3f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde18009950 tx=0x7fde18009c60 comp rx=0 tx=0).stop 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 -- 172.21.15.47:0/3447198414 shutdown_connections 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fde1c0a5300 0x7fde1c0a3f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fde1c137ab0 0x7fde1c137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 --2- 172.21.15.47:0/3447198414 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a56e0 0x7fde1c0a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 -- 172.21.15.47:0/3447198414 >> 172.21.15.47:0/3447198414 conn(0x7fde1c0aa710 msgr2=0x7fde1c0aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 -- 172.21.15.47:0/3447198414 shutdown_connections 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.915+0000 7fde24516700 1 -- 172.21.15.47:0/3447198414 wait complete. 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 10 monclient: build_initial_monmap 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 10 monclient: monmap: 2024-04-20T02:33:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 1 librados: starting msgr at 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 1 librados: starting objecter 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 1 Processor -- start 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 1 -- start start 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 1 librados: setting wanted keys 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 1 librados: calling monclient init 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 10 monclient: init 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.916+0000 7fde24516700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient: _add_conns ranks=[3,6,8,0,5,1,7,2,4] 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a5300 0x7fde1c1cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): picked mon.noname-g con 0x7fde1c0a5300 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fde1c0a56e0 0x7fde1c136ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): picked mon.noname-d con 0x7fde1c0a56e0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde1c137ab0 0x7fde1c137400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): picked mon.noname-e con 0x7fde1c137ab0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c1cd1c0 con 0x7fde1c0a5300 2024-04-20T02:33:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c137940 con 0x7fde1c0a56e0 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c124650 con 0x7fde1c137ab0 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde24516700 10 monclient(hunting): authenticate will time out at 2683.318103s 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fde1c0a56e0 0x7fde1c136ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a5300 0x7fde1c1cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fde1c0a56e0 0x7fde1c136ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50802/0 (socket says 172.21.15.47:50802) 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 -- 172.21.15.47:0/554601081 learned_addr learned my addr 172.21.15.47:0/554601081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): get_auth_request con 0x7fde1c0a56e0 auth_method 0 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): get_auth_request con 0x7fde1c0a5300 auth_method 0 2024-04-20T02:33:23.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde1c137ab0 0x7fde1c137400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): get_auth_request con 0x7fde1c137ab0 auth_method 0 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde22ab3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient(hunting): handle_auth_done global_id 55338 payload 824 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde1c137ab0 msgr2=0x7fde1c137400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fde1c137ab0 0x7fde1c137400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a5300 msgr2=0x7fde1c1cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a5300 0x7fde1c1cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient: _finish_hunting 0 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 monclient: found mon.noname-d 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0094a0 con 0x7fde1c0a56e0 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde222b2700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde1c0a5300 0x7fde1c1cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde21ab1700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fde1c0a56e0 0x7fde1c136ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ebb0 tx=0x7fde0c00eec0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde12ffd700 10 client.?.objecter ms_handle_connect 0x7fde1c0a56e0 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde12ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde0c00ccc0 con 0x7fde1c0a56e0 2024-04-20T02:33:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde12ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde12ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.917+0000 7fde12ffd700 10 monclient: dump: 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c00ce20 con 0x7fde1c0a56e0 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 5 monclient: authenticate success, global_id 55338 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 monclient: _renew_subs 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde14009a70 con 0x7fde1c0a56e0 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde0c0138e0 con 0x7fde1c0a56e0 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:23.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde20aaf700 4 set_mon_vals no callback set 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 10 monclient: dump: 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.939 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 client.55338.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 monclient: _renew_subs 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde18009550 con 0x7fde1c0a56e0 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 1 librados: init done 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fde0c013470 con 0x7fde1c0a56e0 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 0x7fde1c123db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.918+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c068230 con 0x7fde1c06aac0 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 0x7fde1c123db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde0003a2f0 0x7fde0003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient: get_auth_request con 0x7fde1c06aac0 auth_method 0 2024-04-20T02:33:23.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(682..682 src has 266..682) v4 ==== 6674+0+0 (secure 0 0 0) 0x7fde0c04aa90 con 0x7fde1c0a56e0 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 client.55338.objecter ms_dispatch 0x7fde1c001550 osd_map(682..682 src has 266..682) v4 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 3 client.55338.objecter handle_osd_map got epochs [682,682] > 0 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 3 client.55338.objecter handle_osd_map decoding full epoch 682 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 20 client.55338.objecter dump_active .. 0 homeless 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 10 monclient(hunting): handle_auth_done global_id 84523 payload 824 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde1c1242f0 con 0x7fde1c06aac0 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde22ab3700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde0003a2f0 0x7fde0003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde22ab3700 10 monclient: get_auth_request con 0x7fde0003a2f0 auth_method 0 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 client.55338.objecter ms_handle_connect 0x7fde1c06aac0 2024-04-20T02:33:23.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 client.55338.objecter resend_mon_ops 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde222b2700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 0x7fde1c123db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde18009950 tx=0x7fde18004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde18020070 con 0x7fde1c06aac0 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 monclient: dump: 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.942 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde12ffd700 10 client.55338.objecter ms_handle_connect 0x7fde0003a2f0 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.919+0000 7fde22ab3700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde0003a2f0 0x7fde0003c7b0 secure :-1 s=READY pgs=3367 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0a1910 tx=0x7fde1400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fde18004450 con 0x7fde1c06aac0 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 msgr2=0x7fde1c123db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 0x7fde1c123db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde18009950 tx=0x7fde18004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 msgr2=0x7fde1c123db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:23.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.920+0000 7fde12ffd700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c06aac0 0x7fde1c123db0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde18009950 tx=0x7fde18004750 comp rx=0 tx=0).stop 2024-04-20T02:33:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.927+0000 7fde24516700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.927+0000 7fde24516700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 0x7fde1c120d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.927+0000 7fde24516700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.927+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c06a400 con 0x7fde1c120940 2024-04-20T02:33:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 0x7fde1c120d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:23.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient: get_auth_request con 0x7fde1c120940 auth_method 0 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 10 monclient(hunting): handle_auth_done global_id 84532 payload 824 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 1 -- 172.21.15.47:0/554601081 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde1c09dea0 con 0x7fde1c120940 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 client.55338.objecter ms_handle_connect 0x7fde1c120940 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde222b2700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 0x7fde1c120d50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde18009130 tx=0x7fde18011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 client.55338.objecter resend_mon_ops 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fde18003ed0 con 0x7fde1c120940 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 monclient: dump: 2024-04-20T02:33:23.946 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:23.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7fde1802c630 con 0x7fde1c120940 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 msgr2=0x7fde1c120d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 0x7fde1c120d50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde18009130 tx=0x7fde18011000 comp rx=0 tx=0).stop 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 msgr2=0x7fde1c120d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.928+0000 7fde12ffd700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fde1c120940 0x7fde1c120d50 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde18009130 tx=0x7fde18011000 comp rx=0 tx=0).stop 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:23.948 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:23.949 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.950 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:23.951 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:23.952 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.953 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:23.954 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.955 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:23.956 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:23.957 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:23.959 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:23.960 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:23.962 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:23.963 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:23.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:23.966 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:23.967 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:23.968 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 10 librados: watch_flush enter 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 10 librados: watch_flush exit 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 20 client.55338.objecter shutdown clearing up homeless session... 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 10 client.55338.objecter successfully canceled tick 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde0003a2f0 msgr2=0x7fde0003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fde0003a2f0 0x7fde0003c7b0 secure :-1 s=READY pgs=3367 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0a1910 tx=0x7fde1400b040 comp rx=0 tx=0).stop 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 10 monclient: shutdown 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fde1c0a56e0 msgr2=0x7fde1c136ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 --2- 172.21.15.47:0/554601081 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fde1c0a56e0 0x7fde1c136ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ebb0 tx=0x7fde0c00eec0 comp rx=0 tx=0).stop 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde22ab3700 1 -- 172.21.15.47:0/554601081 reap_dead start 2024-04-20T02:33:23.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde22ab3700 1 -- 172.21.15.47:0/554601081 reap_dead start 2024-04-20T02:33:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 shutdown_connections 2024-04-20T02:33:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 >> 172.21.15.47:0/554601081 conn(0x7fde1c0aa710 msgr2=0x7fde1c1bfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 shutdown_connections 2024-04-20T02:33:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 -- 172.21.15.47:0/554601081 wait complete. 2024-04-20T02:33:23.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:23.929+0000 7fde24516700 1 librados: shutdown 2024-04-20T02:33:23.971 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:23.971 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:33:23.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:33:24.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.109+0000 7f4115314700 10 monclient: get_monmap_and_config 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.109+0000 7f4115314700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.109+0000 7f4115314700 10 monclient: monmap: 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:24.111001+0000 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:24.111001+0000 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.110+0000 7f4115314700 1 Processor -- start 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.110+0000 7f4115314700 1 -- start start 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.110+0000 7f4115314700 10 monclient: init 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient: _add_conns ranks=[1,3,9,7,10,4,0,8,5,2,6] 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4110137d90 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41100a09e0 0x7f411013cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): picked mon.noname-g con 0x7f41100a09e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f411013d1c0 0x7f411013f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): picked mon.noname-a con 0x7f411013d1c0 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100751e0 con 0x7f4110137d90 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411009fa40 con 0x7f411013d1c0 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411009dfc0 con 0x7f41100a09e0 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f4115314700 10 monclient(hunting): authenticate will time out at 2683.512194s 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f411013d1c0 0x7f411013f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f411013d1c0 0x7f411013f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45090/0 (socket says 172.21.15.47:45090) 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 -- 172.21.15.47:0/3461310607 learned_addr learned my addr 172.21.15.47:0/3461310607 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): get_auth_request con 0x7f411013d1c0 auth_method 0 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410e7fc700 1 --2- 172.21.15.47:0/3461310607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41100a09e0 0x7f411013cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410e7fc700 10 monclient(hunting): get_auth_request con 0x7f41100a09e0 auth_method 0 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient(hunting): handle_auth_done global_id 104130 payload 274 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 -- 172.21.15.47:0/3461310607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41100a09e0 msgr2=0x7f411013cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 --2- 172.21.15.47:0/3461310607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41100a09e0 0x7f411013cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 -- 172.21.15.47:0/3461310607 >> v1:172.21.15.179:6789/0 conn(0x7f4110137d90 legacy=0x7f4110138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 monclient: found mon.noname-a 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 -- 172.21.15.47:0/3461310607 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110138880 con 0x7f411013d1c0 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.111+0000 7f410f7fe700 1 --2- 172.21.15.47:0/3461310607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411013d1c0 0x7f411013f5b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4108009e80 tx=0x7f410800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b682187b543adac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 1 -- 172.21.15.47:0/3461310607 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f410800e070 con 0x7f411013d1c0 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: dump: 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 5 monclient: authenticate success, global_id 104130 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 1 -- 172.21.15.47:0/3461310607 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41080099b0 con 0x7f411013d1c0 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 4 set_mon_vals no callback set 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 1 -- 172.21.15.47:0/3461310607 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f410800bdf0 con 0x7f411013d1c0 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f410dffb700 10 monclient: dump: 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 10 monclient: shutdown 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 -- 172.21.15.47:0/3461310607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411013d1c0 msgr2=0x7f411013f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 --2- 172.21.15.47:0/3461310607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411013d1c0 0x7f411013f5b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4108009e80 tx=0x7f410800b330 comp rx=0 tx=0).stop 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 -- 172.21.15.47:0/3461310607 shutdown_connections 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 --2- 172.21.15.47:0/3461310607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411013d1c0 0x7f411013f5b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 --2- 172.21.15.47:0/3461310607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41100a09e0 0x7f411013cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 -- 172.21.15.47:0/3461310607 >> 172.21.15.47:0/3461310607 conn(0x7f41100aa8b0 msgr2=0x7f41100a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 -- 172.21.15.47:0/3461310607 shutdown_connections 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.112+0000 7f4115314700 1 -- 172.21.15.47:0/3461310607 wait complete. 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.113+0000 7f4115314700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.113+0000 7f4115314700 10 monclient: monmap: 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.129 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.113+0000 7f4115314700 1 librados: starting msgr at 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.113+0000 7f4115314700 1 librados: starting objecter 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 Processor -- start 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 -- start start 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 librados: setting wanted keys 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 librados: calling monclient init 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient: init 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient: _add_conns ranks=[1,4,8,5,7,2,6,0,3] 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100a09e0 0x7f41101cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): picked mon.noname-f con 0x7f41100a09e0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4110137d90 0x7f41101ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): picked mon.noname-c con 0x7f4110137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f411013d1c0 0x7f41101cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): picked mon.noname-e con 0x7f411013d1c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411009e9e0 con 0x7f41100a09e0 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f411009dc20 con 0x7f4110137d90 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f41101328a0 con 0x7f411013d1c0 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f4115314700 10 monclient(hunting): authenticate will time out at 2683.515728s 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4110137d90 0x7f41101ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4110137d90 0x7f41101ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58020/0 (socket says 172.21.15.47:58020) 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410f7fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f411013d1c0 0x7f41101cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 1 -- 172.21.15.47:0/2831514841 learned_addr learned my addr 172.21.15.47:0/2831514841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 10 monclient(hunting): get_auth_request con 0x7f4110137d90 auth_method 0 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410effd700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100a09e0 0x7f41101cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410f7fe700 10 monclient(hunting): get_auth_request con 0x7f411013d1c0 auth_method 0 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410effd700 10 monclient(hunting): get_auth_request con 0x7f41100a09e0 auth_method 0 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.114+0000 7f410e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 10 monclient(hunting): handle_auth_done global_id 84541 payload 824 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f411013d1c0 msgr2=0x7f41101cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f411013d1c0 0x7f41101cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100a09e0 msgr2=0x7f41101cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100a09e0 0x7f41101cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 monclient: found mon.noname-c 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4108009b30 con 0x7f4110137d90 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410effd700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41100a09e0 0x7f41101cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410f7fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f411013d1c0 0x7f41101cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410e7fc700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4110137d90 0x7f41101ccb10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f410400e7e0 tx=0x7f410400eaf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 client.?.objecter ms_handle_connect 0x7f4110137d90 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f410400c070 con 0x7f4110137d90 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: dump: 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.134 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 5 monclient: authenticate success, global_id 84541 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104013880 con 0x7f4110137d90 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 10 monclient: _renew_subs 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41101318d0 con 0x7f4110137d90 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4104013e60 con 0x7f4110137d90 2024-04-20T02:33:24.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f410d7fa700 4 set_mon_vals no callback set 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 10 client.84541.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f40f77fe700 10 monclient: dump: 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 10 monclient: _renew_subs 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4110131310 con 0x7f4110137d90 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.115+0000 7f4115314700 1 librados: init done 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f4115314700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f4115314700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 0x7f411012e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4110068180 con 0x7f411006aac0 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f41040139f0 con 0x7f4110137d90 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 0x7f411012e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient: get_auth_request con 0x7f411006aac0 auth_method 0 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f40f77fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f40fc03a2f0 0x7f40fc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(682..682 src has 266..682) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f410404a610 con 0x7f4110137d90 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f40f77fe700 10 client.84541.objecter ms_dispatch 0x7f4110001550 osd_map(682..682 src has 266..682) v4 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f40f77fe700 3 client.84541.objecter handle_osd_map got epochs [682,682] > 0 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.116+0000 7f40f77fe700 3 client.84541.objecter handle_osd_map decoding full epoch 682 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f410effd700 10 monclient(hunting): handle_auth_done global_id 55347 payload 824 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f410effd700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41100a6b90 con 0x7f411006aac0 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 20 client.84541.objecter dump_active .. 0 homeless 2024-04-20T02:33:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f410f7fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f40fc03a2f0 0x7f40fc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f410f7fe700 10 monclient: get_auth_request con 0x7f40fc03a2f0 auth_method 0 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 client.84541.objecter ms_handle_connect 0x7f411006aac0 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 client.84541.objecter resend_mon_ops 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f410effd700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 0x7f411012e9b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4100000f60 tx=0x7f410000fdc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4100006cd0 con 0x7f411006aac0 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 monclient: dump: 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 client.84541.objecter ms_handle_connect 0x7f40fc03a2f0 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f410f7fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f40fc03a2f0 0x7f40fc03c7b0 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 crypto rx=0x7f4108009e80 tx=0x7f4108009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4100006e30 con 0x7f411006aac0 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 msgr2=0x7f411012e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 0x7f411012e9b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4100000f60 tx=0x7f410000fdc0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 msgr2=0x7f411012e9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.117+0000 7f40f77fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f411006aac0 0x7f411012e9b0 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4100000f60 tx=0x7f410000fdc0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f4115314700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f4115314700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 0x7f411012eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f4115314700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f411006a400 con 0x7f41100a4340 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 0x7f411012eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 10 monclient: get_auth_request con 0x7f41100a4340 auth_method 0 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.125+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f410effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f410effd700 10 monclient(hunting): handle_auth_done global_id 55356 payload 824 2024-04-20T02:33:24.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f410effd700 1 -- 172.21.15.47:0/2831514841 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f41100a7bc0 con 0x7f41100a4340 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f410effd700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 0x7f411012eef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4100000f60 tx=0x7f4100001120 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 client.84541.objecter ms_handle_connect 0x7f41100a4340 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 client.84541.objecter resend_mon_ops 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4100023070 con 0x7f41100a4340 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 monclient: dump: 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.144 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f410001edf0 con 0x7f41100a4340 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 msgr2=0x7f411012eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 0x7f411012eef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4100000f60 tx=0x7f4100001120 comp rx=0 tx=0).stop 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 msgr2=0x7f411012eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.126+0000 7f40f77fe700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f41100a4340 0x7f411012eef0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4100000f60 tx=0x7f4100001120 comp rx=0 tx=0).stop 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:24.146 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.147 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.148 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:24.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.150 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:24.151 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.152 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.153 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:24.154 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.155 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.156 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.157 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.159 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.160 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:24.161 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.162 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.163 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.164 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:24.165 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 10 librados: watch_flush enter 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 10 librados: watch_flush exit 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 20 client.84541.objecter shutdown clearing up homeless session... 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 10 client.84541.objecter successfully canceled tick 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f40fc03a2f0 msgr2=0x7f40fc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f40fc03a2f0 0x7f40fc03c7b0 secure :-1 s=READY pgs=3368 cs=0 l=1 rev1=1 crypto rx=0x7f4108009e80 tx=0x7f4108009000 comp rx=0 tx=0).stop 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 10 monclient: shutdown 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4110137d90 msgr2=0x7f41101ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 --2- 172.21.15.47:0/2831514841 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4110137d90 0x7f41101ccb10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f410400e7e0 tx=0x7f410400eaf0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f410f7fe700 1 -- 172.21.15.47:0/2831514841 reap_dead start 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f410f7fe700 1 -- 172.21.15.47:0/2831514841 reap_dead start 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 shutdown_connections 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 >> 172.21.15.47:0/2831514841 conn(0x7f41100aa8b0 msgr2=0x7f41100a8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 shutdown_connections 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 -- 172.21.15.47:0/2831514841 wait complete. 2024-04-20T02:33:24.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.127+0000 7f4115314700 1 librados: shutdown 2024-04-20T02:33:24.168 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:24.168 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:33:24.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:33:24.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.308+0000 7f5199cfb700 10 monclient: get_monmap_and_config 2024-04-20T02:33:24.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.308+0000 7f5199cfb700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.308+0000 7f5199cfb700 10 monclient: monmap: 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:24.309530+0000 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:24.309530+0000 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.320 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 Processor -- start 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 -- start start 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient: init 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient: _add_conns ranks=[2,3,0,1,10,5,7,4,8,9,6] 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51940a0830 0x7f51940a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): picked mon.noname-b con 0x7f51940a0830 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a01b0 0x7f51940a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): picked mon.noname-g con 0x7f51940a01b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f51940a1150 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51940751e0 con 0x7f51940a1150 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5194067820 con 0x7f51940a0830 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519409deb0 con 0x7f51940a01b0 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.309+0000 7f5199cfb700 10 monclient(hunting): authenticate will time out at 2683.710924s 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51940a0830 0x7f51940a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5193fff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f51940a1150 0x7f51940a2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33422/0 (socket says 172.21.15.47:33422) 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5193fff700 1 -- 172.21.15.47:0/1222054670 learned_addr learned my addr 172.21.15.47:0/1222054670 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): get_auth_request con 0x7f51940a0830 auth_method 0 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5192ffd700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a01b0 0x7f51940a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5192ffd700 10 monclient(hunting): get_auth_request con 0x7f51940a01b0 auth_method 0 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5192ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5192ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5192ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient(hunting): handle_auth_done global_id 104150 payload 274 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 1 -- 172.21.15.47:0/1222054670 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235762757 0 0) 0x7f51940751e0 con 0x7f51940a1150 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 -- 172.21.15.47:0/1222054670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a01b0 msgr2=0x7f51940a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a01b0 0x7f51940a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 -- 172.21.15.47:0/1222054670 >> v1:172.21.15.47:6789/0 conn(0x7f51940a1150 legacy=0x7f51940a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 monclient: found mon.noname-b 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 -- 172.21.15.47:0/1222054670 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194067330 con 0x7f51940a0830 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5192ffd700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a01b0 0x7f51940a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51937fe700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51940a0830 0x7f51940a0c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5184009870 tx=0x7f5184009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5379fe0b8e7d5a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 1 -- 172.21.15.47:0/1222054670 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5184011aa0 con 0x7f51940a0830 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: dump: 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.324 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 1 -- 172.21.15.47:0/1222054670 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184011c00 con 0x7f51940a0830 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5199cfb700 5 monclient: authenticate success, global_id 104150 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5199cfb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5199cfb700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 1 -- 172.21.15.47:0/1222054670 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5184011d70 con 0x7f51940a0830 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f5199cfb700 4 set_mon_vals no callback set 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.310+0000 7f51927fc700 10 monclient: dump: 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.326 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 10 monclient: shutdown 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 -- 172.21.15.47:0/1222054670 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51940a0830 msgr2=0x7f51940a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51940a0830 0x7f51940a0c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5184009870 tx=0x7f5184009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 -- 172.21.15.47:0/1222054670 shutdown_connections 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51940a0830 0x7f51940a0c10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 --2- 172.21.15.47:0/1222054670 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a01b0 0x7f51940a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 -- 172.21.15.47:0/1222054670 >> 172.21.15.47:0/1222054670 conn(0x7f51940a7570 msgr2=0x7f51940a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 -- 172.21.15.47:0/1222054670 shutdown_connections 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.311+0000 7f5199cfb700 1 -- 172.21.15.47:0/1222054670 wait complete. 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 10 monclient: monmap: 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:24.328 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 1 librados: starting msgr at 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 1 librados: starting objecter 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 1 Processor -- start 2024-04-20T02:33:24.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 1 -- start start 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 1 librados: setting wanted keys 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 1 librados: calling monclient init 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.312+0000 7f5199cfb700 10 monclient: init 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient: _add_conns ranks=[8,1,3,4,0,2,7,6,5] 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 0x7f51941cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): picked mon.noname-e con 0x7f51940a01b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51940a0830 0x7f51941bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): picked mon.noname-f con 0x7f51940a0830 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a1150 0x7f51941bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): picked mon.noname-g con 0x7f51940a1150 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519409e440 con 0x7f51940a0830 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51941a9230 con 0x7f51940a1150 2024-04-20T02:33:24.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f51941aacf0 con 0x7f51940a01b0 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5199cfb700 10 monclient(hunting): authenticate will time out at 2683.714317s 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 0x7f51941cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 0x7f51941cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42778/0 (socket says 172.21.15.47:42778) 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 -- 172.21.15.47:0/186445609 learned_addr learned my addr 172.21.15.47:0/186445609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): get_auth_request con 0x7f51940a01b0 auth_method 0 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5193fff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a1150 0x7f51941bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5192ffd700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51940a0830 0x7f51941bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5193fff700 10 monclient(hunting): get_auth_request con 0x7f51940a1150 auth_method 0 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5193fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5193fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5193fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5192ffd700 10 monclient(hunting): get_auth_request con 0x7f51940a0830 auth_method 0 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5192ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5192ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5192ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient(hunting): handle_auth_done global_id 36134 payload 824 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a1150 msgr2=0x7f51941bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51940a1150 0x7f51941bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51940a0830 msgr2=0x7f51941bbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51940a0830 0x7f51941bbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 monclient: found mon.noname-e 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840094a0 con 0x7f51940a01b0 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f5192ffd700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51940a0830 0x7f51941bbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f51937fe700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 0x7f51941cc250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f518400a010 tx=0x7f5184004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f517bfff700 10 client.?.objecter ms_handle_connect 0x7f51940a01b0 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.313+0000 7f517bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f51840043e0 con 0x7f51940a01b0 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: dump: 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.333 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 5 monclient: authenticate success, global_id 36134 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518401f070 con 0x7f51940a01b0 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5184004560 con 0x7f51940a01b0 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 10 monclient: _renew_subs 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51941aafd0 con 0x7f51940a01b0 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5191ffb700 4 set_mon_vals no callback set 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f517bfff700 10 monclient: dump: 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.335 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 10 client.36134.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 10 monclient: _renew_subs 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51941a9af0 con 0x7f51940a01b0 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 1 librados: init done 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.314+0000 7f5199cfb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5199cfb700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 0x7f51941a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5194068310 con 0x7f519406aac0 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 0x7f51941a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5184021030 con 0x7f51940a01b0 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient: get_auth_request con 0x7f519406aac0 auth_method 0 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f518003a2f0 0x7f518003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 10 monclient(hunting): handle_auth_done global_id 36143 payload 824 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f519409f1c0 con 0x7f519406aac0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(682..682 src has 266..682) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f518404bc50 con 0x7f51940a01b0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 10 client.36134.objecter ms_dispatch 0x7f5194001550 osd_map(682..682 src has 266..682) v4 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 3 client.36134.objecter handle_osd_map got epochs [682,682] > 0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 3 client.36134.objecter handle_osd_map decoding full epoch 682 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 20 client.36134.objecter dump_active .. 0 homeless 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5193fff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f518003a2f0 0x7f518003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5193fff700 10 monclient: get_auth_request con 0x7f518003a2f0 auth_method 0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 10 client.36134.objecter ms_handle_connect 0x7f519406aac0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 10 client.36134.objecter resend_mon_ops 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f5192ffd700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 0x7f51941a7600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f51940a7e80 tx=0x7f518800fb10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5188014ae0 con 0x7f519406aac0 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:24.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.315+0000 7f517bfff700 10 monclient: dump: 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.339 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 10 client.36134.objecter ms_handle_connect 0x7f518003a2f0 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f5193fff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f518003a2f0 0x7f518003c7b0 secure :-1 s=READY pgs=3369 cs=0 l=1 rev1=1 crypto rx=0x7f518c0012e0 tx=0x7f518c0018a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f518800c050 con 0x7f519406aac0 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 msgr2=0x7f51941a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 0x7f51941a7600 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f51940a7e80 tx=0x7f518800fb10 comp rx=0 tx=0).stop 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 msgr2=0x7f51941a7600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.316+0000 7f517bfff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f519406aac0 0x7f51941a7600 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f51940a7e80 tx=0x7f518800fb10 comp rx=0 tx=0).stop 2024-04-20T02:33:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.323+0000 7f5199cfb700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.323+0000 7f5199cfb700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 0x7f51941a7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.323+0000 7f5199cfb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.323+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f519406a400 con 0x7f51941a7b40 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 0x7f51941a7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient: get_auth_request con 0x7f51941a7b40 auth_method 0 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 10 monclient(hunting): handle_auth_done global_id 36152 payload 824 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 1 -- 172.21.15.47:0/186445609 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f51940a4450 con 0x7f51941a7b40 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 client.36134.objecter ms_handle_connect 0x7f51941a7b40 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f5192ffd700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 0x7f51941a7f50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f518801a000 tx=0x7f51880047c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 client.36134.objecter resend_mon_ops 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5188004020 con 0x7f51941a7b40 2024-04-20T02:33:24.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 monclient: dump: 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.344 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9354 (secure 0 0 0) 0x7f5188024070 con 0x7f51941a7b40 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 msgr2=0x7f51941a7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 0x7f51941a7f50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f518801a000 tx=0x7f51880047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 msgr2=0x7f51941a7f50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.324+0000 7f517bfff700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51941a7b40 0x7f51941a7f50 secure :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f518801a000 tx=0x7f51880047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:24.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:24.346 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.347 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:24.348 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:24.349 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.351 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:24.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.353 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:24.354 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.356 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.357 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.358 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:24.359 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.361 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.362 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:24.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.364 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.365 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:24.366 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.367 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:24.368 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 10 librados: watch_flush enter 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 10 librados: watch_flush exit 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 20 client.36134.objecter shutdown clearing up homeless session... 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 10 client.36134.objecter successfully canceled tick 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f518003a2f0 msgr2=0x7f518003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f518003a2f0 0x7f518003c7b0 secure :-1 s=READY pgs=3369 cs=0 l=1 rev1=1 crypto rx=0x7f518c0012e0 tx=0x7f518c0018a0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 10 monclient: shutdown 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 msgr2=0x7f51941cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 0x7f51941cc250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f518400a010 tx=0x7f5184004750 comp rx=0 tx=0).stop 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5193fff700 1 -- 172.21.15.47:0/186445609 reap_dead start 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 shutdown_connections 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 --2- 172.21.15.47:0/186445609 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f51940a01b0 0x7f51941cc250 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 >> 172.21.15.47:0/186445609 conn(0x7f51940a7570 msgr2=0x7f51940a5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 shutdown_connections 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 -- 172.21.15.47:0/186445609 wait complete. 2024-04-20T02:33:24.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.325+0000 7f5199cfb700 1 librados: shutdown 2024-04-20T02:33:24.371 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:24.372 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:33:24.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.510+0000 7fa5d1247700 10 monclient: get_monmap_and_config 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.510+0000 7fa5d1247700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.510+0000 7fa5d1247700 10 monclient: monmap: 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:24.511239+0000 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:24.511239+0000 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.510 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:24.511 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.510+0000 7fa5d1247700 1 Processor -- start 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.510+0000 7fa5d1247700 1 -- start start 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.510+0000 7fa5d1247700 10 monclient: init 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient: _add_conns ranks=[0,5,7,3,8,9,6,10,2,1,4] 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa5cc0a19d0 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc0a1db0 0x7fa5cc13ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): picked mon.noname-h con 0x7fa5cc0a1db0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc13b150 0x7fa5cc13d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): picked mon.noname-i con 0x7fa5cc13b150 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc0751e0 con 0x7fa5cc0a19d0 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc136610 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0a2230 con 0x7fa5cc13b150 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5d1247700 10 monclient(hunting): authenticate will time out at 2683.912390s 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cad9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa5cc0a19d0 0x7fa5cc135f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33426/0 (socket says 172.21.15.47:33426) 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cad9d700 1 -- 172.21.15.47:0/2933790505 learned_addr learned my addr 172.21.15.47:0/2933790505 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc13b150 0x7fa5cc13d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): get_auth_request con 0x7fa5cc13b150 auth_method 0 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc0a1db0 0x7fa5cc13ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): get_auth_request con 0x7fa5cc0a1db0 auth_method 0 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5c9d9b700 1 -- 172.21.15.47:0/2933790505 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938018733 0 0) 0x7fa5cc0751e0 con 0x7fa5cc0a19d0 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5c9d9b700 10 monclient(hunting): _negotiate 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5c9d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5c9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5c9d9b700 10 monclient(hunting): my global_id is 104148 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5c9d9b700 1 -- 172.21.15.47:0/2933790505 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b40035b0 con 0x7fa5cc0a19d0 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_done global_id 84544 payload 274 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 1 -- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc0a1db0 msgr2=0x7fa5cc13ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc0a1db0 0x7fa5cc13ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 1 -- 172.21.15.47:0/2933790505 >> v1:172.21.15.47:6789/0 conn(0x7fa5cc0a19d0 legacy=0x7fa5cc135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 1 monclient: found mon.noname-i 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5cb59e700 1 -- 172.21.15.47:0/2933790505 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0a4040 con 0x7fa5cc13b150 2024-04-20T02:33:24.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.511+0000 7fa5ca59c700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc0a1db0 0x7fa5cc13ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5cb59e700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc13b150 0x7fa5cc13d540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009dc0 tx=0x7fa5c400b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=7c86eb3a096074be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 1 -- 172.21.15.47:0/2933790505 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5c400e070 con 0x7fa5cc13b150 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: dump: 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.529 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 5 monclient: authenticate success, global_id 84544 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 1 -- 172.21.15.47:0/2933790505 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40098f0 con 0x7fa5cc13b150 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 1 -- 172.21.15.47:0/2933790505 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5c40128a0 con 0x7fa5cc13b150 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 4 set_mon_vals no callback set 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5c9d9b700 10 monclient: dump: 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.531 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 10 monclient: shutdown 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 1 -- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc13b150 msgr2=0x7fa5cc13d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc13b150 0x7fa5cc13d540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009dc0 tx=0x7fa5c400b330 comp rx=0 tx=0).stop 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.512+0000 7fa5d1247700 1 -- 172.21.15.47:0/2933790505 shutdown_connections 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc13b150 0x7fa5cc13d540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2933790505 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc0a1db0 0x7fa5cc13ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 1 -- 172.21.15.47:0/2933790505 >> 172.21.15.47:0/2933790505 conn(0x7fa5cc0a4a50 msgr2=0x7fa5cc0a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 1 -- 172.21.15.47:0/2933790505 shutdown_connections 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 1 -- 172.21.15.47:0/2933790505 wait complete. 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.513+0000 7fa5d1247700 10 monclient: monmap: 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:24.533 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 1 librados: starting msgr at 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 1 librados: starting objecter 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 1 Processor -- start 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 1 -- start start 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 1 librados: setting wanted keys 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 1 librados: calling monclient init 2024-04-20T02:33:24.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.514+0000 7fa5d1247700 10 monclient: init 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient: _add_conns ranks=[7,8,5,3,1,6,0,4,2] 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc0a19d0 0x7fa5cc09d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): picked mon.noname-i con 0x7fa5cc0a19d0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa5cc0a1db0 0x7fa5cc1349a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): picked mon.noname-e con 0x7fa5cc0a1db0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc13b150 0x7fa5cc11e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): picked mon.noname-h con 0x7fa5cc13b150 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc121070 con 0x7fa5cc13b150 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1cb250 con 0x7fa5cc0a19d0 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc09e480 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5d1247700 10 monclient(hunting): authenticate will time out at 2683.916129s 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa5cc0a1db0 0x7fa5cc1349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa5cc0a1db0 0x7fa5cc1349a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42802/0 (socket says 172.21.15.47:42802) 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 -- 172.21.15.47:0/2731692831 learned_addr learned my addr 172.21.15.47:0/2731692831 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): get_auth_request con 0x7fa5cc0a1db0 auth_method 0 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc0a19d0 0x7fa5cc09d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): get_auth_request con 0x7fa5cc0a19d0 auth_method 0 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc13b150 0x7fa5cc11e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): get_auth_request con 0x7fa5cc13b150 auth_method 0 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5cb59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient(hunting): handle_auth_done global_id 36161 payload 824 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc0a19d0 msgr2=0x7fa5cc09d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa5cc0a19d0 0x7fa5cc09d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc13b150 msgr2=0x7fa5cc11e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa5cc13b150 0x7fa5cc11e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 monclient: found mon.noname-e 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00094a0 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5ca59c700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa5cc0a1db0 0x7fa5cc1349a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000ebe0 tx=0x7fa5c000eef0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5b37fe700 10 client.?.objecter ms_handle_connect 0x7fa5cc0a1db0 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5b37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.515+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5c000cc60 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: dump: 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.538 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 5 monclient: authenticate success, global_id 36161 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c000cdc0 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5c00138e0 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 monclient: _renew_subs 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c4009a70 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5c959a700 4 set_mon_vals no callback set 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 10 monclient: dump: 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.540 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 client.36161.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 monclient: _renew_subs 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc120ab0 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 1 librados: init done 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 0x7fa5cc02bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc068430 con 0x7fa5cc11fe90 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.516+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa5c0013470 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 0x7fa5cc02bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient: get_auth_request con 0x7fa5cc11fe90 auth_method 0 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5b803a320 0x7fa5b803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(682..682 src has 266..682) v4 ==== 6674+0+0 (secure 0 0 0) 0x7fa5c004aa40 con 0x7fa5cc0a1db0 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 client.36161.objecter ms_dispatch 0x7fa5cc001550 osd_map(682..682 src has 266..682) v4 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 3 client.36161.objecter handle_osd_map got epochs [682,682] > 0 2024-04-20T02:33:24.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 3 client.36161.objecter handle_osd_map decoding full epoch 682 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 20 client.36161.objecter dump_active .. 0 homeless 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cb59e700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5b803a320 0x7fa5b803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cb59e700 10 monclient: get_auth_request con 0x7fa5b803a320 auth_method 0 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_done global_id 55756 payload 824 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5cc072300 con 0x7fa5cc11fe90 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cad9d700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 0x7fa5cc02bbc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005910 tx=0x7fa5bc00fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 client.36161.objecter ms_handle_connect 0x7fa5cc11fe90 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 client.36161.objecter resend_mon_ops 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5cb59e700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5b803a320 0x7fa5b803c7e0 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009dc0 tx=0x7fa5c4011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 client.36161.objecter ms_handle_connect 0x7fa5b803a320 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5bc0070d0 con 0x7fa5cc11fe90 2024-04-20T02:33:24.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.517+0000 7fa5b37fe700 10 monclient: dump: 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.544 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa5bc005b50 con 0x7fa5cc11fe90 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 msgr2=0x7fa5cc02bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 0x7fa5cc02bbc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005910 tx=0x7fa5bc00fb00 comp rx=0 tx=0).stop 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 msgr2=0x7fa5cc02bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.518+0000 7fa5b37fe700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc11fe90 0x7fa5cc02bbc0 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc005910 tx=0x7fa5bc00fb00 comp rx=0 tx=0).stop 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5d1247700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 0x7fa5cc02c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5d1247700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc06a400 con 0x7fa5cc06aac0 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 0x7fa5cc02c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient: get_auth_request con 0x7fa5cc06aac0 auth_method 0 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 10 monclient(hunting): handle_auth_done global_id 55765 payload 824 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.526+0000 7fa5cad9d700 1 -- 172.21.15.47:0/2731692831 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa5cc0a4480 con 0x7fa5cc06aac0 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 client.36161.objecter ms_handle_connect 0x7fa5cc06aac0 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 client.36161.objecter resend_mon_ops 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5cad9d700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 0x7fa5cc02c100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc017000 tx=0x7fa5bc0010a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa5bc023070 con 0x7fa5cc06aac0 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 monclient: dump: 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.549 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9356 (secure 0 0 0) 0x7fa5bc027410 con 0x7fa5cc06aac0 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 msgr2=0x7fa5cc02c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 0x7fa5cc02c100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc017000 tx=0x7fa5bc0010a0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 msgr2=0x7fa5cc02c100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5b37fe700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5cc06aac0 0x7fa5cc02c100 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc017000 tx=0x7fa5bc0010a0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:24.551 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.553 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.555 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:24.556 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.557 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:24.559 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.560 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:24.562 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.563 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.566 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.567 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.569 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.570 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:24.571 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 10 librados: watch_flush enter 2024-04-20T02:33:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 10 librados: watch_flush exit 2024-04-20T02:33:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 20 client.36161.objecter shutdown clearing up homeless session... 2024-04-20T02:33:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 10 client.36161.objecter successfully canceled tick 2024-04-20T02:33:24.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5b803a320 msgr2=0x7fa5b803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa5b803a320 0x7fa5b803c7e0 secure :-1 s=READY pgs=3370 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009dc0 tx=0x7fa5c4011000 comp rx=0 tx=0).stop 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 10 monclient: shutdown 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa5cc0a1db0 msgr2=0x7fa5cc1349a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 1 --2- 172.21.15.47:0/2731692831 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa5cc0a1db0 0x7fa5cc1349a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000ebe0 tx=0x7fa5c000eef0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5cb59e700 1 -- 172.21.15.47:0/2731692831 reap_dead start 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5cb59e700 1 -- 172.21.15.47:0/2731692831 reap_dead start 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 shutdown_connections 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.527+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 >> 172.21.15.47:0/2731692831 conn(0x7fa5cc0a4a50 msgr2=0x7fa5cc0a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.528+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 shutdown_connections 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.528+0000 7fa5d1247700 1 -- 172.21.15.47:0/2731692831 wait complete. 2024-04-20T02:33:24.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.528+0000 7fa5d1247700 1 librados: shutdown 2024-04-20T02:33:24.575 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:24.575 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:33:24.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:33:24.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.717+0000 7f99dedbc700 10 monclient: get_monmap_and_config 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.717+0000 7f99dedbc700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.717+0000 7f99dedbc700 10 monclient: monmap: 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:24.718297+0000 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:24.718297+0000 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.729 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.717+0000 7f99dedbc700 1 Processor -- start 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 -- start start 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient: init 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient: _add_conns ranks=[2,6,10,1,5,7,0,8,9,3,4] 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99d8137d90 0x7f99d80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): picked mon.noname-b con 0x7f99d8137d90 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 0x7f99d80a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): picked mon.noname-d con 0x7f99d8138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f99d81385b0 0x7f99d80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): picked mon.noname-f con 0x7f99d81385b0 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d809fa40 con 0x7f99d81385b0 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d809dfc0 con 0x7f99d8137d90 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99d80a7bc0 con 0x7f99d8138170 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dedbc700 10 monclient(hunting): authenticate will time out at 2684.119567s 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 0x7f99d80a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 0x7f99d80a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50836/0 (socket says 172.21.15.47:50836) 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99d8137d90 0x7f99d80a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 1 -- 172.21.15.47:0/3528051663 learned_addr learned my addr 172.21.15.47:0/3528051663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): get_auth_request con 0x7f99d8138170 auth_method 0 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): get_auth_request con 0x7f99d8137d90 auth_method 0 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dcb58700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dd359700 1 --2- 172.21.15.47:0/3528051663 >> v2:172.21.15.179:3300/0 conn(0x7f99d81385b0 0x7f99d80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dd359700 10 monclient(hunting): get_auth_request con 0x7f99d81385b0 auth_method 0 2024-04-20T02:33:24.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dd359700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dd359700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.718+0000 7f99dd359700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 10 monclient(hunting): handle_auth_done global_id 55365 payload 274 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 -- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99d8137d90 msgr2=0x7f99d80a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 --2- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99d8137d90 0x7f99d80a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 -- 172.21.15.47:0/3528051663 >> v2:172.21.15.179:3300/0 conn(0x7f99d81385b0 msgr2=0x7f99d80a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 --2- 172.21.15.47:0/3528051663 >> v2:172.21.15.179:3300/0 conn(0x7f99d81385b0 0x7f99d80a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 monclient: found mon.noname-d 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 -- 172.21.15.47:0/3528051663 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d80751e0 con 0x7f99d8138170 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99dcb58700 1 --2- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99d8137d90 0x7f99d80a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99dd359700 1 --2- 172.21.15.47:0/3528051663 >> v2:172.21.15.179:3300/0 conn(0x7f99d81385b0 0x7f99d80a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d7fff700 1 --2- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 0x7f99d80a5c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009920 tx=0x7f99c0009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7484e060ad3bfd28 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 1 -- 172.21.15.47:0/3528051663 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99c0011aa0 con 0x7f99d8138170 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: dump: 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.734 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 1 -- 172.21.15.47:0/3528051663 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c0011c00 con 0x7f99d8138170 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99dedbc700 5 monclient: authenticate success, global_id 55365 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99dedbc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99dedbc700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 1 -- 172.21.15.47:0/3528051663 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99c0011d70 con 0x7f99d8138170 2024-04-20T02:33:24.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99dedbc700 4 set_mon_vals no callback set 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.719+0000 7f99d77fe700 10 monclient: dump: 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.736 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 10 monclient: shutdown 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 -- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 msgr2=0x7f99d80a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 --2- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 0x7f99d80a5c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009920 tx=0x7f99c0009c30 comp rx=0 tx=0).stop 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 -- 172.21.15.47:0/3528051663 shutdown_connections 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 --2- 172.21.15.47:0/3528051663 >> v2:172.21.15.179:3300/0 conn(0x7f99d81385b0 0x7f99d80a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 --2- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f99d8137d90 0x7f99d80a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 --2- 172.21.15.47:0/3528051663 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8138170 0x7f99d80a5c90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 -- 172.21.15.47:0/3528051663 >> 172.21.15.47:0/3528051663 conn(0x7f99d80aa8b0 msgr2=0x7f99d80a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 -- 172.21.15.47:0/3528051663 shutdown_connections 2024-04-20T02:33:24.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 1 -- 172.21.15.47:0/3528051663 wait complete. 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.720+0000 7f99dedbc700 10 monclient: monmap: 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.738 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 librados: starting msgr at 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 librados: starting objecter 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 Processor -- start 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 -- start start 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 librados: setting wanted keys 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 librados: calling monclient init 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 10 monclient: init 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 10 monclient: _add_conns ranks=[6,8,7,4,0,5,3,2,1] 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.721+0000 7f99dedbc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8137d90 0x7f99d81cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): picked mon.noname-d con 0x7f99d8137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:24.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f99d8138170 0x7f99d8136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): picked mon.noname-e con 0x7f99d8138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99d81385b0 0x7f99d8137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): picked mon.noname-i con 0x7f99d81385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8137860 con 0x7f99d8137d90 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8122fa0 con 0x7f99d81385b0 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f99d809e9e0 con 0x7f99d8138170 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dedbc700 10 monclient(hunting): authenticate will time out at 2684.123082s 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8137d90 0x7f99d81cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f99d8138170 0x7f99d8136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8137d90 0x7f99d81cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50842/0 (socket says 172.21.15.47:50842) 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 -- 172.21.15.47:0/742208422 learned_addr learned my addr 172.21.15.47:0/742208422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): get_auth_request con 0x7f99d8137d90 auth_method 0 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): get_auth_request con 0x7f99d8138170 auth_method 0 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dd359700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99d81385b0 0x7f99d8137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dd359700 10 monclient(hunting): get_auth_request con 0x7f99d81385b0 auth_method 0 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dd359700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dd359700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dd359700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient(hunting): handle_auth_done global_id 55374 payload 824 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f99d8138170 msgr2=0x7f99d8136de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f99d8138170 0x7f99d8136de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99d81385b0 msgr2=0x7f99d8137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99d81385b0 0x7f99d8137320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 monclient: found mon.noname-d 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:24.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80094a0 con 0x7f99d8137d90 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dd359700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f99d81385b0 0x7f99d8137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d7fff700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f99d8138170 0x7f99d8136de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99dcb58700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8137d90 0x7f99d81cc190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f99c800ebe0 tx=0x7f99c800eef0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 client.?.objecter ms_handle_connect 0x7f99d8137d90 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99c8009780 con 0x7f99d8137d90 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: dump: 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.743 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c800ccc0 con 0x7f99d8137d90 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99c800ce30 con 0x7f99d8137d90 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d6ffd700 4 set_mon_vals no callback set 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.722+0000 7f99d57fa700 10 monclient: dump: 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.745 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 5 monclient: authenticate success, global_id 55374 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 monclient: _renew_subs 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c0009550 con 0x7f99d8137d90 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 client.55374.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:24.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 monclient: _renew_subs 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d81244c0 con 0x7f99d8137d90 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 1 librados: init done 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 0x7f99d81208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8075710 con 0x7f99d806aac0 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.723+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f99c800c850 con 0x7f99d8137d90 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f99b803a2f0 0x7f99b803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 0x7f99d81208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient: get_auth_request con 0x7f99d806aac0 auth_method 0 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(682..682 src has 266..682) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f99c804ab60 con 0x7f99d8137d90 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 10 client.55374.objecter ms_dispatch 0x7f99d8001550 osd_map(682..682 src has 266..682) v4 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 3 client.55374.objecter handle_osd_map got epochs [682,682] > 0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 3 client.55374.objecter handle_osd_map decoding full epoch 682 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 20 client.55374.objecter dump_active .. 0 homeless 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99dd359700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f99b803a2f0 0x7f99b803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99dd359700 10 monclient: get_auth_request con 0x7f99b803a2f0 auth_method 0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 10 monclient(hunting): handle_auth_done global_id 104169 payload 824 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99d809f760 con 0x7f99d806aac0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 10 client.55374.objecter ms_handle_connect 0x7f99b803a2f0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99dd359700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f99b803a2f0 0x7f99b803c7b0 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7f99d81227c0 tx=0x7f99d0003ac0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 10 client.55374.objecter ms_handle_connect 0x7f99d806aac0 2024-04-20T02:33:24.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d57fa700 10 client.55374.objecter resend_mon_ops 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.724+0000 7f99d7fff700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 0x7f99d81208d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99d809e9b0 tx=0x7f99c0004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99c0020070 con 0x7f99d806aac0 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 10 monclient: dump: 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f99c00036a0 con 0x7f99d806aac0 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 msgr2=0x7f99d81208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 0x7f99d81208d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99d809e9b0 tx=0x7f99c0004750 comp rx=0 tx=0).stop 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 msgr2=0x7f99d81208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.725+0000 7f99d57fa700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d806aac0 0x7f99d81208d0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99d809e9b0 tx=0x7f99c0004750 comp rx=0 tx=0).stop 2024-04-20T02:33:24.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.736+0000 7f99dedbc700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.736+0000 7f99dedbc700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 0x7f99d81211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.736+0000 7f99dedbc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.736+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8068180 con 0x7f99d8120e10 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 0x7f99d81211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient: get_auth_request con 0x7f99d8120e10 auth_method 0 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.738+0000 7f99d7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d7fff700 10 monclient(hunting): handle_auth_done global_id 104178 payload 824 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d7fff700 1 -- 172.21.15.47:0/742208422 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f99d80a99f0 con 0x7f99d8120e10 2024-04-20T02:33:24.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d57fa700 10 client.55374.objecter ms_handle_connect 0x7f99d8120e10 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d7fff700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 0x7f99d81211f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99c0000c00 tx=0x7f99c0011000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d57fa700 10 client.55374.objecter resend_mon_ops 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99c002c300 con 0x7f99d8120e10 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d57fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.739+0000 7f99d57fa700 10 monclient: dump: 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.754 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f99c0026900 con 0x7f99d8120e10 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 msgr2=0x7f99d81211f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 0x7f99d81211f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99c0000c00 tx=0x7f99c0011000 comp rx=0 tx=0).stop 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 msgr2=0x7f99d81211f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99d57fa700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f99d8120e10 0x7f99d81211f0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99c0000c00 tx=0x7f99c0011000 comp rx=0 tx=0).stop 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:24.756 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.757 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.758 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:24.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.760 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.762 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:24.764 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.766 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.769 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.771 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.774 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.775 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.776 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.776 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:24.776 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 10 librados: watch_flush enter 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 10 librados: watch_flush exit 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 20 client.55374.objecter shutdown clearing up homeless session... 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 10 client.55374.objecter successfully canceled tick 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f99b803a2f0 msgr2=0x7f99b803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f99b803a2f0 0x7f99b803c7b0 secure :-1 s=READY pgs=3371 cs=0 l=1 rev1=1 crypto rx=0x7f99d81227c0 tx=0x7f99d0003ac0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 10 monclient: shutdown 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8137d90 msgr2=0x7f99d81cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 --2- 172.21.15.47:0/742208422 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f99d8137d90 0x7f99d81cc190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f99c800ebe0 tx=0x7f99c800eef0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dd359700 1 -- 172.21.15.47:0/742208422 reap_dead start 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dd359700 1 -- 172.21.15.47:0/742208422 reap_dead start 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 shutdown_connections 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 >> 172.21.15.47:0/742208422 conn(0x7f99d80aa8b0 msgr2=0x7f99d809d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 shutdown_connections 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 -- 172.21.15.47:0/742208422 wait complete. 2024-04-20T02:33:24.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.740+0000 7f99dedbc700 1 librados: shutdown 2024-04-20T02:33:24.779 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:24.779 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:33:24.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.917+0000 7f2594489700 10 monclient: get_monmap_and_config 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.917+0000 7f2594489700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.917+0000 7f2594489700 10 monclient: monmap: 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:24.919093+0000 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:24.919093+0000 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.918 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:24.919 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.918+0000 7f2594489700 1 Processor -- start 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.918+0000 7f2594489700 1 -- start start 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.918+0000 7f2594489700 10 monclient: init 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient: _add_conns ranks=[7,2,4,10,5,6,3,1,8,0,9] 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f258c0a19d0 0x7f258c0a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): picked mon.noname-i con 0x7f258c0a19d0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 0x7f258c0701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): picked mon.noname-b con 0x7f258c135f00 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 0x7f258c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): picked mon.noname-c con 0x7f258c136380 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0a4040 con 0x7f258c135f00 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f258c072020 con 0x7f258c136380 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f258c072300 con 0x7f258c0a19d0 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2594489700 10 monclient(hunting): authenticate will time out at 2684.320369s 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 0x7f258c0701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 0x7f258c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 0x7f258c0706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58026/0 (socket says 172.21.15.47:58026) 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 0x7f258c0701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43280/0 (socket says 172.21.15.47:43280) 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 -- 172.21.15.47:0/1361424291 learned_addr learned my addr 172.21.15.47:0/1361424291 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): get_auth_request con 0x7f258c136380 auth_method 0 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): get_auth_request con 0x7f258c135f00 auth_method 0 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f258c0a19d0 0x7f258c0a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): get_auth_request con 0x7f258c0a19d0 auth_method 0 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592225700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient(hunting): handle_auth_done global_id 84550 payload 274 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 -- 172.21.15.47:0/1361424291 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f258c0a19d0 msgr2=0x7f258c0a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f258c0a19d0 0x7f258c0a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 -- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 msgr2=0x7f258c0701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 0x7f258c0701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 monclient: found mon.noname-c 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2592a26700 1 -- 172.21.15.47:0/1361424291 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0751e0 con 0x7f258c136380 2024-04-20T02:33:24.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.919+0000 7f2591a24700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 0x7f258c0701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2592a26700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 0x7f258c0706f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2584009e80 tx=0x7f258400b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=68c21292179bc7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 1 -- 172.21.15.47:0/1361424291 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f258400e070 con 0x7f258c136380 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: dump: 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.935 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 5 monclient: authenticate success, global_id 84550 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 1 -- 172.21.15.47:0/1361424291 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258400bd80 con 0x7f258c136380 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 4 set_mon_vals no callback set 2024-04-20T02:33:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 1 -- 172.21.15.47:0/1361424291 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2584012860 con 0x7f258c136380 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2591223700 10 monclient: dump: 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 10 monclient: shutdown 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 -- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 msgr2=0x7f258c0706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 0x7f258c0706f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2584009e80 tx=0x7f258400b330 comp rx=0 tx=0).stop 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 -- 172.21.15.47:0/1361424291 shutdown_connections 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c136380 0x7f258c0706f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f258c0a19d0 0x7f258c0a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 --2- 172.21.15.47:0/1361424291 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f258c135f00 0x7f258c0701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 -- 172.21.15.47:0/1361424291 >> 172.21.15.47:0/1361424291 conn(0x7f258c0a4a50 msgr2=0x7f258c0a4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.920+0000 7f2594489700 1 -- 172.21.15.47:0/1361424291 shutdown_connections 2024-04-20T02:33:24.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.921+0000 7f2594489700 1 -- 172.21.15.47:0/1361424291 wait complete. 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.921+0000 7f2594489700 10 monclient: build_initial_monmap 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.921+0000 7f2594489700 10 monclient: monmap: 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:24.939 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.921+0000 7f2594489700 1 librados: starting msgr at 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.921+0000 7f2594489700 1 librados: starting objecter 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 Processor -- start 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 -- start start 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 librados: setting wanted keys 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 librados: calling monclient init 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient: init 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient: _add_conns ranks=[8,4,6,5,0,3,1,7,2] 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f258c0a19d0 0x7f258c1ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): picked mon.noname-e con 0x7f258c0a19d0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c135f00 0x7f258c134a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): picked mon.noname-c con 0x7f258c135f00 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:24.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f258c136380 0x7f258c134fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): picked mon.noname-d con 0x7f258c136380 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1cb050 con 0x7f258c135f00 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1ca870 con 0x7f258c136380 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f258c071230 con 0x7f258c0a19d0 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2594489700 10 monclient(hunting): authenticate will time out at 2684.323684s 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f258c0a19d0 0x7f258c1ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f258c0a19d0 0x7f258c1ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42818/0 (socket says 172.21.15.47:42818) 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f258c136380 0x7f258c134fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 1 -- 172.21.15.47:0/3674601881 learned_addr learned my addr 172.21.15.47:0/3674601881 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): get_auth_request con 0x7f258c0a19d0 auth_method 0 2024-04-20T02:33:24.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c135f00 0x7f258c134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): get_auth_request con 0x7f258c136380 auth_method 0 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): get_auth_request con 0x7f258c135f00 auth_method 0 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592225700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.922+0000 7f2592a26700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 10 monclient(hunting): handle_auth_done global_id 36179 payload 824 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f258c136380 msgr2=0x7f258c134fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f258c136380 0x7f258c134fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c135f00 msgr2=0x7f258c134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c135f00 0x7f258c134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 10 monclient: _finish_hunting 0 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 monclient: found mon.noname-e 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0094a0 con 0x7f258c0a19d0 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2591a24700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f258c135f00 0x7f258c134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2592225700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f258c0a19d0 0x7f258c1ca330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f258800e930 tx=0x7f258800ec40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 client.?.objecter ms_handle_connect 0x7f258c0a19d0 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2588013af0 con 0x7f258c0a19d0 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: dump: 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.944 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 5 monclient: authenticate success, global_id 36179 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588013c90 con 0x7f258c0a19d0 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:24.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2588013e00 con 0x7f258c0a19d0 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 10 monclient: _renew_subs 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2584009b30 con 0x7f258c0a19d0 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2590a22700 4 set_mon_vals no callback set 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 10 client.36179.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2582ffd700 10 monclient: dump: 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.946 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 10 monclient: _renew_subs 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c120cf0 con 0x7f258c0a19d0 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.923+0000 7f2594489700 1 librados: init done 2024-04-20T02:33:24.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2594489700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2594489700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 0x7f258c11e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f258c068430 con 0x7f258c06aac0 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2588013680 con 0x7f258c0a19d0 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 0x7f258c11e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient: get_auth_request con 0x7f258c06aac0 auth_method 0 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2582ffd700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f257003a2f0 0x7f257003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(683..683 src has 266..683) v4 ==== 6669+0+0 (secure 0 0 0) 0x7f258804a810 con 0x7f258c0a19d0 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2582ffd700 10 client.36179.objecter ms_dispatch 0x7f258c001550 osd_map(683..683 src has 266..683) v4 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2582ffd700 3 client.36179.objecter handle_osd_map got epochs [683,683] > 0 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2582ffd700 3 client.36179.objecter handle_osd_map decoding full epoch 683 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.924+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 20 client.36179.objecter dump_active .. 0 homeless 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2592a26700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f257003a2f0 0x7f257003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2592a26700 10 monclient: get_auth_request con 0x7f257003a2f0 auth_method 0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2591a24700 10 monclient(hunting): handle_auth_done global_id 84542 payload 824 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2591a24700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f258c0a3c30 con 0x7f258c06aac0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 client.36179.objecter ms_handle_connect 0x7f258c06aac0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 client.36179.objecter resend_mon_ops 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2591a24700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 0x7f258c11e5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f258c09e960 tx=0x7f257c015700 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 client.36179.objecter ms_handle_connect 0x7f257003a2f0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2592a26700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f257003a2f0 0x7f257003c7b0 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7f2584009e80 tx=0x7f2584009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f257c01b8f0 con 0x7f258c06aac0 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:24.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 monclient: dump: 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f257c016040 con 0x7f258c06aac0 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 msgr2=0x7f258c11e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 0x7f258c11e5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f258c09e960 tx=0x7f257c015700 comp rx=0 tx=0).stop 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 msgr2=0x7f258c11e5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.925+0000 7f2582ffd700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c06aac0 0x7f258c11e5a0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f258c09e960 tx=0x7f257c015700 comp rx=0 tx=0).stop 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.933+0000 7f2594489700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.933+0000 7f2594489700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 0x7f258c11eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.933+0000 7f2594489700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.933+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f258c06a400 con 0x7f258c11eae0 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.933+0000 7f2591a24700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 0x7f258c11eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient: get_auth_request con 0x7f258c11eae0 auth_method 0 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:24.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 10 monclient(hunting): handle_auth_done global_id 84551 payload 824 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 1 -- 172.21.15.47:0/3674601881 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f258c070c30 con 0x7f258c11eae0 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2591a24700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 0x7f258c11eef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f257c015740 tx=0x7f257c0047c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2582ffd700 10 client.36179.objecter ms_handle_connect 0x7f258c11eae0 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2582ffd700 10 client.36179.objecter resend_mon_ops 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f257c004490 con 0x7f258c11eae0 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2582ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2582ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.934+0000 7f2582ffd700 10 monclient: dump: 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:24.954 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:24.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9524 (secure 0 0 0) 0x7f257c0260d0 con 0x7f258c11eae0 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 msgr2=0x7f258c11eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 0x7f258c11eef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f257c015740 tx=0x7f257c0047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 msgr2=0x7f258c11eef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2582ffd700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f258c11eae0 0x7f258c11eef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:24.956 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.957 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:24.958 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:24.959 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:24.960 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.961 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.962 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:24.963 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.964 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.965 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.967 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.970 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.971 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:24.972 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.973 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:24.975 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.976 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:24.977 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:24.978 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:24.978 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:24.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:24.978 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:24.978 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:24.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 10 librados: watch_flush enter 2024-04-20T02:33:24.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 10 librados: watch_flush exit 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 20 client.36179.objecter shutdown clearing up homeless session... 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 10 client.36179.objecter successfully canceled tick 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f257003a2f0 msgr2=0x7f257003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f257003a2f0 0x7f257003c7b0 secure :-1 s=READY pgs=3372 cs=0 l=1 rev1=1 crypto rx=0x7f2584009e80 tx=0x7f2584009000 comp rx=0 tx=0).stop 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 10 monclient: shutdown 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f258c0a19d0 msgr2=0x7f258c1ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 --2- 172.21.15.47:0/3674601881 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f258c0a19d0 0x7f258c1ca330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f258800e930 tx=0x7f258800ec40 comp rx=0 tx=0).stop 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2592a26700 1 -- 172.21.15.47:0/3674601881 reap_dead start 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2592a26700 1 -- 172.21.15.47:0/3674601881 reap_dead start 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 shutdown_connections 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 >> 172.21.15.47:0/3674601881 conn(0x7f258c0a4a50 msgr2=0x7f258c1bd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 shutdown_connections 2024-04-20T02:33:24.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 -- 172.21.15.47:0/3674601881 wait complete. 2024-04-20T02:33:24.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:24.935+0000 7f2594489700 1 librados: shutdown 2024-04-20T02:33:24.981 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:24.981 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:33:24.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:33:25.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.119+0000 7fa368d9b700 10 monclient: get_monmap_and_config 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.119+0000 7fa368d9b700 10 monclient: build_initial_monmap 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.119+0000 7fa368d9b700 10 monclient: monmap: 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:25.121065+0000 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:25.121065+0000 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:25.120 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:25.121 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:25.121 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.120+0000 7fa368d9b700 1 Processor -- start 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.120+0000 7fa368d9b700 1 -- start start 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.120+0000 7fa368d9b700 10 monclient: init 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient: _add_conns ranks=[8,5,7,1,2,4,6,0,10,3,9] 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 0x7fa3640a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): picked mon.noname-e con 0x7fa3640a07f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa3640a0bd0 0x7fa3640a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): picked mon.noname-h con 0x7fa3640a0bd0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa3640a1050 0x7fa3640a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): picked mon.noname-i con 0x7fa3640a1050 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa364067800 con 0x7fa3640a0bd0 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640672f0 con 0x7fa3640a1050 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa36409ed90 con 0x7fa3640a07f0 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa368d9b700 10 monclient(hunting): authenticate will time out at 2684.522383s 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 0x7fa3640a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 0x7fa3640a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42822/0 (socket says 172.21.15.47:42822) 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 -- 172.21.15.47:0/3176280327 learned_addr learned my addr 172.21.15.47:0/3176280327 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): get_auth_request con 0x7fa3640a07f0 auth_method 0 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa362d9d700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa3640a1050 0x7fa3640a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa361d9b700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa3640a0bd0 0x7fa3640a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa362d9d700 10 monclient(hunting): get_auth_request con 0x7fa3640a1050 auth_method 0 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa362d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa362d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa362d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa361d9b700 10 monclient(hunting): get_auth_request con 0x7fa3640a0bd0 auth_method 0 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa361d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa361d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa361d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient(hunting): handle_auth_done global_id 36188 payload 274 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 -- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa3640a1050 msgr2=0x7fa3640a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa3640a1050 0x7fa3640a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 -- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa3640a0bd0 msgr2=0x7fa3640a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa3640a0bd0 0x7fa3640a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 monclient: found mon.noname-e 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa36259c700 1 -- 172.21.15.47:0/3176280327 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640751e0 con 0x7fa3640a07f0 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.121+0000 7fa361d9b700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa3640a0bd0 0x7fa3640a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36259c700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 0x7fa3640a0010 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa354009920 tx=0x7fa354009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=772656a40f6cb9de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 1 -- 172.21.15.47:0/3176280327 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa354009e70 con 0x7fa3640a07f0 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: dump: 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.141 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: _finish_auth 0 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: _check_auth_tickets 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 5 monclient: authenticate success, global_id 36188 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 1 -- 172.21.15.47:0/3176280327 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35400bcd0 con 0x7fa3640a07f0 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 4 set_mon_vals no callback set 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 1 -- 172.21.15.47:0/3176280327 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa35400be40 con 0x7fa3640a07f0 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa36159a700 10 monclient: dump: 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.143 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.144 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 10 monclient: shutdown 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 -- 172.21.15.47:0/3176280327 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 msgr2=0x7fa3640a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 0x7fa3640a0010 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa354009920 tx=0x7fa354009c30 comp rx=0 tx=0).stop 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 -- 172.21.15.47:0/3176280327 shutdown_connections 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa3640a1050 0x7fa3640a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a07f0 0x7fa3640a0010 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 --2- 172.21.15.47:0/3176280327 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa3640a0bd0 0x7fa3640a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 -- 172.21.15.47:0/3176280327 >> 172.21.15.47:0/3176280327 conn(0x7fa3640a3f90 msgr2=0x7fa36406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 -- 172.21.15.47:0/3176280327 shutdown_connections 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.122+0000 7fa368d9b700 1 -- 172.21.15.47:0/3176280327 wait complete. 2024-04-20T02:33:25.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.123+0000 7fa368d9b700 10 monclient: build_initial_monmap 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.123+0000 7fa368d9b700 10 monclient: monmap: 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:25.146 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.123+0000 7fa368d9b700 1 librados: starting msgr at 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.123+0000 7fa368d9b700 1 librados: starting objecter 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 Processor -- start 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 -- start start 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 librados: setting wanted keys 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 librados: calling monclient init 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient: init 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient: _add_conns ranks=[1,8,2,5,3,6,4,7,0] 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3640a07f0 0x7fa3641a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): picked mon.noname-f con 0x7fa3640a07f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:25.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a0bd0 0x7fa3641c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): picked mon.noname-e con 0x7fa3640a0bd0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3640a1050 0x7fa3641bac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): picked mon.noname-b con 0x7fa3640a1050 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641c9b30 con 0x7fa3640a07f0 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641a83d0 con 0x7fa3640a1050 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa36409e360 con 0x7fa3640a0bd0 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa368d9b700 10 monclient(hunting): authenticate will time out at 2684.525701s 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a0bd0 0x7fa3641c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3640a1050 0x7fa3641bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a0bd0 0x7fa3641c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42830/0 (socket says 172.21.15.47:42830) 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 1 -- 172.21.15.47:0/2858190285 learned_addr learned my addr 172.21.15.47:0/2858190285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): get_auth_request con 0x7fa3640a0bd0 auth_method 0 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): get_auth_request con 0x7fa3640a1050 auth_method 0 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa361d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa362d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.124+0000 7fa36259c700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3640a07f0 0x7fa3641a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): get_auth_request con 0x7fa3640a07f0 auth_method 0 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 10 monclient(hunting): handle_auth_done global_id 36197 payload 824 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3640a1050 msgr2=0x7fa3641bac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa3640a1050 0x7fa3641bac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3640a07f0 msgr2=0x7fa3641a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3640a07f0 0x7fa3641a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 monclient: found mon.noname-e 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009550 con 0x7fa3640a0bd0 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa361d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a0bd0 0x7fa3641c95f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009fd0 tx=0x7fa34c00eac0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 client.?.objecter ms_handle_connect 0x7fa3640a0bd0 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa34c013880 con 0x7fa3640a0bd0 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: dump: 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: _finish_auth 0 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:25.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c013a00 con 0x7fa3640a0bd0 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 5 monclient: authenticate success, global_id 36197 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 10 monclient: _renew_subs 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36409cfb0 con 0x7fa3640a0bd0 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 10 client.36197.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 10 monclient: _renew_subs 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3641a7d80 con 0x7fa3640a0bd0 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa368d9b700 1 librados: init done 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa34c012370 con 0x7fa3640a0bd0 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa35affd700 10 monclient: dump: 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.154 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.125+0000 7fa360d99700 4 set_mon_vals no callback set 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa368d9b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa368d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 0x7fa348004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa348007d10 con 0x7fa348002850 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa34c013b70 con 0x7fa3640a0bd0 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa35affd700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa34403a2f0 0x7fa34403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa36259c700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 0x7fa348004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa36259c700 10 monclient: get_auth_request con 0x7fa348002850 auth_method 0 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa36259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa36259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa36259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.126+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(683..683 src has 266..683) v4 ==== 6669+0+0 (secure 0 0 0) 0x7fa34c04a390 con 0x7fa3640a0bd0 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 client.36197.objecter ms_dispatch 0x7fa364001550 osd_map(683..683 src has 266..683) v4 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 3 client.36197.objecter handle_osd_map got epochs [683,683] > 0 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 3 client.36197.objecter handle_osd_map decoding full epoch 683 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa362d9d700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa34403a2f0 0x7fa34403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 20 client.36197.objecter dump_active .. 0 homeless 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa362d9d700 10 monclient: get_auth_request con 0x7fa34403a2f0 auth_method 0 2024-04-20T02:33:25.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa36259c700 10 monclient(hunting): handle_auth_done global_id 84589 payload 824 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa36259c700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa348007ed0 con 0x7fa348002850 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 client.36197.objecter ms_handle_connect 0x7fa348002850 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 client.36197.objecter resend_mon_ops 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa36259c700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 0x7fa348004cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fd0 tx=0x7fa354004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 client.36197.objecter ms_handle_connect 0x7fa34403a2f0 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa362d9d700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa34403a2f0 0x7fa34403c7b0 secure :-1 s=READY pgs=3373 cs=0 l=1 rev1=1 crypto rx=0x7fa3641a67a0 tx=0x7fa35c009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa354020070 con 0x7fa348002850 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.127+0000 7fa35affd700 10 monclient: dump: 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.157 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa35400bb60 con 0x7fa348002850 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:25.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 msgr2=0x7fa348004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 0x7fa348004cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fd0 tx=0x7fa354004750 comp rx=0 tx=0).stop 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 msgr2=0x7fa348004cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.128+0000 7fa35affd700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348002850 0x7fa348004cf0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fd0 tx=0x7fa354004750 comp rx=0 tx=0).stop 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.135+0000 7fa368d9b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.135+0000 7fa368d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 0x7fa3480071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.135+0000 7fa368d9b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.135+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800a1d0 con 0x7fa348006d50 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 0x7fa3480071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient: get_auth_request con 0x7fa348006d50 auth_method 0 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 10 monclient(hunting): handle_auth_done global_id 84598 payload 824 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 1 -- 172.21.15.47:0/2858190285 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa34800a390 con 0x7fa348006d50 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa35affd700 10 client.36197.objecter ms_handle_connect 0x7fa348006d50 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa36259c700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 0x7fa3480071c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa354009130 tx=0x7fa354011000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.136+0000 7fa35affd700 10 client.36197.objecter resend_mon_ops 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa35400be40 con 0x7fa348006d50 2024-04-20T02:33:25.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 10 monclient: dump: 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.161 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7fa3540045e0 con 0x7fa348006d50 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 msgr2=0x7fa3480071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 0x7fa3480071c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa354009130 tx=0x7fa354011000 comp rx=0 tx=0).stop 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 msgr2=0x7fa3480071c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:25.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa35affd700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa348006d50 0x7fa3480071c0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa354009130 tx=0x7fa354011000 comp rx=0 tx=0).stop 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:25.163 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:25.164 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:25.165 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:25.166 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:25.167 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:25.169 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.170 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:25.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.173 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.174 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.175 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:25.176 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:25.177 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.179 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.180 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:25.182 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:25.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:25.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.183 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:25.183 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:25.184 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:25.185 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 10 librados: watch_flush enter 2024-04-20T02:33:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 10 librados: watch_flush exit 2024-04-20T02:33:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 20 client.36197.objecter shutdown clearing up homeless session... 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 10 client.36197.objecter successfully canceled tick 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa34403a2f0 msgr2=0x7fa34403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa34403a2f0 0x7fa34403c7b0 secure :-1 s=READY pgs=3373 cs=0 l=1 rev1=1 crypto rx=0x7fa3641a67a0 tx=0x7fa35c009780 comp rx=0 tx=0).stop 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 10 monclient: shutdown 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a0bd0 msgr2=0x7fa3641c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 --2- 172.21.15.47:0/2858190285 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa3640a0bd0 0x7fa3641c95f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009fd0 tx=0x7fa34c00eac0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa362d9d700 1 -- 172.21.15.47:0/2858190285 reap_dead start 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa362d9d700 1 -- 172.21.15.47:0/2858190285 reap_dead start 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 shutdown_connections 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 >> 172.21.15.47:0/2858190285 conn(0x7fa3640a3f90 msgr2=0x7fa3641bd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 shutdown_connections 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 -- 172.21.15.47:0/2858190285 wait complete. 2024-04-20T02:33:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.137+0000 7fa368d9b700 1 librados: shutdown 2024-04-20T02:33:25.188 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:25.188 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:33:25.188 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:33:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.325+0000 7f6e6ec24700 10 monclient: get_monmap_and_config 2024-04-20T02:33:25.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.325+0000 7f6e6ec24700 10 monclient: build_initial_monmap 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.325+0000 7f6e6ec24700 10 monclient: monmap: 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:25.326590+0000 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:25.326590+0000 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:25.326 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:25.327 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:25.327 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.326+0000 7f6e6ec24700 1 Processor -- start 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.326+0000 7f6e6ec24700 1 -- start start 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.326+0000 7f6e6ec24700 10 monclient: init 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient: _add_conns ranks=[1,8,3,7,5,10,4,6,2,9,0] 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6e68135b60 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 0x7f6e6813a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): picked mon.noname-e con 0x7f6e6809b6e0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e6813d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): picked mon.noname-g con 0x7f6e6813ae90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e680751e0 con 0x7f6e68135b60 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6809a9d0 con 0x7f6e6813ae90 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6809bb40 con 0x7f6e6809b6e0 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6ec24700 10 monclient(hunting): authenticate will time out at 2684.728291s 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 0x7f6e6813a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 0x7f6e6813a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42842/0 (socket says 172.21.15.47:42842) 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 -- 172.21.15.47:0/1346202118 learned_addr learned my addr 172.21.15.47:0/1346202118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): get_auth_request con 0x7f6e6809b6e0 auth_method 0 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6d1c1700 1 --2- 172.21.15.47:0/1346202118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e6813d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6d1c1700 10 monclient(hunting): get_auth_request con 0x7f6e6813ae90 auth_method 0 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6d1c1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6d1c1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e6d1c1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient(hunting): handle_auth_done global_id 36206 payload 274 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 -- 172.21.15.47:0/1346202118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 msgr2=0x7f6e6813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 --2- 172.21.15.47:0/1346202118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e6813d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 -- 172.21.15.47:0/1346202118 >> v1:172.21.15.179:6789/0 conn(0x7f6e68135b60 legacy=0x7f6e68135f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 monclient: found mon.noname-e 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 -- 172.21.15.47:0/1346202118 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680a13e0 con 0x7f6e6809b6e0 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.327+0000 7f6e67fff700 1 --2- 172.21.15.47:0/1346202118 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 0x7f6e6813a950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009920 tx=0x7f6e5c009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=544d8d400db3f192 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 1 -- 172.21.15.47:0/1346202118 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e5c009e70 con 0x7f6e6809b6e0 2024-04-20T02:33:25.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: dump: 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.343 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.344 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:25.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 1 -- 172.21.15.47:0/1346202118 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00bcd0 con 0x7f6e6809b6e0 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 5 monclient: authenticate success, global_id 36206 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 1 -- 172.21.15.47:0/1346202118 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e5c00be40 con 0x7f6e6809b6e0 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 4 set_mon_vals no callback set 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e677fe700 10 monclient: dump: 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.346 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 10 monclient: shutdown 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 -- 172.21.15.47:0/1346202118 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 msgr2=0x7f6e6813a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/1346202118 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 0x7f6e6813a950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009920 tx=0x7f6e5c009c30 comp rx=0 tx=0).stop 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 -- 172.21.15.47:0/1346202118 shutdown_connections 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/1346202118 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e6813d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/1346202118 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f6e6809b6e0 0x7f6e6813a950 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 -- 172.21.15.47:0/1346202118 >> 172.21.15.47:0/1346202118 conn(0x7f6e680a7570 msgr2=0x7f6e680a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 -- 172.21.15.47:0/1346202118 shutdown_connections 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.328+0000 7f6e6ec24700 1 -- 172.21.15.47:0/1346202118 wait complete. 2024-04-20T02:33:25.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.329+0000 7f6e6ec24700 10 monclient: build_initial_monmap 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.329+0000 7f6e6ec24700 10 monclient: monmap: 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:25.348 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.329+0000 7f6e6ec24700 1 librados: starting msgr at 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.329+0000 7f6e6ec24700 1 librados: starting objecter 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.329+0000 7f6e6ec24700 1 Processor -- start 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 -- start start 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 librados: setting wanted keys 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 librados: calling monclient init 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient: init 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient: _add_conns ranks=[0,7,3,1,2,4,8,5,6] 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 0x7f6e680a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): picked mon.noname-a con 0x7f6e6809b6e0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6e68135b60 0x7f6e681ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): picked mon.noname-i con 0x7f6e68135b60 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e681cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): picked mon.noname-g con 0x7f6e6813ae90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e681cb190 con 0x7f6e6809b6e0 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6812a870 con 0x7f6e6813ae90 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68071eb0 con 0x7f6e68135b60 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6ec24700 10 monclient(hunting): authenticate will time out at 2684.731697s 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 0x7f6e680a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 0x7f6e680a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45106/0 (socket says 172.21.15.47:45106) 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 1 -- 172.21.15.47:0/2877062388 learned_addr learned my addr 172.21.15.47:0/2877062388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): get_auth_request con 0x7f6e6809b6e0 auth_method 0 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6d1c1700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e681cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e67fff700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6e68135b60 0x7f6e681ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6d1c1700 10 monclient(hunting): get_auth_request con 0x7f6e6813ae90 auth_method 0 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6d1c1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6d1c1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6d1c1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e67fff700 10 monclient(hunting): get_auth_request con 0x7f6e68135b60 auth_method 0 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e67fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e67fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e67fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.330+0000 7f6e6c9c0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6d1c1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6d1c1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6d1c1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e67fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e67fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e67fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 10 monclient(hunting): handle_auth_done global_id 104157 payload 824 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 -- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6e68135b60 msgr2=0x7f6e681ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6e68135b60 0x7f6e681ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 -- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 msgr2=0x7f6e681cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e681cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 10 monclient: _finish_hunting 0 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 monclient: found mon.noname-a 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 -- 172.21.15.47:0/2877062388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c009550 con 0x7f6e6809b6e0 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6c9c0700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 0x7f6e680a5900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c4e0 tx=0x7f6e5800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 client.?.objecter ms_handle_connect 0x7f6e6809b6e0 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e58005540 con 0x7f6e6809b6e0 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: dump: 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.353 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: _finish_auth 0 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: _check_auth_tickets 2024-04-20T02:33:25.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e58003680 con 0x7f6e6809b6e0 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 5 monclient: authenticate success, global_id 104157 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 10 monclient: _renew_subs 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e680706b0 con 0x7f6e6809b6e0 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6e58003c60 con 0x7f6e6809b6e0 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:25.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e657fa700 10 monclient: dump: 2024-04-20T02:33:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.356 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:25.357 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e66ffd700 4 set_mon_vals no callback set 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 10 client.104157.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 10 monclient: _renew_subs 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e680718f0 con 0x7f6e6809b6e0 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.331+0000 7f6e6ec24700 1 librados: init done 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f6e580037f0 con 0x7f6e6809b6e0 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e6ec24700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:25.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e6ec24700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e6ec24700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e5403a2f0 0x7f6e5403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6809df10 con 0x7f6e6809b6e0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(683..683 src has 266..683) v4 ==== 6669+0+0 (secure 0 0 0) 0x7f6e58049800 con 0x7f6e6809b6e0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 10 client.104157.objecter ms_dispatch 0x7f6e68001550 osd_map(683..683 src has 266..683) v4 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 3 client.104157.objecter handle_osd_map got epochs [683,683] > 0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 3 client.104157.objecter handle_osd_map decoding full epoch 683 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e657fa700 20 client.104157.objecter dump_active .. 0 homeless 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e67fff700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e5403a2f0 0x7f6e5403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.332+0000 7f6e67fff700 10 monclient: get_auth_request con 0x7f6e5403a2f0 auth_method 0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.333+0000 7f6e67fff700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e5403a2f0 0x7f6e5403c7b0 secure :-1 s=READY pgs=3374 cs=0 l=1 rev1=1 crypto rx=0x7f6e680a7e20 tx=0x7f6e5c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.333+0000 7f6e657fa700 10 client.104157.objecter ms_handle_connect 0x7f6e5403a2f0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.337+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f6e58017020 con 0x7f6e6809b6e0 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.337+0000 7f6e657fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:25.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.337+0000 7f6e657fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.644+0000 7f6e6ec24700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:33:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.644+0000 7f6e6ec24700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:33:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.644+0000 7f6e6ec24700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.644+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6e6812d390 con 0x7f6e6809b6e0 2024-04-20T02:33:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.646+0000 7f6e657fa700 1 -- 172.21.15.47:0/2877062388 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f6e580050d0 con 0x7f6e6809b6e0 2024-04-20T02:33:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.646+0000 7f6e657fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:33:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.646+0000 7f6e657fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:25.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=19,config=2,health=12,kv=11,logm=56} crc {auth=4290480187,config=692619927,health=1551972114,kv=1530195301,logm=1924824470}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=388966291,config=692619927,health=1551972114,kv=1530195301,logm=1835160034}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=20,config=2,health=12,kv=9,logm=57} crc {auth=3188339172,config=692619927,health=3063334479,kv=2389660009,logm=1908935022}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=3188339172,config=3678701417,health=1551972114,kv=1530195301,logm=1232050550}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=19,config=2,health=12,kv=10,logm=57} crc {auth=2788211505,config=692619927,health=1551972114,kv=105211281,logm=2017782665}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=20,config=2,health=11,kv=10,logm=57} crc {auth=3188339172,config=692619927,health=2679528959,kv=1530195301,logm=122603020}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=3188339172,config=692619927,health=4118407985,kv=2226345239,logm=1622676550}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=10,kv=11,logm=57} crc {auth=3188339172,config=692619927,health=2731986593,kv=1530195301,logm=706548143}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.646+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=20,config=2,health=11,kv=10,logm=57} crc {auth=3188339172,config=692619927,health=2257693759,kv=3003869462,logm=784166779}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3894629518}) 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2290879594}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1343807243}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.648+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1151585576}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=365752077}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=973968913}) 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1778592273}) 2024-04-20T02:33:25.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:25.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2273339305}) 2024-04-20T02:33:25.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.649+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2586521862}) 2024-04-20T02:33:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.649+0000 7f6e6ec24700 10 librados: watch_flush enter 2024-04-20T02:33:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 10 librados: watch_flush exit 2024-04-20T02:33:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 20 client.104157.objecter shutdown clearing up homeless session... 2024-04-20T02:33:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 10 client.104157.objecter successfully canceled tick 2024-04-20T02:33:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e5403a2f0 msgr2=0x7f6e5403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e5403a2f0 0x7f6e5403c7b0 secure :-1 s=READY pgs=3374 cs=0 l=1 rev1=1 crypto rx=0x7f6e680a7e20 tx=0x7f6e5c008030 comp rx=0 tx=0).stop 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 10 monclient: shutdown 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 msgr2=0x7f6e680a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 0x7f6e680a5900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c4e0 tx=0x7f6e5800c7f0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 shutdown_connections 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f6e5403a2f0 0x7f6e5403c7b0 unknown :-1 s=CLOSED pgs=3374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6e6813ae90 0x7f6e681cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e6809b6e0 0x7f6e680a5900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 --2- 172.21.15.47:0/2877062388 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f6e68135b60 0x7f6e681ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 >> 172.21.15.47:0/2877062388 conn(0x7f6e680a7570 msgr2=0x7f6e680a2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 shutdown_connections 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 -- 172.21.15.47:0/2877062388 wait complete. 2024-04-20T02:33:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:25.651+0000 7f6e6ec24700 1 librados: shutdown 2024-04-20T02:33:25.676 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989809219}) 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4237022317}) 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.003 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4080420352}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3882473721}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=896721933}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2486662086}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.004 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3975862613}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1335574213}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.651+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=431179124}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3049850467}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3372576480}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2278880681}) 2024-04-20T02:33:26.005 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3742521349}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2190305111}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2590938439}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2142792318}) 2024-04-20T02:33:26.006 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2211791344}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.653+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=686993400}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1097895758}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2494564309}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4289502042}) 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.007 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3694448848}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=939014125}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3577055821}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3262042509}) 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.008 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1809879860}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.657+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1755324581}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3381524750}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1557553211}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=331448364}) 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.009 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1226221780}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=535947999}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2823800850}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=55809579}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2090799394}) 2024-04-20T02:33:26.010 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.659+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1141604943}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=36098394}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2831821788}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1040231367}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.011 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=297892521}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=533740213}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2792768240}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=272550105}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465934736}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.661+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1158895216}) 2024-04-20T02:33:26.012 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4162601132}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3473778638}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3045627643}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=364275671}) 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.013 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3550997690}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3696067388}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1310172379}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1375871828}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.663+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=220789115}) 2024-04-20T02:33:26.014 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4203778040}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1011118006}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1641604537}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3847636618}) 2024-04-20T02:33:26.015 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3252590159}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1546347592}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.664+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3569140242}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.665+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.665+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=465078730}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.665+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3918809764}) 2024-04-20T02:33:26.016 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4058594207}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=852207021}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=568888129}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3920145628}) 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.017 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1852454676}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796851390}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2776796802}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=368413144}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.666+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3451209997}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.018 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1915173259}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=787279476}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.667+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1027319441}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3210582123}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.019 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=23528109}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4188847695}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1811743869}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=737561718}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.668+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1771898317}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3262410733}) 2024-04-20T02:33:26.020 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=454483853}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=760207090}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3425858335}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760129928}) 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.021 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=936351892}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.669+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.670+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3095019087}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.670+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.670+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2863475742}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.670+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1130038902}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3964169701}) 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.022 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=309011806}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=615493700}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=845426694}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=962732979}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.023 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1195462015}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2539096135}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3883597345}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.671+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2928289576}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=277336643}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4248059661}) 2024-04-20T02:33:26.024 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=14026778}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3944866598}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1379421338}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1610434132}) 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.025 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3079149927}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2082701656}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.672+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1610355358}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3738705219}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4286717573}) 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.026 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1038782315}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3843963863}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1585968983}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2785661929}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.027 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1470948219}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3129149186}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.674+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2798638727}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488073256}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410459550}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3558095553}) 2024-04-20T02:33:26.028 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1497781938}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.675+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1852324907}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4246664688}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2837000944}) 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.029 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3420831122}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.676+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=896862274}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3819896375}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3113410622}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2929901758}) 2024-04-20T02:33:26.030 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1919724928}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1068129831}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4293897090}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3080898129}) 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.031 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38265263}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.677+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443280947}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.678+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4150216746}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1623374282}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488846695}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.032 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=399460560}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1652323957}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3202802176}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3387365107}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4174537215}) 2024-04-20T02:33:26.033 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.679+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3339765509}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=471192414}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4213596781}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1970855565}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.034 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2379469006}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2286541018}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1697252722}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1398670700}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2848249568}) 2024-04-20T02:33:26.035 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.680+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2326511631}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1298777560}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2697451481}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2196762466}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3747070766}) 2024-04-20T02:33:26.036 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=83079080}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2423357367}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1921444102}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100936153}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.682+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=39745955}) 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.037 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2489060413}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2473791007}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3776298485}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2077907874}) 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.683+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.038 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1389386028}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1996310220}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=877509860}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=481430854}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.684+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1836267317}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.039 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1217524312}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2018908991}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1650529453}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2906884833}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=447656844}) 2024-04-20T02:33:26.040 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1928544226}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2344320774}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=571498384}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.685+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2164171961}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159770970}) 2024-04-20T02:33:26.041 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2726349595}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=268335350}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=116143518}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1388696744}) 2024-04-20T02:33:26.042 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1698351283}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=979568775}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2956004222}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.687+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3037523580}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083579429}) 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.043 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1196837594}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2880219396}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=968765211}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3071861538}) 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.044 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1168415842}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.688+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4140163211}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.689+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.689+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3977963051}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.689+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2730946976}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2856876418}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.045 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3060225694}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4062062482}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=687096374}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2336017494}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.046 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=377796206}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4034913808}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=80086470}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.690+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751898215}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.691+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.691+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.691+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1163301442}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.047 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3981305503}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3970135003}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2350465355}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2621939574}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1826338315}) 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.048 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3982717902}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158732971}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.692+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557955347}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.693+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.693+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.693+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=507284753}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.693+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2971920189}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.049 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3363132110}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=505243020}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4062957232}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1310365630}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2730324757}) 2024-04-20T02:33:26.050 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3245351694}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.694+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=150301085}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2883804016}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3119415437}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3244203928}) 2024-04-20T02:33:26.051 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2770426918}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095940641}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3473686383}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=897073878}) 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.052 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=355851265}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.696+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4147022730}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.697+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.697+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.697+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=784529006}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.697+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.697+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2915885669}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1247971231}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1664948978}) 2024-04-20T02:33:26.053 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=783298620}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2718212549}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3326687702}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2676950771}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.698+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1240493008}) 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.054 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=204992218}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4102544950}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=438388883}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1300390325}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.055 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2800137019}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4214989867}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=223842747}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1054520582}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.699+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3934218737}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.056 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939384167}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3553930959}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=759472554}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.700+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1174214910}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=826495405}) 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.057 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3416189867}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2550454058}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196151625}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.701+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=981849803}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4135124659}) 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.058 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3876922124}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3009305351}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3130704461}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1846550446}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2404050631}) 2024-04-20T02:33:26.059 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1219566508}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2965304108}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.702+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1323417592}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3729613004}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=128176059}) 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.060 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2034444948}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3139859071}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3691028251}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.704+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1972072000}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.705+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.705+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.061 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.705+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2496084751}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.705+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.705+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538064961}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.705+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=14363049}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3346696073}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3830286685}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.062 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1372029316}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1749685406}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3965787627}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2598758859}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3462297189}) 2024-04-20T02:33:26.063 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1239825933}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.706+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1773141267}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598499233}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2044316537}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4125971759}) 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.064 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=788675750}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3298391822}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=913740290}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3431673244}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3154595005}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.708+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1612256825}) 2024-04-20T02:33:26.065 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3919922991}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=621713264}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2150315849}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.709+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1592712951}) 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.066 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3501871132}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3509019720}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1104715626}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=977256259}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.710+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2205393767}) 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.067 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2074034079}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3199623683}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3378670757}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=108886241}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=883765476}) 2024-04-20T02:33:26.068 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1709433075}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2394873879}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=140492947}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.711+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1243077584}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1709334382}) 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.069 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=887558263}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=823902683}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3128547323}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1328196891}) 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.070 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2227395717}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1346521173}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1002570857}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.713+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=697156249}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=278522899}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.071 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1243271622}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3799498308}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2677041707}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1203190616}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=417695919}) 2024-04-20T02:33:26.072 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3452001051}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3757743870}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.714+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963114232}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2411786805}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3518085636}) 2024-04-20T02:33:26.073 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=339054526}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3931166636}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3228379427}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1330030958}) 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.074 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2749305252}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869566705}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.716+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=289015031}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3704400564}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2055367412}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.075 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1306335010}) 2024-04-20T02:33:26.076 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.717+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.076 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.076 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1406716689}) 2024-04-20T02:33:26.076 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.076 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=179968579}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3517993174}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3924964937}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4103314250}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.718+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088736901}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.077 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=257238494}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1714002348}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=784909629}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2118360932}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.078 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3106163932}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=991404637}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4217730274}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102065948}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.719+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3445772316}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.720+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.720+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.720+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196630498}) 2024-04-20T02:33:26.079 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.720+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.720+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3183692391}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2971015260}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1266189405}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=351656029}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.080 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3076328248}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=986129850}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837890800}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.721+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290580082}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=16319119}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4164651562}) 2024-04-20T02:33:26.081 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2888842790}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1668473892}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1071314406}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=13524984}) 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.082 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3461456107}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693762121}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.722+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2750937253}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1860713279}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2520739745}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.083 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1237572698}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=532482838}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=528714292}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=96289828}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=396799939}) 2024-04-20T02:33:26.084 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1342768567}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.724+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1093250262}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=874669538}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1535618282}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3540411497}) 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.085 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4006249883}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532402085}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1953543205}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1952619634}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3564273599}) 2024-04-20T02:33:26.086 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.725+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=549343688}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4040096875}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=147928324}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1907094373}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=600581065}) 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.087 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=150825682}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=346435697}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2159311601}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=203144993}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.727+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4001163216}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.088 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3300785100}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2941067333}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=214705078}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2829667648}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2005379510}) 2024-04-20T02:33:26.089 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3721635370}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1522341669}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2028522779}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.729+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4051504448}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=543617432}) 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.090 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1864651954}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1437968751}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3601740135}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3711528670}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.091 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3490227508}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2532535493}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2540931770}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.731+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=48126908}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2350267124}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3432548610}) 2024-04-20T02:33:26.092 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3102001429}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.734+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1338323230}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3302002725}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1290308548}) 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.093 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2993766699}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3042856706}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.735+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=542121345}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2370560679}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=735045554}) 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.742+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.094 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2777104022}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1462707506}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=882204209}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=613727005}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.095 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=579952688}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=358500313}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.743+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3198870844}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.746+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3270721108}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3873376757}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.096 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=745412940}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2094070516}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519010309}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1112807994}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3845973001}) 2024-04-20T02:33:26.097 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=260705446}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.747+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2955989812}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253621797}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2906489765}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2838606845}) 2024-04-20T02:33:26.098 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1025137703}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855660105}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2709346669}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.771+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4259115372}) 2024-04-20T02:33:26.099 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.772+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.772+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=762525508}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.772+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2856308787}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3878905591}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=988051000}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1165760453}) 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.100 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1269973598}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2888310671}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=500271450}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1989619637}) 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.101 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629347121}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.773+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1990210999}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3734917546}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3957040420}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2685132448}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.102 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1298779325}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2500322558}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3388590984}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3128782473}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=246342082}) 2024-04-20T02:33:26.103 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.775+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3475344484}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=870076022}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1339255610}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3413271203}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1471067374}) 2024-04-20T02:33:26.104 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=754628777}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1776727274}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.777+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.778+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=138042413}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.778+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.778+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1176774617}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.778+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2901541944}) 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.105 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3438618906}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1963094250}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1005643604}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1970104030}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.106 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3093962169}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4285995053}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=836970378}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038658898}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.779+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1612451004}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1254164320}) 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.107 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1615905375}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1258433383}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=983614987}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261919508}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.108 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3382538050}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1010319222}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=371140998}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.781+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3596594137}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.782+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.782+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.782+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1358345731}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.782+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.782+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1644051371}) 2024-04-20T02:33:26.109 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2712049489}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=524007545}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689549630}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2041061869}) 2024-04-20T02:33:26.110 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=524269467}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3163541957}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.783+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683466697}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.784+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.784+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2706955574}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1884824990}) 2024-04-20T02:33:26.111 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1994686713}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3835468595}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=334994398}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2569068138}) 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.112 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=990587614}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1398662930}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.785+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1833201942}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=745060792}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=134339906}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.113 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3221983910}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4052696808}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1790265136}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1821222050}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.114 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2146831047}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2173463316}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.786+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=880473087}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2303919616}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3292086004}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.115 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1625723880}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=802736500}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3901642004}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1389320602}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.116 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1088841059}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=64572464}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.788+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1530399064}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=500832093}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2121224899}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3765236042}) 2024-04-20T02:33:26.117 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2979485823}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=7136295}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=974896705}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=852372172}) 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.118 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=351084697}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.790+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2753488181}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=155039062}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2008269464}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=752118851}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.791+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.119 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3643412387}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261450054}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2904638524}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3647589611}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3817450458}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.792+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3535975896}) 2024-04-20T02:33:26.120 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3509306729}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1845620524}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1081210716}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3158523622}) 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.121 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1492701622}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4051100229}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=788595763}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1050318174}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.794+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=101862563}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.122 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2386046381}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4290173262}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.795+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2549664311}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3729105296}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3392539093}) 2024-04-20T02:33:26.123 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=460669484}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1316548903}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2053598762}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.796+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3104590132}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347670006}) 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.124 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2764403175}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=627837372}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4081232988}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3341958435}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.125 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3294196590}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1992124323}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3065044821}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.797+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1302309364}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=424374615}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3093887958}) 2024-04-20T02:33:26.126 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2967944612}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2823398175}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2165698290}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4179987750}) 2024-04-20T02:33:26.127 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1484769424}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1222386020}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.799+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=310112910}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.800+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.800+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.800+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3180497007}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.800+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.800+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2462463746}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.128 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3644321607}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=261636674}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2627961625}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3318396906}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3280187304}) 2024-04-20T02:33:26.129 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2237669553}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.801+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2626404455}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196602203}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2894188857}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=773803512}) 2024-04-20T02:33:26.130 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1298784623}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=672131204}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2161594079}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.131 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3503822606}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640298132}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.802+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3845995068}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1885110504}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522335358}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3292760628}) 2024-04-20T02:33:26.132 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3712242669}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3526640351}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1768465098}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2322161474}) 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.133 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3788630189}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.804+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3713458386}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1944191631}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1591873602}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.805+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1485581312}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.134 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2324302383}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1761658818}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=395395564}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1665412123}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1294577795}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.806+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2824073864}) 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.135 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207016187}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=455273455}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1429393124}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2690382946}) 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4074273331}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2517148283}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3906072911}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3443829354}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.807+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1874709029}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=522115496}) 2024-04-20T02:33:26.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2386054061}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=781298028}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1672975619}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=792215307}) 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=574580593}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2804915989}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=136759601}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.809+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2918287076}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3808480094}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2555446926}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=485113553}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2162199694}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=875869797}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2822460121}) 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.810+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.811+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2130236352}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.811+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.811+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2734260483}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.811+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=299567775}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=512971423}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1344018408}) 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.141 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2031660246}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1474740195}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=33948746}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2833739644}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.142 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2140746517}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612133194}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.812+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1565553591}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1392554227}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=351552990}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=237236119}) 2024-04-20T02:33:26.143 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1550063075}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1418017347}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2250710732}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.813+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.814+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.814+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1871817208}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.814+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.814+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=619773779}) 2024-04-20T02:33:26.144 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.814+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2624673802}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302463986}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2730167127}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1751489374}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4259805402}) 2024-04-20T02:33:26.145 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1566843394}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2923892282}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=78758468}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1082008828}) 2024-04-20T02:33:26.146 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.815+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2787577351}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=442259469}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1501796589}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1583626921}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3512908968}) 2024-04-20T02:33:26.147 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=320841999}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.816+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909285584}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4237909016}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3030992511}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.817+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2885209289}) 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.148 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011031512}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1486399845}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=348649954}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3122731116}) 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.149 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3731076617}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1436717382}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2212323551}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2374365164}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.818+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1653093937}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.819+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.819+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.150 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.819+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906113611}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.819+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.819+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.819+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=25945219}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3635087500}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1439654997}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=479498728}) 2024-04-20T02:33:26.151 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1646520814}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2684148509}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2268020282}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.820+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4268026360}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146409544}) 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.152 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4169236887}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=955742032}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3292397119}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2916229265}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1423315816}) 2024-04-20T02:33:26.153 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1623696233}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4227205510}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.821+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3453472671}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1179811885}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1576359884}) 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.154 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1511921398}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2229091939}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123413012}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4267055186}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.155 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2467580259}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1700560817}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.823+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2735900497}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=265139678}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3890322840}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=137709269}) 2024-04-20T02:33:26.156 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3448063673}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2064409343}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=643251777}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=124947344}) 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.157 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3220709007}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.824+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=860338145}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1376328579}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=920693265}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2178900289}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.158 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2284405644}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1287771833}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1203736542}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3915914819}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=328770147}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.826+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=756914720}) 2024-04-20T02:33:26.159 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.827+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2841688188}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2953790017}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=486844617}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1637627090}) 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.160 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=106940082}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3199191464}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4178254085}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622697551}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.828+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3028755161}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.161 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2338858911}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=270147844}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2928424402}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2729820236}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.162 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=225580594}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3054601260}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2994908857}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398836747}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.829+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=379975746}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.163 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1003059729}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4059428797}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4274372233}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3772189794}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4172527137}) 2024-04-20T02:33:26.164 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2090799615}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=548636156}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3255374712}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.831+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4099778784}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674738982}) 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.165 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1099094478}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1407988974}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1977119399}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2531072530}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.832+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.166 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1271418660}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3988063627}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2409245735}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.833+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3857258574}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2579613178}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.167 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2622428100}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2654232222}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2574600365}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1215824038}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.168 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2455422142}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2891355268}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3176871337}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.834+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2869858086}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3964485879}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.169 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=702995393}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1178809301}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2990284458}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3011347510}) 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.170 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2188941996}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2457857232}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=601782201}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.836+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043405118}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1746918792}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.171 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3484127156}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.837+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=565563121}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1720415033}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=546171289}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4072666025}) 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.172 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2767179034}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466886820}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.838+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3000606581}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3932710245}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1927899348}) 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.173 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3640935757}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=134868645}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2455754077}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3169694248}) 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.174 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=331086267}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=284048656}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.839+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1158935639}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.840+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.840+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3340461223}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4263505076}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3658783757}) 2024-04-20T02:33:26.175 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=352661079}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1360027959}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1014616294}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1271882478}) 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.176 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1580380876}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.841+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1194424274}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1242564557}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3706300462}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1274870477}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3211028433}) 2024-04-20T02:33:26.177 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3868216157}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3466926182}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=923407320}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3624858853}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.842+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2558529478}) 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.178 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1483208849}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1672211185}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3145906071}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3766433856}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.179 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2719263224}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2098266294}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=935008045}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2552607295}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.844+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3479318654}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2877539778}) 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.180 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3525568897}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1733016143}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3427214744}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=459118742}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.181 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3382532433}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2036008968}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.845+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937648335}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.846+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3468283006}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4098434752}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.182 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=961593423}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1724280728}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2522188818}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4272485377}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2939688777}) 2024-04-20T02:33:26.183 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.847+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=937553103}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.848+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.848+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2761879904}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.848+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=321215463}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2308675206}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2665155731}) 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.184 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4191623885}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3768956875}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2605080176}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1422093415}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.185 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2718135645}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878151336}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.849+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=142076759}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701761923}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1515409606}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.186 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1114756086}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4146966866}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1627646383}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.850+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1166376154}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3339753796}) 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.187 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605612078}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.851+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2599731884}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=255018699}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=114543800}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3793426585}) 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.188 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=552117834}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1638426687}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123736470}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2783950252}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319389145}) 2024-04-20T02:33:26.189 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.852+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3516201029}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2984851837}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3956303747}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3865145772}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2112040093}) 2024-04-20T02:33:26.190 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2103244586}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=566504053}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=37317803}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1956831276}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.854+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1108905900}) 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.191 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3651763227}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1010755957}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2185496790}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=999076751}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4109162965}) 2024-04-20T02:33:26.192 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1482047529}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1496332308}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3921597288}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.856+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4029605912}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2167361473}) 2024-04-20T02:33:26.193 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1742681443}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2841625956}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3296701802}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2028047545}) 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.194 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.857+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=15216145}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.858+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.858+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.858+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=587091147}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.858+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.858+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90219271}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.858+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1535496570}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3145159886}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.195 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617352676}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2503632760}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=756810654}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2617714013}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2175578336}) 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.196 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2215240297}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1339984522}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.859+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1551910709}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=648832742}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=183443726}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.197 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=852829134}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3352845124}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=445568384}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.860+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2514494330}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2114032337}) 2024-04-20T02:33:26.198 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932617236}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.861+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1208918090}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=745111729}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3010870616}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4223258793}) 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.199 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2242551252}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=98575031}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2960527846}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=788150718}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3099117626}) 2024-04-20T02:33:26.200 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.862+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4091198679}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1475114416}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2339598367}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3436668435}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.863+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2775659598}) 2024-04-20T02:33:26.201 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2567875710}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3283953353}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=940879459}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432372984}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.864+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=517344028}) 2024-04-20T02:33:26.202 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2924927515}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3114487790}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=541634912}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=593645968}) 2024-04-20T02:33:26.203 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1613801249}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=659477391}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=857235592}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=109072574}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.865+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3668467281}) 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.866+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.204 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4036722015}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2555524442}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2254166112}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=520665515}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.205 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3436329893}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1651741610}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1824771447}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1709976285}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.867+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3019453878}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307710137}) 2024-04-20T02:33:26.206 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2701826403}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2687335524}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1620681368}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1140568349}) 2024-04-20T02:33:26.207 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2720033465}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.868+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3933564326}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.869+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.869+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649547380}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.869+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1110173642}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2512582269}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.208 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3886465946}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1707701729}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3455706752}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=791527572}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2705604714}) 2024-04-20T02:33:26.209 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1156206053}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=961578590}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.870+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2320927081}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2853861706}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2282092178}) 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.210 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=825972661}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3215363402}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3709133020}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3130537420}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.211 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3095950994}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3082554962}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.872+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=421016255}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3692726626}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1895945889}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2009822073}) 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.212 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2709620583}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2871800898}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3045798801}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2840174409}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2718934908}) 2024-04-20T02:33:26.213 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.874+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3342585062}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1887695882}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2263091328}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=611082629}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2701382088}) 2024-04-20T02:33:26.214 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3863298518}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1610822468}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=604813762}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=415086101}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.876+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1548399210}) 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.215 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2059611281}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4171027332}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1612235908}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3714524584}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.216 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=285191908}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=572165647}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1276962762}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=245748695}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.878+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2870350955}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.879+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.879+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.217 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1908819859}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4171806374}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1910776335}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1441641373}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4294703436}) 2024-04-20T02:33:26.218 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2438421204}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2300040364}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752287257}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.880+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2012010158}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3998497569}) 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.219 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2922580657}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4115950477}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.881+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=912181560}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=245135175}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.220 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1849516666}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3466339936}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2066255838}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.882+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=147698012}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4042992968}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1780742239}) 2024-04-20T02:33:26.221 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=478840330}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3412671855}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.883+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2739963940}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1609117781}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.222 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3912762209}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=882884388}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.884+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=678064841}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3375616661}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1472323311}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1365301391}) 2024-04-20T02:33:26.223 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1315372056}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1424647518}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1235860492}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4012403434}) 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.224 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523339138}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.885+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3236841972}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2313731927}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=193196336}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1471188312}) 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.225 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4011901018}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2647841413}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2018892760}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2347179970}) 2024-04-20T02:33:26.226 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482129264}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.887+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2270804840}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3234340246}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2573718213}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.888+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=30702381}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.227 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1322944635}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=670020912}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2629808674}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3673252822}) 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.228 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647585378}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.889+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=192345836}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2501145554}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1314882523}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2197117243}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3903587163}) 2024-04-20T02:33:26.229 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3346648025}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.890+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2848513951}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.891+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.891+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.891+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=637808792}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.891+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.891+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705239989}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.891+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=887629741}) 2024-04-20T02:33:26.230 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=229281064}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3504924096}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3091294954}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2189227082}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.231 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739227038}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1474005946}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2546572324}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=853948420}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.892+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3775676178}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2501829290}) 2024-04-20T02:33:26.232 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3218267982}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2392566427}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=111546776}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4014585656}) 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.233 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1139198769}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2472855658}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382160208}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.894+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2037803024}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.895+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=743271009}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.234 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1352403816}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3025442700}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3389682434}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486459715}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=350191135}) 2024-04-20T02:33:26.235 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1072235615}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2768142259}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.896+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3952271575}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863842035}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2565011100}) 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.236 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.897+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1585532651}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4258090619}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1924910940}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2252664933}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.237 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2046565217}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2387781991}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.898+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1878560091}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1188747161}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3502444824}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2832658730}) 2024-04-20T02:33:26.238 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.899+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2469055329}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2866267828}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2048210288}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4033873398}) 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.239 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=537970381}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.900+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3948720443}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1287087167}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3894975502}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2339991814}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.240 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2632138586}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3809155129}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1815072457}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3626306249}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2226706084}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.901+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=526379479}) 2024-04-20T02:33:26.241 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1595787313}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3336695234}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=620047216}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=436926719}) 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.242 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=799245636}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2971209501}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1434483185}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1395414894}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.903+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2194197228}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.243 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1385047946}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=958564274}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4218488652}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3633892250}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1262745996}) 2024-04-20T02:33:26.244 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3882769007}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=983483214}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2067794399}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.905+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=841913478}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3397746176}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.245 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1551679065}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=724747692}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=100689682}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2634774769}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.246 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1802973482}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3277892017}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3544525242}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.907+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3038807985}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3772929689}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2593388237}) 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.247 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3410191753}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1739129753}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929785314}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2407820070}) 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.248 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1755857981}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.908+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.909+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3134401585}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.909+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=798264826}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.909+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.909+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.909+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4286307069}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.909+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2371294422}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2538249215}) 2024-04-20T02:33:26.249 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2029576195}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1240953291}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2907606031}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.250 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3659027761}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3878216224}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.910+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=399989970}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202136789}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2532596543}) 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.251 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=651798093}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4181268912}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1581727013}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2804076760}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3675008885}) 2024-04-20T02:33:26.252 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2360423486}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.911+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2356939164}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752623684}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1615681916}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4255596492}) 2024-04-20T02:33:26.253 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=286122676}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2266186977}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2065780080}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3214031405}) 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.254 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=970662138}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.912+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=251268902}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758127761}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2457191571}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3463713772}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.255 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1172850942}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1549341528}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1371070785}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2216037666}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2158955444}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.913+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3636743471}) 2024-04-20T02:33:26.256 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2291891480}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3883107603}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.914+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=212644395}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2482635222}) 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.257 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=105523065}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2245621976}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=846525765}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1823896621}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.915+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3048860375}) 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.258 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=316116572,mdsmap=3585761405}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379659011}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=4144051037,mdsmap=2826658659}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=3666890245,mdsmap=1531151339}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.259 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1298049223}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1918212242}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.916+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.917+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.917+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=93,mdsmap=3,mgr=4} crc {logm=4124674572,mdsmap=1531151339,mgr=3036405304}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.917+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.917+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=2543113190,mdsmap=2826658659}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.917+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=2010194326,mdsmap=3585761405}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.260 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=41,osdmap=17} crc {mdsmap=3140514009,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=396910797,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=1272777559,osdmap=41398207}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=16} crc {mdsmap=3140514009,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1024276302,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=249402865,osdmap=2250212415}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=41,osdmap=19} crc {mdsmap=3140514009,mgr=2883445498,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3317215038,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=2839639639,osdmap=2861408419}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=16} crc {mdsmap=3140514009,mgr=2933050016,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=396910797,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=2682459432,osdmap=4288932316}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.261 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=2,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=22} crc {mdsmap=3140514009,mgr=2422868175,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1793095640,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=113746222,osdmap=4268097101}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=10,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=40,osdmap=21} crc {mdsmap=3140514009,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=718502087,mon_config_key=3905112211,monmap=2370797809,osd_metadata=1133748595,osd_pg_creating=617498565,osd_snap=3060606496,osdmap=4122227524}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mdsmap=2,mgr=8,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=17} crc {mdsmap=3140514009,mgr=1787245756,mgr_metadata=2530826944,mgrstat=528087614,mon_config_key=2968193071,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=2848252589,osdmap=45687313}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=1,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=19} crc {mdsmap=1228700967,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1748166207,mon_config_key=2868358642,monmap=2370797809,osd_metadata=3566517106,osd_pg_creating=617498565,osd_snap=1715349490,osdmap=1199004743}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.919+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=20} crc {mdsmap=3140514009,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1765326958,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3566517105,osd_pg_creating=617498565,osd_snap=3553842260,osdmap=3835868093}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.262 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3747884711}) 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=926735881}) 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1230795597}) 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.263 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=916502595}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.921+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2982994029}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1423358299}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1499700329}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1375378080}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.922+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=731060057}) 2024-04-20T02:33:26.264 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2963582097}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2807014909}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3751638638}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.923+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1384377912}) 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.265 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2883508822}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3685833994}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1732806844}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2560185707}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.924+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=46774044}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.266 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3899039563}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=606220048}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=530522960}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.925+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3023862027}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3252998122}) 2024-04-20T02:33:26.267 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=930448698}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2505475203}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2038609175}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.926+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3095185101}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.927+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.927+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3442041354}) 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.268 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3555737046}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1204003710}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4025993594}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=141266914}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=599089566}) 2024-04-20T02:33:26.269 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4111407071}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=798781696}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.928+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=196021942}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3613213251}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4131062189}) 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.270 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1934379946}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.931+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2197453357}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1373342009}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=545011257}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.271 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2833269284}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4267644691}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.932+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=323318455}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=96} crc {osdmap=1833520064}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=96} crc {osdmap=3933928790}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=96} crc {osdmap=1081831573}) 2024-04-20T02:33:26.272 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=95} crc {osdmap=3693470370}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=95} crc {osdmap=1293260818}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=98} crc {osdmap=3083706016}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.945+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.946+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=94} crc {osdmap=662979215}) 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.946+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:33:26.273 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.946+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=96} crc {osdmap=697460589}) 2024-04-20T02:33:26.274 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:33:25.946+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=97} crc {osdmap=2532132846}) 2024-04-20T02:33:26.677 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:33:26.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.845+0000 7fe76f272700 10 monclient: get_monmap_and_config 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.845+0000 7fe76f272700 10 monclient: build_initial_monmap 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.845+0000 7fe76f272700 10 monclient: monmap: 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:26.846378+0000 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:26.846378+0000 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:26.846 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:26.847 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:26.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.846+0000 7fe76f272700 1 Processor -- start 2024-04-20T02:33:26.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.846+0000 7fe76f272700 1 -- start start 2024-04-20T02:33:26.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.846+0000 7fe76f272700 10 monclient: init 2024-04-20T02:33:26.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient: _add_conns ranks=[9,2,10,0,3,6,5,1,8,4,7] 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): picked mon.noname-a con 0x7fe7680a1860 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76809faa0 0x7fe76809fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): picked mon.noname-b con 0x7fe76809faa0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe76809a930 0x7fe76809ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): picked mon.noname-f con 0x7fe76809a930 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680a03c0 con 0x7fe7680a1860 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76809cc90 con 0x7fe76809a930 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680a1c40 con 0x7fe76809faa0 2024-04-20T02:33:26.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.847+0000 7fe76f272700 10 monclient(hunting): authenticate will time out at 2686.248578s 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45114/0 (socket says 172.21.15.47:45114) 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 learned_addr learned my addr 172.21.15.47:0/1163127155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): get_auth_request con 0x7fe7680a1860 auth_method 0 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d80f700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.179:3300/0 conn(0x7fe76809a930 0x7fe76809ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d80f700 10 monclient(hunting): get_auth_request con 0x7fe76809a930 auth_method 0 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d80f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:26.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d80f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.849+0000 7fe76d80f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 10 monclient(hunting): handle_auth_done global_id 104166 payload 274 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76809faa0 msgr2=0x7fe76809fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe76809faa0 0x7fe76809fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 >> v2:172.21.15.179:3300/0 conn(0x7fe76809a930 msgr2=0x7fe76809ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.179:3300/0 conn(0x7fe76809a930 0x7fe76809ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 10 monclient: _finish_hunting 0 2024-04-20T02:33:26.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 monclient: found mon.noname-a 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680751e0 con 0x7fe7680a1860 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 msgr2=0x7fe768133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 msgr2=0x7fe768133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7fe76809c6b0 tx=0x7fe76400d330 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:26.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:26.850+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient: get_auth_request con 0x7fe7680a1860 auth_method 0 2024-04-20T02:33:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 0 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-20T02:33:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient: ms_handle_reset current mon v2:172.21.15.47:3300/0 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 msgr2=0x7fe768133f30 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 --2- 172.21.15.47:0/1163127155 >> v2:172.21.15.47:3300/0 conn(0x7fe7680a1860 0x7fe768133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient: _add_conns ranks=[4,5,6,3,1,7,0,8] 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe754005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): picked mon.noname-c con 0x7fe7540036c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe754006420 0x7fe7540088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): picked mon.noname-h con 0x7fe754006420 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe754009220 0x7fe75400b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): picked mon.noname-d con 0x7fe754009220 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe75400c030 con 0x7fe7540036c0 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe75400c170 con 0x7fe754006420 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe75400c330 con 0x7fe754009220 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe75ffff700 10 monclient(hunting): ms_handle_reset stray mon v2:172.21.15.47:3300/0 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe754005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): get_auth_request con 0x7fe7540036c0 auth_method 0 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe754009220 0x7fe75400b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): get_auth_request con 0x7fe754009220 auth_method 0 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76c80d700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe754006420 0x7fe7540088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76d80f700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76c80d700 10 monclient(hunting): get_auth_request con 0x7fe754006420 auth_method 0 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76c80d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76c80d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.051+0000 7fe76c80d700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 10 monclient(hunting): handle_auth_done global_id 104166 payload 293 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe754009220 msgr2=0x7fe75400b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fe754009220 0x7fe75400b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe754006420 msgr2=0x7fe7540088b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fe754006420 0x7fe7540088b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 monclient: found mon.noname-c 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 -- 172.21.15.47:0/1163127155 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75400c5b0 con 0x7fe7540036c0 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d80f700 1 -- 172.21.15.47:0/1163127155 reap_dead start 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76d00e700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe754005b10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe76400d730 tx=0x7fe7640082b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e5080616cf730311 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe764010070 con 0x7fe7540036c0 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: dump: 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76f272700 5 monclient: authenticate success, global_id 104166 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76f272700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7640178c0 con 0x7fe7540036c0 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76f272700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe76f272700 4 set_mon_vals no callback set 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 1 -- 172.21.15.47:0/1163127155 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe764017ea0 con 0x7fe7540036c0 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.052+0000 7fe75ffff700 10 monclient: dump: 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.066 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.068 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.068 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.068 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.068 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 10 monclient: shutdown 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 -- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 msgr2=0x7fe754005b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe754005b10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe76400d730 tx=0x7fe7640082b0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 -- 172.21.15.47:0/1163127155 shutdown_connections 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 --2- 172.21.15.47:0/1163127155 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe754005b10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 -- 172.21.15.47:0/1163127155 >> 172.21.15.47:0/1163127155 conn(0x7fe7680a6480 msgr2=0x7fe7680a2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 -- 172.21.15.47:0/1163127155 shutdown_connections 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.053+0000 7fe76f272700 1 -- 172.21.15.47:0/1163127155 wait complete. 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.054+0000 7fe76f272700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.054+0000 7fe76f272700 10 monclient: monmap: 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:27.069 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.054+0000 7fe76f272700 1 librados: starting msgr at 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.054+0000 7fe76f272700 1 librados: starting objecter 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.054+0000 7fe76f272700 1 Processor -- start 2024-04-20T02:33:27.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 -- start start 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 librados: setting wanted keys 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 librados: calling monclient init 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient: init 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient: _add_conns ranks=[4,2,8,5,7,6,1,0,3] 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe7681cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): picked mon.noname-c con 0x7fe7540036c0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe768123200 0x7fe768132ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): picked mon.noname-b con 0x7fe768123200 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe768133400 0x7fe7681c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): picked mon.noname-e con 0x7fe768133400 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe768124950 con 0x7fe768123200 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe76809c210 con 0x7fe7540036c0 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe76809bcc0 con 0x7fe768133400 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76f272700 10 monclient(hunting): authenticate will time out at 2686.456654s 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe7681cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d80f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe768133400 0x7fe7681c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe7681cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58038/0 (socket says 172.21.15.47:58038) 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76c80d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe768123200 0x7fe768132ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 1 -- 172.21.15.47:0/2472857779 learned_addr learned my addr 172.21.15.47:0/2472857779 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 10 monclient(hunting): get_auth_request con 0x7fe7540036c0 auth_method 0 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d80f700 10 monclient(hunting): get_auth_request con 0x7fe768133400 auth_method 0 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d80f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d80f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d80f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76c80d700 10 monclient(hunting): get_auth_request con 0x7fe768123200 auth_method 0 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76c80d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76c80d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76c80d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.055+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d80f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d80f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d80f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76c80d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76c80d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76c80d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 10 monclient(hunting): handle_auth_done global_id 84568 payload 824 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 -- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe768133400 msgr2=0x7fe7681c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe768133400 0x7fe7681c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 -- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe768123200 msgr2=0x7fe768132ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe768123200 0x7fe768132ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 monclient: found mon.noname-c 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 -- 172.21.15.47:0/2472857779 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768122510 con 0x7fe7540036c0 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76d00e700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe7681cc190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe76400ddc0 tx=0x7fe764004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 client.?.objecter ms_handle_connect 0x7fe7540036c0 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:27.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe764026070 con 0x7fe7540036c0 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: dump: 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.075 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764008e50 con 0x7fe7540036c0 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe764017ea0 con 0x7fe7540036c0 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76f272700 5 monclient: authenticate success, global_id 84568 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76f272700 10 monclient: _renew_subs 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76f272700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe768124040 con 0x7fe7540036c0 2024-04-20T02:33:27.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75f7fe700 4 set_mon_vals no callback set 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.056+0000 7fe75dffb700 10 monclient: dump: 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.077 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.078 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.079 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.057+0000 7fe76f272700 10 client.84568.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.057+0000 7fe76f272700 10 monclient: _renew_subs 2024-04-20T02:33:27.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.057+0000 7fe76f272700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.057+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7681c91d0 con 0x7fe7540036c0 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.057+0000 7fe76f272700 1 librados: init done 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe7640089e0 con 0x7fe7540036c0 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe76f272700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe76f272700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe76f272700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7681337e0 con 0x7fe7540036c0 2024-04-20T02:33:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe74c03a320 0x7fe74c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(685..685 src has 266..685) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fe76404b710 con 0x7fe7540036c0 2024-04-20T02:33:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 10 client.84568.objecter ms_dispatch 0x7fe768001550 osd_map(685..685 src has 266..685) v4 2024-04-20T02:33:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 3 client.84568.objecter handle_osd_map got epochs [685,685] > 0 2024-04-20T02:33:27.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 3 client.84568.objecter handle_osd_map decoding full epoch 685 2024-04-20T02:33:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe75dffb700 20 client.84568.objecter dump_active .. 0 homeless 2024-04-20T02:33:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe76c80d700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe74c03a320 0x7fe74c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.058+0000 7fe76c80d700 10 monclient: get_auth_request con 0x7fe74c03a320 auth_method 0 2024-04-20T02:33:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.059+0000 7fe76c80d700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe74c03a320 0x7fe74c03c7e0 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7fe76809f260 tx=0x7fe758008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.059+0000 7fe75dffb700 10 client.84568.objecter ms_handle_connect 0x7fe74c03a320 2024-04-20T02:33:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.063+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fe764004590 con 0x7fe7540036c0 2024-04-20T02:33:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.063+0000 7fe75dffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.063+0000 7fe75dffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:27.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe76f272700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:33:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe76f272700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe76f272700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe76809c210 con 0x7fe7540036c0 2024-04-20T02:33:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe75dffb700 1 -- 172.21.15.47:0/2472857779 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7fe76400dba0 con 0x7fe7540036c0 2024-04-20T02:33:27.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe75dffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:27.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.376+0000 7fe75dffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:27.378 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":108,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:33:27.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 10 librados: watch_flush enter 2024-04-20T02:33:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 10 librados: watch_flush exit 2024-04-20T02:33:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 20 client.84568.objecter shutdown clearing up homeless session... 2024-04-20T02:33:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 10 client.84568.objecter successfully canceled tick 2024-04-20T02:33:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe74c03a320 msgr2=0x7fe74c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe74c03a320 0x7fe74c03c7e0 secure :-1 s=READY pgs=3375 cs=0 l=1 rev1=1 crypto rx=0x7fe76809f260 tx=0x7fe758008040 comp rx=0 tx=0).stop 2024-04-20T02:33:27.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 10 monclient: shutdown 2024-04-20T02:33:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 msgr2=0x7fe7681cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe7681cc190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe76400ddc0 tx=0x7fe764004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 shutdown_connections 2024-04-20T02:33:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fe74c03a320 0x7fe74c03c7e0 unknown :-1 s=CLOSED pgs=3375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fe768133400 0x7fe7681c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe7540036c0 0x7fe7681cc190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 --2- 172.21.15.47:0/2472857779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe768123200 0x7fe768132ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 >> 172.21.15.47:0/2472857779 conn(0x7fe7680a6480 msgr2=0x7fe7680a3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 shutdown_connections 2024-04-20T02:33:27.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 -- 172.21.15.47:0/2472857779 wait complete. 2024-04-20T02:33:27.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.379+0000 7fe76f272700 1 librados: shutdown 2024-04-20T02:33:27.392 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:33:27.392 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-20T02:33:27.393 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:33:27.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:33:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.529+0000 7f7a0845c700 10 monclient: get_monmap_and_config 2024-04-20T02:33:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.529+0000 7f7a0845c700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.529+0000 7f7a0845c700 10 monclient: monmap: 2024-04-20T02:33:27.529 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:27.530313+0000 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:27.530313+0000 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.530 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:27.531 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.529+0000 7f7a0845c700 1 Processor -- start 2024-04-20T02:33:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.529+0000 7f7a0845c700 1 -- start start 2024-04-20T02:33:27.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.529+0000 7f7a0845c700 10 monclient: init 2024-04-20T02:33:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient: _add_conns ranks=[7,4,10,5,9,2,1,8,0,6,3] 2024-04-20T02:33:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a000a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): picked mon.noname-i con 0x7f7a00137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 0x7f7a000a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): picked mon.noname-c con 0x7f7a00138170 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 0x7f7a000a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): picked mon.noname-f con 0x7f7a001385b0 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0009fa40 con 0x7f7a001385b0 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0009dfc0 con 0x7f7a00138170 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000a7bc0 con 0x7f7a00137d90 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a0845c700 10 monclient(hunting): authenticate will time out at 2686.931660s 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 0x7f7a000a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 0x7f7a000a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58050/0 (socket says 172.21.15.47:58050) 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 1 -- 172.21.15.47:0/198406530 learned_addr learned my addr 172.21.15.47:0/198406530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): get_auth_request con 0x7f7a00138170 auth_method 0 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a059f7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a069f9700 1 -- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 msgr2=0x7f7a000a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a069f9700 1 -- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 msgr2=0x7f7a000a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a061f8700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a000a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.530+0000 7f7a069f9700 1 --2- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 0x7f7a000a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a061f8700 10 monclient(hunting): get_auth_request con 0x7f7a00137d90 auth_method 0 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a069f9700 1 --2- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 0x7f7a000a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a061f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a061f8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a061f8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 10 monclient(hunting): handle_auth_done global_id 84577 payload 274 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 -- 172.21.15.47:0/198406530 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 msgr2=0x7f7a000a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a000a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 -- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 msgr2=0x7f7a000a61d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 --2- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 0x7f7a000a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 monclient: found mon.noname-c 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 -- 172.21.15.47:0/198406530 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000751e0 con 0x7f7a00138170 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a061f8700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a000a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a059f7700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 0x7f7a000a5c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f79f00098f0 tx=0x7f79f0009c00 comp rx=0 tx=0).ready entity=mon.4 client_cookie=48f63f61999e881f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 1 -- 172.21.15.47:0/198406530 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79f000bb10 con 0x7f7a00138170 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: dump: 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 1 -- 172.21.15.47:0/198406530 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f000bc70 con 0x7f7a00138170 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a0845c700 5 monclient: authenticate success, global_id 84577 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a0845c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 1 -- 172.21.15.47:0/198406530 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79f000bde0 con 0x7f7a00138170 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a051f6700 10 monclient: dump: 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a0845c700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.531+0000 7f7a0845c700 4 set_mon_vals no callback set 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 10 monclient: shutdown 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 -- 172.21.15.47:0/198406530 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 msgr2=0x7f7a000a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 0x7f7a000a5c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f79f00098f0 tx=0x7f79f0009c00 comp rx=0 tx=0).stop 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 -- 172.21.15.47:0/198406530 shutdown_connections 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 --2- 172.21.15.47:0/198406530 >> v2:172.21.15.179:3300/0 conn(0x7f7a001385b0 0x7f7a000a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a000a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 --2- 172.21.15.47:0/198406530 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a00138170 0x7f7a000a5c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 -- 172.21.15.47:0/198406530 >> 172.21.15.47:0/198406530 conn(0x7f7a000aa8b0 msgr2=0x7f7a000a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 -- 172.21.15.47:0/198406530 shutdown_connections 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 1 -- 172.21.15.47:0/198406530 wait complete. 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.532+0000 7f7a0845c700 10 monclient: monmap: 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.549 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 librados: starting msgr at 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 librados: starting objecter 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 Processor -- start 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 -- start start 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 librados: setting wanted keys 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 librados: calling monclient init 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient: init 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient: _add_conns ranks=[7,5,2,4,8,3,1,0,6] 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a001c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): picked mon.noname-i con 0x7f7a00137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7a00138170 0x7f7a00136e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): picked mon.noname-h con 0x7f7a00138170 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a001385b0 0x7f7a001cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): picked mon.noname-b con 0x7f7a001385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a00124600 con 0x7f7a001385b0 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a00121e20 con 0x7f7a00138170 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0009dc20 con 0x7f7a00137d90 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.533+0000 7f7a0845c700 10 monclient(hunting): authenticate will time out at 2686.934967s 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a001385b0 0x7f7a001cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a001c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a001385b0 0x7f7a001cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43322/0 (socket says 172.21.15.47:43322) 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a001c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:50776/0 (socket says 172.21.15.47:50776) 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 -- 172.21.15.47:0/436753815 learned_addr learned my addr 172.21.15.47:0/436753815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7a00138170 0x7f7a00136e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): get_auth_request con 0x7f7a001385b0 auth_method 0 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): get_auth_request con 0x7f7a00137d90 auth_method 0 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): get_auth_request con 0x7f7a00138170 auth_method 0 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient(hunting): handle_auth_done global_id 104195 payload 824 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 msgr2=0x7f7a001c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7a00137d90 0x7f7a001c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7a00138170 msgr2=0x7f7a00136e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7a00138170 0x7f7a00136e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 monclient: found mon.noname-b 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009550 con 0x7f7a001385b0 2024-04-20T02:33:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a059f7700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f7a00138170 0x7f7a00136e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a069f9700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a001385b0 0x7f7a001cc270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f800be00 tx=0x7f79f8009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 client.?.objecter ms_handle_connect 0x7f7a001385b0 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79f800ce00 con 0x7f7a001385b0 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: dump: 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.554 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8012340 con 0x7f7a001385b0 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a0845c700 5 monclient: authenticate success, global_id 104195 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a0845c700 10 monclient: _renew_subs 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a0845c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0009e9e0 con 0x7f7a001385b0 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79f8012920 con 0x7f7a001385b0 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f79f6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f79f6ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.534+0000 7f7a049f5700 4 set_mon_vals no callback set 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f79f6ffd700 10 monclient: dump: 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 10 client.104195.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 10 monclient: _renew_subs 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a00137800 con 0x7f7a001385b0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 1 librados: init done 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f79f80124b0 con 0x7f7a001385b0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 0x7f7a0006aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.535+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a00068180 con 0x7f7a001373d0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79e403a2f0 0x7f79e403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(685..685 src has 266..685) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f79f804a230 con 0x7f7a001385b0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 0x7f7a0006aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 client.104195.objecter ms_dispatch 0x7f7a00001550 osd_map(685..685 src has 266..685) v4 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 3 client.104195.objecter handle_osd_map got epochs [685,685] > 0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 3 client.104195.objecter handle_osd_map decoding full epoch 685 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient: get_auth_request con 0x7f7a001373d0 auth_method 0 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 20 client.104195.objecter dump_active .. 0 homeless 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a059f7700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79e403a2f0 0x7f79e403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a059f7700 10 monclient: get_auth_request con 0x7f79e403a2f0 auth_method 0 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 10 monclient(hunting): handle_auth_done global_id 104175 payload 824 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a0009f760 con 0x7f7a001373d0 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 client.104195.objecter ms_handle_connect 0x7f7a001373d0 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 client.104195.objecter resend_mon_ops 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a061f8700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 0x7f7a0006aac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011620 tx=0x7f79fc0119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79fc0043a0 con 0x7f7a001373d0 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 monclient: dump: 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.560 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 client.104195.objecter ms_handle_connect 0x7f79e403a2f0 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f7a059f7700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79e403a2f0 0x7f79e403c7b0 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 crypto rx=0x7f79f0000c00 tx=0x7f79f0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f79fc004500 con 0x7f7a001373d0 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 msgr2=0x7f7a0006aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.536+0000 7f79f6ffd700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 0x7f7a0006aac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011620 tx=0x7f79fc0119e0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.537+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 msgr2=0x7f7a0006aac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:27.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.537+0000 7f79f6ffd700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a001373d0 0x7f7a0006aac0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011620 tx=0x7f79fc0119e0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.544+0000 7f7a0845c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.544+0000 7f7a0845c700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 0x7f7a001cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.544+0000 7f7a0845c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.544+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0006a400 con 0x7f7a00123050 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 0x7f7a001cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient: get_auth_request con 0x7f7a00123050 auth_method 0 2024-04-20T02:33:27.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 10 monclient(hunting): handle_auth_done global_id 104184 payload 824 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 1 -- 172.21.15.47:0/436753815 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7a00124460 con 0x7f7a00123050 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f7a061f8700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 0x7f7a001cc7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011620 tx=0x7f79fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 client.104195.objecter ms_handle_connect 0x7f7a00123050 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 client.104195.objecter resend_mon_ops 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f79fc0158d0 con 0x7f7a00123050 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 monclient: dump: 2024-04-20T02:33:27.564 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f79fc015460 con 0x7f7a00123050 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 msgr2=0x7f7a001cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.545+0000 7f79f6ffd700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 0x7f7a001cc7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011620 tx=0x7f79fc004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f79f6ffd700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 msgr2=0x7f7a001cc7b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f79f6ffd700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a00123050 0x7f7a001cc7b0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011620 tx=0x7f79fc004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:27.566 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:27.567 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.568 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:27.569 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:27.570 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.571 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.572 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.573 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.575 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:27.576 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.577 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.578 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.579 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.580 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.581 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.582 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.584 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:27.585 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:27.586 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:27.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 10 librados: watch_flush enter 2024-04-20T02:33:27.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 10 librados: watch_flush exit 2024-04-20T02:33:27.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 20 client.104195.objecter shutdown clearing up homeless session... 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 10 client.104195.objecter successfully canceled tick 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79e403a2f0 msgr2=0x7f79e403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f79e403a2f0 0x7f79e403c7b0 secure :-1 s=READY pgs=3376 cs=0 l=1 rev1=1 crypto rx=0x7f79f0000c00 tx=0x7f79f0008030 comp rx=0 tx=0).stop 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 10 monclient: shutdown 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a001385b0 msgr2=0x7f7a001cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 --2- 172.21.15.47:0/436753815 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a001385b0 0x7f7a001cc270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f800be00 tx=0x7f79f8009ed0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a069f9700 1 -- 172.21.15.47:0/436753815 reap_dead start 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a069f9700 1 -- 172.21.15.47:0/436753815 reap_dead start 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 shutdown_connections 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 >> 172.21.15.47:0/436753815 conn(0x7f7a000aa8b0 msgr2=0x7f7a000a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 shutdown_connections 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 -- 172.21.15.47:0/436753815 wait complete. 2024-04-20T02:33:27.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.546+0000 7f7a0845c700 1 librados: shutdown 2024-04-20T02:33:27.590 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:27.590 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:33:27.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.729+0000 7fa0ce2cd700 10 monclient: get_monmap_and_config 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.729+0000 7fa0ce2cd700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.729+0000 7fa0ce2cd700 10 monclient: monmap: 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:27.730537+0000 2024-04-20T02:33:27.729 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:27.730537+0000 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:27.730 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 Processor -- start 2024-04-20T02:33:27.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 -- start start 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient: init 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient: _add_conns ranks=[10,9,7,4,2,6,5,0,8,1,3] 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa0c80a07f0 0x7fa0c80a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): picked mon.noname-f con 0x7fa0c80a07f0 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0c80a0bd0 0x7fa0c80a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): picked mon.noname-a con 0x7fa0c80a0bd0 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa0c80a1080 0x7fa0c80a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): picked mon.noname-i con 0x7fa0c80a1080 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c8067800 con 0x7fa0c80a0bd0 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c80672f0 con 0x7fa0c80a07f0 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c809ed90 con 0x7fa0c80a1080 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0ce2cd700 10 monclient(hunting): authenticate will time out at 2687.131771s 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0c80a0bd0 0x7fa0c80a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c7fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa0c80a07f0 0x7fa0c80a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0c80a0bd0 0x7fa0c80a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45136/0 (socket says 172.21.15.47:45136) 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 1 -- 172.21.15.47:0/853236556 learned_addr learned my addr 172.21.15.47:0/853236556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 10 monclient(hunting): get_auth_request con 0x7fa0c80a0bd0 auth_method 0 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.730+0000 7fa0cc86a700 1 --2- 172.21.15.47:0/853236556 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa0c80a1080 0x7fa0c80a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): get_auth_request con 0x7fa0c80a07f0 auth_method 0 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): get_auth_request con 0x7fa0c80a1080 auth_method 0 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_done global_id 104193 payload 274 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 -- 172.21.15.47:0/853236556 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa0c80a1080 msgr2=0x7fa0c80a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 --2- 172.21.15.47:0/853236556 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa0c80a1080 0x7fa0c80a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 -- 172.21.15.47:0/853236556 >> v2:172.21.15.179:3300/0 conn(0x7fa0c80a07f0 msgr2=0x7fa0c80a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 --2- 172.21.15.47:0/853236556 >> v2:172.21.15.179:3300/0 conn(0x7fa0c80a07f0 0x7fa0c80a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 monclient: found mon.noname-a 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 -- 172.21.15.47:0/853236556 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80751e0 con 0x7fa0c80a0bd0 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c77fe700 1 --2- 172.21.15.47:0/853236556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c80a0bd0 0x7fa0c80a2870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0c806d660 tx=0x7fa0bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfb201422049277 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 1 -- 172.21.15.47:0/853236556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0bc011aa0 con 0x7fa0c80a0bd0 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: dump: 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.746 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0ce2cd700 5 monclient: authenticate success, global_id 104193 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0ce2cd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 1 -- 172.21.15.47:0/853236556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc011c00 con 0x7fa0c80a0bd0 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0ce2cd700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 1 -- 172.21.15.47:0/853236556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0bc011d70 con 0x7fa0c80a0bd0 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0ce2cd700 4 set_mon_vals no callback set 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.731+0000 7fa0c6ffd700 10 monclient: dump: 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.748 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 10 monclient: shutdown 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/853236556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c80a0bd0 msgr2=0x7fa0c80a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/853236556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c80a0bd0 0x7fa0c80a2870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0c806d660 tx=0x7fa0bc009bd0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/853236556 shutdown_connections 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/853236556 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa0c80a1080 0x7fa0c80a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/853236556 >> v2:172.21.15.179:3300/0 conn(0x7fa0c80a07f0 0x7fa0c80a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/853236556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c80a0bd0 0x7fa0c80a2870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/853236556 >> 172.21.15.47:0/853236556 conn(0x7fa0c80a3f90 msgr2=0x7fa0c806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/853236556 shutdown_connections 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.732+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/853236556 wait complete. 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 10 monclient: monmap: 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:27.750 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 1 librados: starting msgr at 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 1 librados: starting objecter 2024-04-20T02:33:27.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 1 Processor -- start 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 1 -- start start 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 1 librados: setting wanted keys 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 1 librados: calling monclient init 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.733+0000 7fa0ce2cd700 10 monclient: init 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient: _add_conns ranks=[1,5,2,4,0,6,8,7,3] 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80a07f0 0x7fa0c81cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): picked mon.noname-f con 0x7fa0c80a07f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa0c80a0bd0 0x7fa0c81ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): picked mon.noname-h con 0x7fa0c80a0bd0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 0x7fa0c81c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): picked mon.noname-b con 0x7fa0c80a1080 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a8130 con 0x7fa0c80a07f0 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c809d580 con 0x7fa0c80a1080 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c809e360 con 0x7fa0c80a0bd0 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0ce2cd700 10 monclient(hunting): authenticate will time out at 2687.135257s 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 0x7fa0c81c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 0x7fa0c81c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43324/0 (socket says 172.21.15.47:43324) 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 -- 172.21.15.47:0/1515285704 learned_addr learned my addr 172.21.15.47:0/1515285704 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): get_auth_request con 0x7fa0c80a1080 auth_method 0 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa0c80a0bd0 0x7fa0c81ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80a07f0 0x7fa0c81cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): get_auth_request con 0x7fa0c80a0bd0 auth_method 0 2024-04-20T02:33:27.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): get_auth_request con 0x7fa0c80a07f0 auth_method 0 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c77fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient(hunting): handle_auth_done global_id 104204 payload 824 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa0c80a0bd0 msgr2=0x7fa0c81ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fa0c80a0bd0 0x7fa0c81ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80a07f0 msgr2=0x7fa0c81cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c80a07f0 0x7fa0c81cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 monclient: found mon.noname-b 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009550 con 0x7fa0c80a1080 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0cc86a700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 0x7fa0c81c90e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000efc0 tx=0x7fa0c000c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c4ff9700 10 client.?.objecter ms_handle_connect 0x7fa0c80a1080 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.734+0000 7fa0c4ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:27.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0c000cce0 con 0x7fa0c80a1080 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: dump: 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.756 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 5 monclient: authenticate success, global_id 104204 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c000f040 con 0x7fa0c80a1080 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0c000ce60 con 0x7fa0c80a1080 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 monclient: _renew_subs 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c81a7b70 con 0x7fa0c80a1080 2024-04-20T02:33:27.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c67fc700 4 set_mon_vals no callback set 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0c4ff9700 10 monclient: dump: 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.758 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 client.104204.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 monclient: _renew_subs 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81bb080 con 0x7fa0c80a1080 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 1 librados: init done 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 0x7fa0c81c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.735+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8068280 con 0x7fa0c806aaa0 2024-04-20T02:33:27.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 0x7fa0c81c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient: get_auth_request con 0x7fa0c806aaa0 auth_method 0 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa0c000c870 con 0x7fa0c80a1080 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_done global_id 104213 payload 824 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0c809f350 con 0x7fa0c806aaa0 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0b403a2f0 0x7fa0b403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(685..685 src has 266..685) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fa0c004b070 con 0x7fa0c80a1080 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 10 client.104204.objecter ms_dispatch 0x7fa0c8001550 osd_map(685..685 src has 266..685) v4 2024-04-20T02:33:27.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 3 client.104204.objecter handle_osd_map got epochs [685,685] > 0 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 3 client.104204.objecter handle_osd_map decoding full epoch 685 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c7fff700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 0x7fa0c81c9620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80119d0 tx=0x7fa0b8011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 20 client.104204.objecter dump_active .. 0 homeless 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c77fe700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0b403a2f0 0x7fa0b403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 10 client.104204.objecter ms_handle_connect 0x7fa0c806aaa0 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 10 client.104204.objecter resend_mon_ops 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c77fe700 10 monclient: get_auth_request con 0x7fa0b403a2f0 auth_method 0 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0b80036a0 con 0x7fa0c806aaa0 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.736+0000 7fa0c4ff9700 10 monclient: dump: 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.761 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa0b8004240 con 0x7fa0c806aaa0 2024-04-20T02:33:27.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 msgr2=0x7fa0c81c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 0x7fa0c81c9620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80119d0 tx=0x7fa0b8011d90 comp rx=0 tx=0).stop 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 msgr2=0x7fa0c81c9620 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c806aaa0 0x7fa0c81c9620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c77fe700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0b403a2f0 0x7fa0b403c7b0 secure :-1 s=READY pgs=3377 cs=0 l=1 rev1=1 crypto rx=0x7fa0c806d660 tx=0x7fa0bc008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.737+0000 7fa0c4ff9700 10 client.104204.objecter ms_handle_connect 0x7fa0b403a2f0 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.744+0000 7fa0ce2cd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.744+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 0x7fa0c81c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.744+0000 7fa0ce2cd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.744+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a7d00 con 0x7fa0c81cd250 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 0x7fa0c81c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient: get_auth_request con 0x7fa0c81cd250 auth_method 0 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 10 monclient(hunting): handle_auth_done global_id 104222 payload 824 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 1 -- 172.21.15.47:0/1515285704 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa0c806d160 con 0x7fa0c81cd250 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c7fff700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 0x7fa0c81c9b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0b801b000 tx=0x7fa0b8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 client.104204.objecter ms_handle_connect 0x7fa0c81cd250 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 client.104204.objecter resend_mon_ops 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa0b800fe80 con 0x7fa0c81cd250 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 monclient: dump: 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.766 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7fa0b8029400 con 0x7fa0c81cd250 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:27.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 msgr2=0x7fa0c81c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.745+0000 7fa0c4ff9700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 0x7fa0c81c9b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0b801b000 tx=0x7fa0b8004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0c4ff9700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 msgr2=0x7fa0c81c9b60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0c4ff9700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c81cd250 0x7fa0c81c9b60 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0b801b000 tx=0x7fa0b8004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:27.768 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:27.769 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.770 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:27.771 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.772 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.773 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:27.774 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:27.776 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.777 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.779 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.780 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.781 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:27.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:27.783 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.785 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.786 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:27.787 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:27.788 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:27.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 10 librados: watch_flush enter 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 10 librados: watch_flush exit 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 20 client.104204.objecter shutdown clearing up homeless session... 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 10 client.104204.objecter successfully canceled tick 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0b403a2f0 msgr2=0x7fa0b403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa0b403a2f0 0x7fa0b403c7b0 secure :-1 s=READY pgs=3377 cs=0 l=1 rev1=1 crypto rx=0x7fa0c806d660 tx=0x7fa0bc008030 comp rx=0 tx=0).stop 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 10 monclient: shutdown 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 msgr2=0x7fa0c81c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 0x7fa0c81c90e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000efc0 tx=0x7fa0c000c0c0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0cc86a700 1 -- 172.21.15.47:0/1515285704 reap_dead start 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 shutdown_connections 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 --2- 172.21.15.47:0/1515285704 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa0c80a1080 0x7fa0c81c90e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 >> 172.21.15.47:0/1515285704 conn(0x7fa0c80a3f90 msgr2=0x7fa0c81bdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 shutdown_connections 2024-04-20T02:33:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 -- 172.21.15.47:0/1515285704 wait complete. 2024-04-20T02:33:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.746+0000 7fa0ce2cd700 1 librados: shutdown 2024-04-20T02:33:27.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:27.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:33:27.792 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:33:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.940+0000 7f4f8560e700 10 monclient: get_monmap_and_config 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.940+0000 7f4f8560e700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.940+0000 7f4f8560e700 10 monclient: monmap: 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:27.941937+0000 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:27.941937+0000 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.941 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:27.942 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:27.942 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.941+0000 7f4f8560e700 1 Processor -- start 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.941+0000 7f4f8560e700 1 -- start start 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.941+0000 7f4f8560e700 10 monclient: init 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.941+0000 7f4f8560e700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.941+0000 7f4f8560e700 10 monclient: _add_conns ranks=[3,10,2,1,8,6,4,7,9,5,0] 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.941+0000 7f4f8560e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f80137d90 0x7f4f800a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): picked mon.noname-g con 0x7f4f80137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4f80138170 0x7f4f800a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): picked mon.noname-f con 0x7f4f80138170 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 0x7f4f800a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): picked mon.noname-b con 0x7f4f801385b0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8009fa40 con 0x7f4f80138170 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8009dfc0 con 0x7f4f801385b0 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800a7bc0 con 0x7f4f80137d90 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f8560e700 10 monclient(hunting): authenticate will time out at 2687.343083s 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 0x7f4f800a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 0x7f4f800a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43354/0 (socket says 172.21.15.47:43354) 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 -- 172.21.15.47:0/1934321062 learned_addr learned my addr 172.21.15.47:0/1934321062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): get_auth_request con 0x7f4f801385b0 auth_method 0 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7e7fc700 1 --2- 172.21.15.47:0/1934321062 >> v2:172.21.15.179:3300/0 conn(0x7f4f80138170 0x7f4f800a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7e7fc700 10 monclient(hunting): get_auth_request con 0x7f4f80138170 auth_method 0 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7effd700 1 --2- 172.21.15.47:0/1934321062 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f80137d90 0x7f4f800a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7effd700 10 monclient(hunting): get_auth_request con 0x7f4f80137d90 auth_method 0 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient(hunting): handle_auth_done global_id 104231 payload 274 2024-04-20T02:33:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 -- 172.21.15.47:0/1934321062 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f80137d90 msgr2=0x7f4f800a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1934321062 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f80137d90 0x7f4f800a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 -- 172.21.15.47:0/1934321062 >> v2:172.21.15.179:3300/0 conn(0x7f4f80138170 msgr2=0x7f4f800a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1934321062 >> v2:172.21.15.179:3300/0 conn(0x7f4f80138170 0x7f4f800a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 monclient: found mon.noname-b 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 -- 172.21.15.47:0/1934321062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800751e0 con 0x7f4f801385b0 2024-04-20T02:33:27.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7e7fc700 1 --2- 172.21.15.47:0/1934321062 >> v2:172.21.15.179:3300/0 conn(0x7f4f80138170 0x7f4f800a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1934321062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 0x7f4f800a61d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009e80 tx=0x7f4f7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f315d7fff1dc3b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7dffb700 1 -- 172.21.15.47:0/1934321062 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f7800e070 con 0x7f4f801385b0 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.942+0000 7f4f7dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: dump: 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 1 -- 172.21.15.47:0/1934321062 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7800bdf0 con 0x7f4f801385b0 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 5 monclient: authenticate success, global_id 104231 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 1 -- 172.21.15.47:0/1934321062 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f78012860 con 0x7f4f801385b0 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 4 set_mon_vals no callback set 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f7dffb700 10 monclient: dump: 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 10 monclient: shutdown 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 -- 172.21.15.47:0/1934321062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 msgr2=0x7f4f800a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1934321062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 0x7f4f800a61d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009e80 tx=0x7f4f7800b330 comp rx=0 tx=0).stop 2024-04-20T02:33:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 -- 172.21.15.47:0/1934321062 shutdown_connections 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1934321062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4f801385b0 0x7f4f800a61d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1934321062 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4f80137d90 0x7f4f800a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1934321062 >> v2:172.21.15.179:3300/0 conn(0x7f4f80138170 0x7f4f800a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 -- 172.21.15.47:0/1934321062 >> 172.21.15.47:0/1934321062 conn(0x7f4f800aa8b0 msgr2=0x7f4f800a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 -- 172.21.15.47:0/1934321062 shutdown_connections 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.943+0000 7f4f8560e700 1 -- 172.21.15.47:0/1934321062 wait complete. 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.944+0000 7f4f8560e700 10 monclient: build_initial_monmap 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.944+0000 7f4f8560e700 10 monclient: monmap: 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.944+0000 7f4f8560e700 1 librados: starting msgr at 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.944+0000 7f4f8560e700 1 librados: starting objecter 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.945+0000 7f4f8560e700 1 Processor -- start 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.945+0000 7f4f8560e700 1 -- start start 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.945+0000 7f4f8560e700 1 librados: setting wanted keys 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.945+0000 7f4f8560e700 1 librados: calling monclient init 2024-04-20T02:33:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.945+0000 7f4f8560e700 10 monclient: init 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.945+0000 7f4f8560e700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient: _add_conns ranks=[4,6,1,7,3,5,0,2,8] 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f80137d90 0x7f4f801cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): picked mon.noname-c con 0x7f4f80137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4f80138170 0x7f4f80136c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): picked mon.noname-d con 0x7f4f80138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f801385b0 0x7f4f80137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): picked mon.noname-f con 0x7f4f801385b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8009ea50 con 0x7f4f801385b0 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80123a00 con 0x7f4f80137d90 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4f801222f0 con 0x7f4f80138170 2024-04-20T02:33:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f8560e700 10 monclient(hunting): authenticate will time out at 2687.347118s 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f80137d90 0x7f4f801cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f80137d90 0x7f4f801cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58074/0 (socket says 172.21.15.47:58074) 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 1 -- 172.21.15.47:0/1557131003 learned_addr learned my addr 172.21.15.47:0/1557131003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4f80138170 0x7f4f80136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): get_auth_request con 0x7f4f80137d90 auth_method 0 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): get_auth_request con 0x7f4f80138170 auth_method 0 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f801385b0 0x7f4f80137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7f7fe700 10 monclient(hunting): get_auth_request con 0x7f4f801385b0 auth_method 0 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient(hunting): handle_auth_done global_id 55392 payload 824 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f80137d90 msgr2=0x7f4f801cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f80137d90 0x7f4f801cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f801385b0 msgr2=0x7f4f80137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f801385b0 0x7f4f80137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:27.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 monclient: found mon.noname-d 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f78009b30 con 0x7f4f80138170 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f801385b0 0x7f4f80137170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7effd700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f80137d90 0x7f4f801cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f7e7fc700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4f80138170 0x7f4f80136c30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800e7e0 tx=0x7f4f6800eaf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f777fe700 10 client.?.objecter ms_handle_connect 0x7f4f80138170 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f777fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f6800c070 con 0x7f4f80138170 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.946+0000 7f4f777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: dump: 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.967 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f68013880 con 0x7f4f80138170 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 5 monclient: authenticate success, global_id 55392 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 monclient: _renew_subs 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8009dc20 con 0x7f4f80138170 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f68013e60 con 0x7f4f80138170 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 10 monclient: dump: 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.969 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f7d7fa700 4 set_mon_vals no callback set 2024-04-20T02:33:27.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 client.55392.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 monclient: _renew_subs 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f80122e60 con 0x7f4f80138170 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 1 librados: init done 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 0x7f4f801208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f4f680139f0 con 0x7f4f80138170 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.947+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80124110 con 0x7f4f801376b0 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 0x7f4f801208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient: get_auth_request con 0x7f4f801376b0 auth_method 0 2024-04-20T02:33:27.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4f6003a2f0 0x7f4f6003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(686..686 src has 266..686) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f4f6804a5d0 con 0x7f4f80138170 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 10 client.55392.objecter ms_dispatch 0x7f4f80001550 osd_map(686..686 src has 266..686) v4 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 3 client.55392.objecter handle_osd_map got epochs [686,686] > 0 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 3 client.55392.objecter handle_osd_map decoding full epoch 686 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 10 monclient(hunting): handle_auth_done global_id 84595 payload 824 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 20 client.55392.objecter dump_active .. 0 homeless 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8009f760 con 0x7f4f801376b0 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4f6003a2f0 0x7f4f6003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7f7fe700 10 monclient: get_auth_request con 0x7f4f6003a2f0 auth_method 0 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 10 client.55392.objecter ms_handle_connect 0x7f4f801376b0 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f7effd700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 0x7f4f801208d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f70011ab0 tx=0x7f4f70011e70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 10 client.55392.objecter resend_mon_ops 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f7000c070 con 0x7f4f801376b0 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.948+0000 7f4f777fe700 10 monclient: dump: 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.973 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 10 client.55392.objecter ms_handle_connect 0x7f4f6003a2f0 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f7f7fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4f6003a2f0 0x7f4f6003c7b0 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 crypto rx=0x7f4f800a6d50 tx=0x7f4f78009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4f7000f920 con 0x7f4f801376b0 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:27.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 msgr2=0x7f4f801208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 0x7f4f801208d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f70011ab0 tx=0x7f4f70011e70 comp rx=0 tx=0).stop 2024-04-20T02:33:27.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 msgr2=0x7f4f801208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:27.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.949+0000 7f4f777fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f801376b0 0x7f4f801208d0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f70011ab0 tx=0x7f4f70011e70 comp rx=0 tx=0).stop 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.956+0000 7f4f8560e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.956+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 0x7f4f80120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.956+0000 7f4f8560e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80068180 con 0x7f4f8006aac0 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 0x7f4f80120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient: get_auth_request con 0x7f4f8006aac0 auth_method 0 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 10 monclient(hunting): handle_auth_done global_id 84604 payload 824 2024-04-20T02:33:27.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 1 -- 172.21.15.47:0/1557131003 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f8009e430 con 0x7f4f8006aac0 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f777fe700 10 client.55392.objecter ms_handle_connect 0x7f4f8006aac0 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f7effd700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 0x7f4f80120e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f7001b000 tx=0x7f4f70004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f777fe700 10 client.55392.objecter resend_mon_ops 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f700044a0 con 0x7f4f8006aac0 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f777fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.957+0000 7f4f777fe700 10 monclient: dump: 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.978 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f4f70024040 con 0x7f4f8006aac0 2024-04-20T02:33:27.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 msgr2=0x7f4f80120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 0x7f4f80120e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f7001b000 tx=0x7f4f70004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 msgr2=0x7f4f80120e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f777fe700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4f8006aac0 0x7f4f80120e10 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4f7001b000 tx=0x7f4f70004750 comp rx=0 tx=0).stop 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:27.980 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 4, 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:27.981 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.982 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:27.983 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:27.984 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.986 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.987 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.988 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.989 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.990 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.991 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.992 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.994 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.995 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:27.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.997 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.998 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:27.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:28.000 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 10 librados: watch_flush enter 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 10 librados: watch_flush exit 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 20 client.55392.objecter shutdown clearing up homeless session... 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 10 client.55392.objecter successfully canceled tick 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4f6003a2f0 msgr2=0x7f4f6003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f4f6003a2f0 0x7f4f6003c7b0 secure :-1 s=READY pgs=3378 cs=0 l=1 rev1=1 crypto rx=0x7f4f800a6d50 tx=0x7f4f78009000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 10 monclient: shutdown 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4f80138170 msgr2=0x7f4f80136c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4f80138170 0x7f4f80136c30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800e7e0 tx=0x7f4f6800eaf0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f7f7fe700 1 -- 172.21.15.47:0/1557131003 reap_dead start 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 shutdown_connections 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 --2- 172.21.15.47:0/1557131003 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f4f80138170 0x7f4f80136c30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 >> 172.21.15.47:0/1557131003 conn(0x7f4f800aa8b0 msgr2=0x7f4f800a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 shutdown_connections 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 -- 172.21.15.47:0/1557131003 wait complete. 2024-04-20T02:33:28.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:27.958+0000 7f4f8560e700 1 librados: shutdown 2024-04-20T02:33:28.007 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:28.007 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:33:28.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.147+0000 7f11b25bd700 10 monclient: get_monmap_and_config 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.147+0000 7f11b25bd700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.147+0000 7f11b25bd700 10 monclient: monmap: 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:28.148257+0000 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:28.148257+0000 2024-04-20T02:33:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.148 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:28.149 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:28.149 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.147+0000 7f11b25bd700 1 Processor -- start 2024-04-20T02:33:28.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 1 -- start start 2024-04-20T02:33:28.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient: init 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient: _add_conns ranks=[9,7,0,3,1,6,4,8,5,2,10] 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11ac09d060 0x7f11ac09c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): picked mon.noname-a con 0x7f11ac09d060 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): picked mon.noname-i con 0x7f11ac0a1f10 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11ac0a2870 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac0751e0 con 0x7f11ac0a2870 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac0a3dd0 con 0x7f11ac09d060 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac09d440 con 0x7f11ac0a1f10 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b25bd700 10 monclient(hunting): authenticate will time out at 2687.549598s 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f11ac09d060 0x7f11ac09c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b0b5a700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f11ac0a2870 0x7f11ac13e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33438/0 (socket says 172.21.15.47:33438) 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11b0b5a700 1 -- 172.21.15.47:0/3627072207 learned_addr learned my addr 172.21.15.47:0/3627072207 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): get_auth_request con 0x7f11ac09d060 auth_method 0 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11ab7fe700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11ab7fe700 10 monclient(hunting): get_auth_request con 0x7f11ac0a1f10 auth_method 0 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11ab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11ab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.148+0000 7f11ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 10 monclient(hunting): handle_auth_done global_id 104202 payload 274 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 1 -- 172.21.15.47:0/3627072207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11ac0a1f10 msgr2=0x7f11ac0a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:28.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 1 -- 172.21.15.47:0/3627072207 >> v1:172.21.15.47:6789/0 conn(0x7f11ac0a2870 legacy=0x7f11ac13e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 1 monclient: found mon.noname-a 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 1 -- 172.21.15.47:0/3627072207 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac09b730 con 0x7f11ac09d060 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 1 -- 172.21.15.47:0/3627072207 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624601643 0 0) 0x7f11ac0751e0 con 0x7f11ac0a2870 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11ab7fe700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11abfff700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac09d060 0x7f11ac09c3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f119c009840 tx=0x7f119c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f7ea4e0c91545c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 1 -- 172.21.15.47:0/3627072207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f119c011aa0 con 0x7f11ac09d060 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: dump: 2024-04-20T02:33:28.164 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.165 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11b25bd700 5 monclient: authenticate success, global_id 104202 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11b25bd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 1 -- 172.21.15.47:0/3627072207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c011c00 con 0x7f11ac09d060 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11b25bd700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11b25bd700 4 set_mon_vals no callback set 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 1 -- 172.21.15.47:0/3627072207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f119c011d70 con 0x7f11ac09d060 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.149+0000 7f11aaffd700 10 monclient: dump: 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.166 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.167 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 10 monclient: shutdown 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 -- 172.21.15.47:0/3627072207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac09d060 msgr2=0x7f11ac09c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac09d060 0x7f11ac09c3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f119c009840 tx=0x7f119c009b50 comp rx=0 tx=0).stop 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 -- 172.21.15.47:0/3627072207 shutdown_connections 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac09d060 0x7f11ac09c3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3627072207 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 -- 172.21.15.47:0/3627072207 >> 172.21.15.47:0/3627072207 conn(0x7f11ac0a3000 msgr2=0x7f11ac0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 -- 172.21.15.47:0/3627072207 shutdown_connections 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 -- 172.21.15.47:0/3627072207 wait complete. 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 10 monclient: monmap: 2024-04-20T02:33:28.168 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.169 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 librados: starting msgr at 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.150+0000 7f11b25bd700 1 librados: starting objecter 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 Processor -- start 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 -- start start 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 librados: setting wanted keys 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 librados: calling monclient init 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient: init 2024-04-20T02:33:28.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient: _add_conns ranks=[4,6,0,5,7,1,2,3,8] 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11ac09d060 0x7f11ac072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): picked mon.noname-c con 0x7f11ac09d060 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0902d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): picked mon.noname-d con 0x7f11ac0a1f10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac0a2870 0x7f11ac090810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): picked mon.noname-a con 0x7f11ac0a2870 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac09ae70 con 0x7f11ac0a2870 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0930f0 con 0x7f11ac09d060 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac09a940 con 0x7f11ac0a1f10 2024-04-20T02:33:28.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.151+0000 7f11b25bd700 10 monclient(hunting): authenticate will time out at 2687.552872s 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11ac09d060 0x7f11ac072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50878/0 (socket says 172.21.15.47:50878) 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac0a2870 0x7f11ac090810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 1 -- 172.21.15.47:0/3376840648 learned_addr learned my addr 172.21.15.47:0/3376840648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): get_auth_request con 0x7f11ac0a1f10 auth_method 0 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): get_auth_request con 0x7f11ac09d060 auth_method 0 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): get_auth_request con 0x7f11ac0a2870 auth_method 0 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient(hunting): handle_auth_done global_id 104220 payload 824 2024-04-20T02:33:28.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac0a1f10 msgr2=0x7f11ac0902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0902d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11ac09d060 msgr2=0x7f11ac072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11ac09d060 0x7f11ac072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 monclient: found mon.noname-a 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0094a0 con 0x7f11ac0a2870 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11ab7fe700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac0a1f10 0x7f11ac0902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11abfff700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11ac09d060 0x7f11ac072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b0b5a700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac0a2870 0x7f11ac090810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f11a400cad0 tx=0x7f11a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 client.?.objecter ms_handle_connect 0x7f11ac0a2870 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11a400ac10 con 0x7f11ac0a2870 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: dump: 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.176 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b25bd700 5 monclient: authenticate success, global_id 104220 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b25bd700 10 monclient: _renew_subs 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a400ad90 con 0x7f11ac0a2870 2024-04-20T02:33:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b25bd700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac092930 con 0x7f11ac0a2870 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11a4007540 con 0x7f11ac0a2870 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11aa7fc700 4 set_mon_vals no callback set 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11b25bd700 10 client.104220.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.152+0000 7f11a8ff9700 10 monclient: dump: 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.178 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.182 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.182 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.182 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 10 monclient: _renew_subs 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac0913d0 con 0x7f11ac0a2870 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 1 librados: init done 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f11a40070d0 con 0x7f11ac0a2870 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 0x7f11ac1c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.153+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac068180 con 0x7f11ac06aac0 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 0x7f11ac1c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119803a2f0 0x7f119803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient: get_auth_request con 0x7f11ac06aac0 auth_method 0 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(686..686 src has 266..686) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f11a404a0f0 con 0x7f11ac0a2870 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 client.104220.objecter ms_dispatch 0x7f11ac001550 osd_map(686..686 src has 266..686) v4 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 3 client.104220.objecter handle_osd_map got epochs [686,686] > 0 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 3 client.104220.objecter handle_osd_map decoding full epoch 686 2024-04-20T02:33:28.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 20 client.104220.objecter dump_active .. 0 homeless 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 10 monclient(hunting): handle_auth_done global_id 55410 payload 824 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11ac09c440 con 0x7f11ac06aac0 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11ab7fe700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119803a2f0 0x7f119803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11ab7fe700 10 monclient: get_auth_request con 0x7f119803a2f0 auth_method 0 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 client.104220.objecter ms_handle_connect 0x7f11ac06aac0 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 client.104220.objecter resend_mon_ops 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11abfff700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 0x7f11ac1c8e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f119c009840 tx=0x7f119c004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f119c020070 con 0x7f11ac06aac0 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 monclient: dump: 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 client.104220.objecter ms_handle_connect 0x7f119803a2f0 2024-04-20T02:33:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11ab7fe700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119803a2f0 0x7f119803c7b0 secure :-1 s=READY pgs=3379 cs=0 l=1 rev1=1 crypto rx=0x7f11ac072ce0 tx=0x7f11a0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f119c0038b0 con 0x7f11ac06aac0 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 msgr2=0x7f11ac1c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 0x7f11ac1c8e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f119c009840 tx=0x7f119c004750 comp rx=0 tx=0).stop 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 msgr2=0x7f11ac1c8e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:28.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.154+0000 7f11a8ff9700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac06aac0 0x7f11ac1c8e20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11b25bd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 0x7f11ac1c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11b25bd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac06a400 con 0x7f11ac090db0 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 0x7f11ac1c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient: get_auth_request con 0x7f11ac090db0 auth_method 0 2024-04-20T02:33:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.162+0000 7f11abfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11abfff700 10 monclient(hunting): handle_auth_done global_id 55419 payload 824 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11abfff700 1 -- 172.21.15.47:0/3376840648 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f11ac0a2cf0 con 0x7f11ac090db0 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 client.104220.objecter ms_handle_connect 0x7f11ac090db0 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11abfff700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 0x7f11ac1c9360 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f119c024000 tx=0x7f119c011000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 client.104220.objecter resend_mon_ops 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f119c01b960 con 0x7f11ac090db0 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 monclient: dump: 2024-04-20T02:33:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.191 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f119c01b4f0 con 0x7f11ac090db0 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 msgr2=0x7f11ac1c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 0x7f11ac1c9360 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f119c024000 tx=0x7f119c011000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 msgr2=0x7f11ac1c9360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.163+0000 7f11a8ff9700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f11ac090db0 0x7f11ac1c9360 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f119c024000 tx=0x7f119c011000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:28.192 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:28.193 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.194 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.195 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:28.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.197 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:28.199 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:28.200 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:28.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.204 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:28.211 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.212 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:28.213 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.214 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:28.215 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:28.216 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.217 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:28.218 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 10 librados: watch_flush enter 2024-04-20T02:33:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 10 librados: watch_flush exit 2024-04-20T02:33:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 20 client.104220.objecter shutdown clearing up homeless session... 2024-04-20T02:33:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 10 client.104220.objecter successfully canceled tick 2024-04-20T02:33:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119803a2f0 msgr2=0x7f119803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f119803a2f0 0x7f119803c7b0 secure :-1 s=READY pgs=3379 cs=0 l=1 rev1=1 crypto rx=0x7f11ac072ce0 tx=0x7f11a0008040 comp rx=0 tx=0).stop 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 10 monclient: shutdown 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac0a2870 msgr2=0x7f11ac090810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 --2- 172.21.15.47:0/3376840648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11ac0a2870 0x7f11ac090810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f11a400cad0 tx=0x7f11a400cde0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b0b5a700 1 -- 172.21.15.47:0/3376840648 reap_dead start 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b0b5a700 1 -- 172.21.15.47:0/3376840648 reap_dead start 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 shutdown_connections 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 >> 172.21.15.47:0/3376840648 conn(0x7f11ac0a3000 msgr2=0x7f11ac1bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 shutdown_connections 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 -- 172.21.15.47:0/3376840648 wait complete. 2024-04-20T02:33:28.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.164+0000 7f11b25bd700 1 librados: shutdown 2024-04-20T02:33:28.221 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:28.222 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:33:28.222 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.361+0000 7f745fde7700 10 monclient: get_monmap_and_config 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.361+0000 7f745fde7700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.361+0000 7f745fde7700 10 monclient: monmap: 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:28.362269+0000 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:28.362269+0000 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:28.361 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:28.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.361+0000 7f745fde7700 1 Processor -- start 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.361+0000 7f745fde7700 1 -- start start 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient: init 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient: _add_conns ranks=[2,7,0,1,8,6,3,10,9,4,5] 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): picked mon.noname-b con 0x7f74580a50f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74581378a0 0x7f7458138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): picked mon.noname-i con 0x7f74581378a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f74580a38f0 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580751e0 con 0x7f74580a38f0 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580a54d0 con 0x7f74580a50f0 2024-04-20T02:33:28.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f74581387b0 con 0x7f74581378a0 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745fde7700 10 monclient(hunting): authenticate will time out at 2687.763564s 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745e384700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f74580a38f0 0x7f74580a3d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33446/0 (socket says 172.21.15.47:33446) 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745e384700 1 -- 172.21.15.47:0/4213837498 learned_addr learned my addr 172.21.15.47:0/4213837498 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): get_auth_request con 0x7f74580a50f0 auth_method 0 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745d382700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74581378a0 0x7f7458138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745d382700 10 monclient(hunting): get_auth_request con 0x7f74581378a0 auth_method 0 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745d382700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745d382700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.362+0000 7f745d382700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 10 monclient(hunting): handle_auth_done global_id 104240 payload 274 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 1 -- 172.21.15.47:0/4213837498 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74581378a0 msgr2=0x7f7458138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74581378a0 0x7f7458138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 1 -- 172.21.15.47:0/4213837498 >> v1:172.21.15.47:6789/0 conn(0x7f74580a38f0 legacy=0x7f74580a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 1 monclient: found mon.noname-b 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 1 -- 172.21.15.47:0/4213837498 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745806da20 con 0x7f74580a50f0 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 1 -- 172.21.15.47:0/4213837498 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225888915 0 0) 0x7f74580751e0 con 0x7f74580a38f0 2024-04-20T02:33:28.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745d382700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74581378a0 0x7f7458138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745db83700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458137d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7454009870 tx=0x7f7454009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ed9eba1aeaceec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 1 -- 172.21.15.47:0/4213837498 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7454011aa0 con 0x7f74580a50f0 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: dump: 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.379 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 1 -- 172.21.15.47:0/4213837498 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7454011c00 con 0x7f74580a50f0 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745fde7700 5 monclient: authenticate success, global_id 104240 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745fde7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:28.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745fde7700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 1 -- 172.21.15.47:0/4213837498 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7454011d70 con 0x7f74580a50f0 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745fde7700 4 set_mon_vals no callback set 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.363+0000 7f745cb81700 10 monclient: dump: 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.381 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 10 monclient: shutdown 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 -- 172.21.15.47:0/4213837498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 msgr2=0x7f7458137d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458137d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7454009870 tx=0x7f7454009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 -- 172.21.15.47:0/4213837498 shutdown_connections 2024-04-20T02:33:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458137d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 --2- 172.21.15.47:0/4213837498 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74581378a0 0x7f7458138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 -- 172.21.15.47:0/4213837498 >> 172.21.15.47:0/4213837498 conn(0x7f74580a8510 msgr2=0x7f74580a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 -- 172.21.15.47:0/4213837498 shutdown_connections 2024-04-20T02:33:28.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 1 -- 172.21.15.47:0/4213837498 wait complete. 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.364+0000 7f745fde7700 10 monclient: monmap: 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:28.384 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 librados: starting msgr at 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 librados: starting objecter 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 Processor -- start 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 -- start start 2024-04-20T02:33:28.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 librados: setting wanted keys 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 librados: calling monclient init 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient: init 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient: _add_conns ranks=[7,2,6,5,4,3,0,8,1] 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74580a38f0 0x7f74581cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): picked mon.noname-i con 0x7f74580a38f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458136910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): picked mon.noname-b con 0x7f74580a50f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7458136e50 0x7f74581202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): picked mon.noname-d con 0x7f7458136e50 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7458123a30 con 0x7f74580a50f0 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f74581cceb0 con 0x7f7458136e50 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7458072260 con 0x7f74580a38f0 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745fde7700 10 monclient(hunting): authenticate will time out at 2687.766770s 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745e384700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7458136e50 0x7f74581202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458136910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43368/0 (socket says 172.21.15.47:43368) 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 1 -- 172.21.15.47:0/1089340117 learned_addr learned my addr 172.21.15.47:0/1089340117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 10 monclient(hunting): get_auth_request con 0x7f74580a50f0 auth_method 0 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745d382700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745e384700 10 monclient(hunting): get_auth_request con 0x7f7458136e50 auth_method 0 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745e384700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745e384700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.365+0000 7f745e384700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74580a38f0 0x7f74581cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): get_auth_request con 0x7f74580a38f0 auth_method 0 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745e384700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745e384700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745e384700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 10 monclient(hunting): handle_auth_done global_id 104249 payload 824 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74580a38f0 msgr2=0x7f74581cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f74580a38f0 0x7f74581cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7458136e50 msgr2=0x7f74581202f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7458136e50 0x7f74581202f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 monclient: found mon.noname-b 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540094a0 con 0x7f74580a50f0 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745e384700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7458136e50 0x7f74581202f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745d382700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458136910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f744800e810 tx=0x7f744800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 client.?.objecter ms_handle_connect 0x7f74580a50f0 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:28.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f744800c070 con 0x7f74580a50f0 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: dump: 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.390 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448013880 con 0x7f74580a50f0 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7448013e60 con 0x7f74580a50f0 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 5 monclient: authenticate success, global_id 104249 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 10 monclient: _renew_subs 2024-04-20T02:33:28.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7458071e80 con 0x7f74580a50f0 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744ffff700 4 set_mon_vals no callback set 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f744e7fc700 10 monclient: dump: 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.392 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.393 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.394 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 10 client.104249.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:28.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 10 monclient: _renew_subs 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7458121f70 con 0x7f74580a50f0 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.366+0000 7f745fde7700 1 librados: init done 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f74480139f0 con 0x7f74580a50f0 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745fde7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745fde7700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 0x7f7458120830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f74580682f0 con 0x7f74581378a0 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f743c03a2f0 0x7f743c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 0x7f7458120830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient: get_auth_request con 0x7f74581378a0 auth_method 0 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(686..686 src has 266..686) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f744804a5b0 con 0x7f74580a50f0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 10 client.104249.objecter ms_dispatch 0x7f7458001550 osd_map(686..686 src has 266..686) v4 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 3 client.104249.objecter handle_osd_map got epochs [686,686] > 0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 3 client.104249.objecter handle_osd_map decoding full epoch 686 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.367+0000 7f744e7fc700 20 client.104249.objecter dump_active .. 0 homeless 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f745db83700 10 monclient(hunting): handle_auth_done global_id 36224 payload 824 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f745e384700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f743c03a2f0 0x7f743c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f745db83700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f745806e370 con 0x7f74581378a0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f745e384700 10 monclient: get_auth_request con 0x7f743c03a2f0 auth_method 0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 client.104249.objecter ms_handle_connect 0x7f74581378a0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 client.104249.objecter resend_mon_ops 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f745db83700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 0x7f7458120830 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7454009870 tx=0x7f7454004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7454020070 con 0x7f74581378a0 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 monclient: dump: 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 client.104249.objecter ms_handle_connect 0x7f743c03a2f0 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f745e384700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f743c03a2f0 0x7f743c03c7b0 secure :-1 s=READY pgs=3380 cs=0 l=1 rev1=1 crypto rx=0x7f74500012e0 tx=0x7f7450007080 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f74540036a0 con 0x7f74581378a0 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 msgr2=0x7f7458120830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 0x7f7458120830 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7454009870 tx=0x7f7454004750 comp rx=0 tx=0).stop 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 msgr2=0x7f7458120830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:28.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.368+0000 7f744e7fc700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f74581378a0 0x7f7458120830 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7454009870 tx=0x7f7454004750 comp rx=0 tx=0).stop 2024-04-20T02:33:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745fde7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745fde7700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 0x7f7458120d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745fde7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f745806a400 con 0x7f745806aac0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 0x7f7458120d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient: get_auth_request con 0x7f745806aac0 auth_method 0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.376+0000 7f745db83700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f745db83700 10 monclient(hunting): handle_auth_done global_id 36233 payload 824 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f745db83700 1 -- 172.21.15.47:0/1089340117 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f745806e650 con 0x7f745806aac0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f745db83700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 0x7f7458120d70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7454009bc0 tx=0x7f7454011000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 client.104249.objecter ms_handle_connect 0x7f745806aac0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 client.104249.objecter resend_mon_ops 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7454004660 con 0x7f745806aac0 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 monclient: dump: 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.402 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9354 (secure 0 0 0) 0x7f745401b430 con 0x7f745806aac0 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 msgr2=0x7f7458120d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 0x7f7458120d70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7454009bc0 tx=0x7f7454011000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 msgr2=0x7f7458120d70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.377+0000 7f744e7fc700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f745806aac0 0x7f7458120d70 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7454009bc0 tx=0x7f7454011000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:28.403 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.404 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:28.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:28.406 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:28.407 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.408 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.411 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:28.413 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.415 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:28.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.418 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:28.422 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.424 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:28.424 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:28.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 10 librados: watch_flush enter 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 10 librados: watch_flush exit 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 20 client.104249.objecter shutdown clearing up homeless session... 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 10 client.104249.objecter successfully canceled tick 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f743c03a2f0 msgr2=0x7f743c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f743c03a2f0 0x7f743c03c7b0 secure :-1 s=READY pgs=3380 cs=0 l=1 rev1=1 crypto rx=0x7f74500012e0 tx=0x7f7450007080 comp rx=0 tx=0).stop 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 10 monclient: shutdown 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 msgr2=0x7f7458136910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 --2- 172.21.15.47:0/1089340117 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f74580a50f0 0x7f7458136910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f744800e810 tx=0x7f744800eb20 comp rx=0 tx=0).stop 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745e384700 1 -- 172.21.15.47:0/1089340117 reap_dead start 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745e384700 1 -- 172.21.15.47:0/1089340117 reap_dead start 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 shutdown_connections 2024-04-20T02:33:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 >> 172.21.15.47:0/1089340117 conn(0x7f74580a8510 msgr2=0x7f7458070450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 shutdown_connections 2024-04-20T02:33:28.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 -- 172.21.15.47:0/1089340117 wait complete. 2024-04-20T02:33:28.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.378+0000 7f745fde7700 1 librados: shutdown 2024-04-20T02:33:28.427 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:28.427 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:33:28.427 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.574+0000 7fc22759e700 10 monclient: get_monmap_and_config 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.574+0000 7fc22759e700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.574+0000 7fc22759e700 10 monclient: monmap: 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:28.575352+0000 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:28.575352+0000 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.574 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:28.576 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.574+0000 7fc22759e700 1 Processor -- start 2024-04-20T02:33:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 -- start start 2024-04-20T02:33:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient: init 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient: _add_conns ranks=[8,9,4,0,6,5,3,7,1,2,10] 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 0x7fc2200a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): picked mon.noname-e con 0x7fc2200a07f0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2200a0bd0 0x7fc2200a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): picked mon.noname-a con 0x7fc2200a0bd0 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2200a1050 0x7fc2200a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): picked mon.noname-c con 0x7fc2200a1050 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:28.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc220067800 con 0x7fc2200a0bd0 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200672f0 con 0x7fc2200a1050 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc22009ed90 con 0x7fc2200a07f0 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22759e700 10 monclient(hunting): authenticate will time out at 2687.976700s 2024-04-20T02:33:28.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 0x7fc2200a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc225b3b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2200a1050 0x7fc2200a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 0x7fc2200a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42864/0 (socket says 172.21.15.47:42864) 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 1 -- 172.21.15.47:0/2505825997 learned_addr learned my addr 172.21.15.47:0/2505825997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 10 monclient(hunting): get_auth_request con 0x7fc2200a07f0 auth_method 0 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc225b3b700 10 monclient(hunting): get_auth_request con 0x7fc2200a1050 auth_method 0 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc225b3b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc225b3b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc225b3b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc224b39700 1 --2- 172.21.15.47:0/2505825997 >> v2:172.21.15.47:3300/0 conn(0x7fc2200a0bd0 0x7fc2200a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc224b39700 10 monclient(hunting): get_auth_request con 0x7fc2200a0bd0 auth_method 0 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc224b39700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc224b39700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc224b39700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.575+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc225b3b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc225b3b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc225b3b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc224b39700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc224b39700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc224b39700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 10 monclient(hunting): handle_auth_done global_id 36242 payload 274 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 -- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2200a1050 msgr2=0x7fc2200a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 --2- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2200a1050 0x7fc2200a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 -- 172.21.15.47:0/2505825997 >> v2:172.21.15.47:3300/0 conn(0x7fc2200a0bd0 msgr2=0x7fc2200a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 --2- 172.21.15.47:0/2505825997 >> v2:172.21.15.47:3300/0 conn(0x7fc2200a0bd0 0x7fc2200a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 monclient: found mon.noname-e 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 -- 172.21.15.47:0/2505825997 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200751e0 con 0x7fc2200a07f0 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc225b3b700 1 --2- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2200a1050 0x7fc2200a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc224b39700 1 --2- 172.21.15.47:0/2505825997 >> v2:172.21.15.47:3300/0 conn(0x7fc2200a0bd0 0x7fc2200a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22533a700 1 --2- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 0x7fc2200a0010 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c009c00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5b3956a735ec2717 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 1 -- 172.21.15.47:0/2505825997 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c011aa0 con 0x7fc2200a07f0 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: dump: 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.593 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 1 -- 172.21.15.47:0/2505825997 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c011c00 con 0x7fc2200a07f0 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22759e700 5 monclient: authenticate success, global_id 36242 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22759e700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:28.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 1 -- 172.21.15.47:0/2505825997 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c011d70 con 0x7fc2200a07f0 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc22759e700 4 set_mon_vals no callback set 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.576+0000 7fc217fff700 10 monclient: dump: 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.595 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 10 monclient: shutdown 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 -- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 msgr2=0x7fc2200a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 --2- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 0x7fc2200a0010 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c009c00 comp rx=0 tx=0).stop 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 -- 172.21.15.47:0/2505825997 shutdown_connections 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 --2- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2200a1050 0x7fc2200a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 --2- 172.21.15.47:0/2505825997 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a07f0 0x7fc2200a0010 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 --2- 172.21.15.47:0/2505825997 >> v2:172.21.15.47:3300/0 conn(0x7fc2200a0bd0 0x7fc2200a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 -- 172.21.15.47:0/2505825997 >> 172.21.15.47:0/2505825997 conn(0x7fc2200a3f90 msgr2=0x7fc22006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 -- 172.21.15.47:0/2505825997 shutdown_connections 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.577+0000 7fc22759e700 1 -- 172.21.15.47:0/2505825997 wait complete. 2024-04-20T02:33:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 10 monclient: monmap: 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 1 librados: starting msgr at 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 1 librados: starting objecter 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 1 Processor -- start 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 1 -- start start 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 1 librados: setting wanted keys 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 1 librados: calling monclient init 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.578+0000 7fc22759e700 10 monclient: init 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient: _add_conns ranks=[3,8,6,5,4,0,7,2,1] 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2200a07f0 0x7fc2201c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): picked mon.noname-g con 0x7fc2200a07f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 0x7fc2201bee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): picked mon.noname-e con 0x7fc2200a0bd0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:28.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc2200a1050 0x7fc2201cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): picked mon.noname-d con 0x7fc2200a1050 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201ac810 con 0x7fc2200a07f0 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201abfe0 con 0x7fc2200a1050 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201ab110 con 0x7fc2200a0bd0 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22759e700 10 monclient(hunting): authenticate will time out at 2687.980141s 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 0x7fc2201bee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc2200a1050 0x7fc2201cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 0x7fc2201bee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42868/0 (socket says 172.21.15.47:42868) 2024-04-20T02:33:28.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 -- 172.21.15.47:0/3256050840 learned_addr learned my addr 172.21.15.47:0/3256050840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): get_auth_request con 0x7fc2200a0bd0 auth_method 0 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): get_auth_request con 0x7fc2200a1050 auth_method 0 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2200a07f0 0x7fc2201c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): get_auth_request con 0x7fc2200a07f0 auth_method 0 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient(hunting): handle_auth_done global_id 36251 payload 824 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc2200a1050 msgr2=0x7fc2201cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc2200a1050 0x7fc2201cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2200a07f0 msgr2=0x7fc2201c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2200a07f0 0x7fc2201c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 monclient: found mon.noname-e 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100094a0 con 0x7fc2200a0bd0 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc225b3b700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc2200a1050 0x7fc2201cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc224b39700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 0x7fc2201bee50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc21000ebb0 tx=0x7fc21000eec0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc215ffb700 10 client.?.objecter ms_handle_connect 0x7fc2200a0bd0 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc215ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21000cc60 con 0x7fc2200a0bd0 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc215ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc215ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.579+0000 7fc215ffb700 10 monclient: dump: 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21000cdc0 con 0x7fc2200a0bd0 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 5 monclient: authenticate success, global_id 36251 2024-04-20T02:33:28.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2100138e0 con 0x7fc2200a0bd0 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 10 monclient: _renew_subs 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc218009a70 con 0x7fc2200a0bd0 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc2177fe700 4 set_mon_vals no callback set 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc215ffb700 10 monclient: dump: 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 10 client.36251.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 10 monclient: _renew_subs 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c009550 con 0x7fc2200a0bd0 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.580+0000 7fc22759e700 1 librados: init done 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22759e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc210013470 con 0x7fc2200a0bd0 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22759e700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 0x7fc2201cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220068280 con 0x7fc22006aac0 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc20403a2f0 0x7fc20403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(687..687 src has 266..687) v4 ==== 6674+0+0 (secure 0 0 0) 0x7fc21004aab0 con 0x7fc2200a0bd0 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 0x7fc2201cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 10 client.36251.objecter ms_dispatch 0x7fc220001550 osd_map(687..687 src has 266..687) v4 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 3 client.36251.objecter handle_osd_map got epochs [687,687] > 0 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 3 client.36251.objecter handle_osd_map decoding full epoch 687 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient: get_auth_request con 0x7fc22006aac0 auth_method 0 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 20 client.36251.objecter dump_active .. 0 homeless 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc225b3b700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc20403a2f0 0x7fc20403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc225b3b700 10 monclient: get_auth_request con 0x7fc20403a2f0 auth_method 0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 10 monclient(hunting): handle_auth_done global_id 55846 payload 824 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc22009f350 con 0x7fc22006aac0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 10 client.36251.objecter ms_handle_connect 0x7fc22006aac0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc215ffb700 10 client.36251.objecter resend_mon_ops 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.581+0000 7fc22533a700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 0x7fc2201cc8a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c020070 con 0x7fc22006aac0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc225b3b700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc20403a2f0 0x7fc20403c7b0 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 crypto rx=0x7fc218009fd0 tx=0x7fc21800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 10 monclient: dump: 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 10 client.36251.objecter ms_handle_connect 0x7fc20403a2f0 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc21c004250 con 0x7fc22006aac0 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 msgr2=0x7fc2201cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 0x7fc2201cc8a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c004750 comp rx=0 tx=0).stop 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 msgr2=0x7fc2201cc8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.582+0000 7fc215ffb700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc22006aac0 0x7fc2201cc8a0 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c004750 comp rx=0 tx=0).stop 2024-04-20T02:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22759e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22759e700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 0x7fc2201ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22759e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22006a400 con 0x7fc2201bf390 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 0x7fc2201ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient: get_auth_request con 0x7fc2201bf390 auth_method 0 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.590+0000 7fc22533a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc22533a700 10 monclient(hunting): handle_auth_done global_id 55855 payload 824 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc22533a700 1 -- 172.21.15.47:0/3256050840 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc22006d160 con 0x7fc2201bf390 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc22533a700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 0x7fc2201ccde0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009840 tx=0x7fc21c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 client.36251.objecter ms_handle_connect 0x7fc2201bf390 2024-04-20T02:33:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 client.36251.objecter resend_mon_ops 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c004610 con 0x7fc2201bf390 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 monclient: dump: 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9356 (secure 0 0 0) 0x7fc21c02b830 con 0x7fc2201bf390 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 msgr2=0x7fc2201ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 0x7fc2201ccde0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009840 tx=0x7fc21c011000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 msgr2=0x7fc2201ccde0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:28.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.591+0000 7fc215ffb700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2201bf390 0x7fc2201ccde0 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009840 tx=0x7fc21c011000 comp rx=0 tx=0).stop 2024-04-20T02:33:28.618 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:28.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:28.619 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 5, 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.620 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.621 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:28.622 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:28.623 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.624 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.625 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.626 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:28.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.628 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.630 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.632 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.633 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.634 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:28.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.637 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:28.638 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.639 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:28.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:28.641 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:28.641 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 10 librados: watch_flush enter 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 10 librados: watch_flush exit 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 20 client.36251.objecter shutdown clearing up homeless session... 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 10 client.36251.objecter successfully canceled tick 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc20403a2f0 msgr2=0x7fc20403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc20403a2f0 0x7fc20403c7b0 secure :-1 s=READY pgs=3381 cs=0 l=1 rev1=1 crypto rx=0x7fc218009fd0 tx=0x7fc21800b040 comp rx=0 tx=0).stop 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 10 monclient: shutdown 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 msgr2=0x7fc2201bee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 0x7fc2201bee50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc21000ebb0 tx=0x7fc21000eec0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc225b3b700 1 -- 172.21.15.47:0/3256050840 reap_dead start 2024-04-20T02:33:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 shutdown_connections 2024-04-20T02:33:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 --2- 172.21.15.47:0/3256050840 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fc2200a0bd0 0x7fc2201bee50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 >> 172.21.15.47:0/3256050840 conn(0x7fc2200a3f90 msgr2=0x7fc2200a26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 shutdown_connections 2024-04-20T02:33:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 -- 172.21.15.47:0/3256050840 wait complete. 2024-04-20T02:33:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.592+0000 7fc22759e700 1 librados: shutdown 2024-04-20T02:33:28.644 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:28.644 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:33:28.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:33:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.815+0000 7f125fccf700 10 monclient: get_monmap_and_config 2024-04-20T02:33:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.815+0000 7f125fccf700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.815+0000 7f125fccf700 10 monclient: monmap: 2024-04-20T02:33:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.815 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:28.816308+0000 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:28.816308+0000 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:28.817 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.815+0000 7f125fccf700 1 Processor -- start 2024-04-20T02:33:28.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 -- start start 2024-04-20T02:33:28.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient: init 2024-04-20T02:33:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient: _add_conns ranks=[10,9,8,5,2,4,7,6,0,1,3] 2024-04-20T02:33:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f125809fc30 0x7f1258135da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): picked mon.noname-f con 0x7f125809fc30 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 0x7f12580a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): picked mon.noname-a con 0x7f12580a0010 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 0x7f12580a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): picked mon.noname-e con 0x7f12580a0470 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:28.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125809cd20 con 0x7f12580a0010 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12581362e0 con 0x7f125809fc30 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f125809de80 con 0x7f12580a0470 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125fccf700 10 monclient(hunting): authenticate will time out at 2688.217562s 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 0x7f12580a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 0x7f12580a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f125809fc30 0x7f1258135da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f125809fc30 0x7f1258135da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:59634/0 (socket says 172.21.15.47:59634) 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 0x7f12580a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42882/0 (socket says 172.21.15.47:42882) 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 0x7f12580a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45170/0 (socket says 172.21.15.47:45170) 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 1 -- 172.21.15.47:0/194690427 learned_addr learned my addr 172.21.15.47:0/194690427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): get_auth_request con 0x7f125809fc30 auth_method 0 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): get_auth_request con 0x7f12580a0010 auth_method 0 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): get_auth_request con 0x7f12580a0470 auth_method 0 2024-04-20T02:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125d26a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125e26c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.816+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 10 monclient(hunting): handle_auth_done global_id 36260 payload 274 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 -- 172.21.15.47:0/194690427 >> v2:172.21.15.179:3300/0 conn(0x7f125809fc30 msgr2=0x7f1258135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 --2- 172.21.15.47:0/194690427 >> v2:172.21.15.179:3300/0 conn(0x7f125809fc30 0x7f1258135da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 -- 172.21.15.47:0/194690427 >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 msgr2=0x7f12580a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 --2- 172.21.15.47:0/194690427 >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 0x7f12580a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 monclient: found mon.noname-e 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 -- 172.21.15.47:0/194690427 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580751e0 con 0x7f12580a0470 2024-04-20T02:33:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125d26a700 1 --2- 172.21.15.47:0/194690427 >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 0x7f12580a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125e26c700 1 --2- 172.21.15.47:0/194690427 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 0x7f12580a2450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1258074cc0 tx=0x7f125000b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ac2a12ca4bc98a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 1 -- 172.21.15.47:0/194690427 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f125000e070 con 0x7f12580a0470 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: dump: 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.838 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 1 -- 172.21.15.47:0/194690427 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12500098f0 con 0x7f12580a0470 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125fccf700 5 monclient: authenticate success, global_id 36260 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125fccf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125fccf700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 1 -- 172.21.15.47:0/194690427 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12500128a0 con 0x7f12580a0470 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125fccf700 4 set_mon_vals no callback set 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.817+0000 7f125ca69700 10 monclient: dump: 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.841 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 10 monclient: shutdown 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 -- 172.21.15.47:0/194690427 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 msgr2=0x7f12580a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 --2- 172.21.15.47:0/194690427 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 0x7f12580a2450 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1258074cc0 tx=0x7f125000b330 comp rx=0 tx=0).stop 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 -- 172.21.15.47:0/194690427 shutdown_connections 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 --2- 172.21.15.47:0/194690427 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f12580a0470 0x7f12580a2450 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 --2- 172.21.15.47:0/194690427 >> v2:172.21.15.179:3300/0 conn(0x7f125809fc30 0x7f1258135da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 --2- 172.21.15.47:0/194690427 >> v2:172.21.15.47:3300/0 conn(0x7f12580a0010 0x7f12580a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 -- 172.21.15.47:0/194690427 >> 172.21.15.47:0/194690427 conn(0x7f12580a3000 msgr2=0x7f12580a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 -- 172.21.15.47:0/194690427 shutdown_connections 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 1 -- 172.21.15.47:0/194690427 wait complete. 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 10 monclient: build_initial_monmap 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.818+0000 7f125fccf700 10 monclient: monmap: 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 librados: starting msgr at 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 librados: starting objecter 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 Processor -- start 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 -- start start 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 librados: setting wanted keys 2024-04-20T02:33:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 librados: calling monclient init 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient: init 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient: _add_conns ranks=[3,2,1,6,8,0,5,4,7] 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125809fc30 0x7f12581caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): picked mon.noname-g con 0x7f125809fc30 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12580a0010 0x7f12581cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): picked mon.noname-b con 0x7f12580a0010 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12580a0470 0x7f125812bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): picked mon.noname-f con 0x7f12580a0470 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1258134b30 con 0x7f12580a0470 2024-04-20T02:33:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125812dfa0 con 0x7f12580a0010 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12581356b0 con 0x7f125809fc30 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125fccf700 10 monclient(hunting): authenticate will time out at 2688.220806s 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12580a0010 0x7f12581cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12580a0010 0x7f12581cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43384/0 (socket says 172.21.15.47:43384) 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 1 -- 172.21.15.47:0/2192958119 learned_addr learned my addr 172.21.15.47:0/2192958119 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 10 monclient(hunting): get_auth_request con 0x7f12580a0010 auth_method 0 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.819+0000 7f125d26a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125e26c700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12580a0470 0x7f125812bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125809fc30 0x7f12581caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125e26c700 10 monclient(hunting): get_auth_request con 0x7f12580a0470 auth_method 0 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125e26c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125e26c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125e26c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 10 monclient(hunting): handle_auth_done global_id 104258 payload 824 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125809fc30 msgr2=0x7f12581caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125809fc30 0x7f12581caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12580a0470 msgr2=0x7f125812bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12580a0470 0x7f125812bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 10 monclient: _finish_hunting 0 2024-04-20T02:33:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 monclient: found mon.noname-b 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480094a0 con 0x7f12580a0010 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125da6b700 10 monclient: get_auth_request con 0x7f125809fc30 auth_method 0 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125809fc30 0x7f12581caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125d26a700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12580a0010 0x7f12581cb460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f124800ebe0 tx=0x7f1248009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 client.?.objecter ms_handle_connect 0x7f12580a0010 2024-04-20T02:33:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f124800cd80 con 0x7f12580a0010 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: dump: 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 5 monclient: authenticate success, global_id 104258 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1248013470 con 0x7f12580a0010 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 10 monclient: _renew_subs 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1250009a70 con 0x7f12580a0010 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f124800cee0 con 0x7f12580a0010 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124ffff700 4 set_mon_vals no callback set 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 10 client.104258.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f124e7fc700 10 monclient: dump: 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 10 monclient: _renew_subs 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125809caa0 con 0x7f12580a0010 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.820+0000 7f125fccf700 1 librados: init done 2024-04-20T02:33:28.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125fccf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125fccf700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1258068230 con 0x7f125806aac0 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f124800c910 con 0x7f12580a0010 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 10 monclient: get_auth_request con 0x7f125806aac0 auth_method 0 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 msgr2=0x7f125812fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 msgr2=0x7f125812fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.821+0000 7f124e7fc700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f123c03a2f0 0x7f123c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(687..687 src has 266..687) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f124804abc0 con 0x7f12580a0010 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f124e7fc700 10 client.104258.objecter ms_dispatch 0x7f1258001550 osd_map(687..687 src has 266..687) v4 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f124e7fc700 3 client.104258.objecter handle_osd_map got epochs [687,687] > 0 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f124e7fc700 3 client.104258.objecter handle_osd_map decoding full epoch 687 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f124e7fc700 20 client.104258.objecter dump_active .. 0 homeless 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f125e26c700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f123c03a2f0 0x7f123c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f125e26c700 10 monclient: get_auth_request con 0x7f123c03a2f0 auth_method 0 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f125e26c700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f123c03a2f0 0x7f123c03c7b0 secure :-1 s=READY pgs=3382 cs=0 l=1 rev1=1 crypto rx=0x7f12580683f0 tx=0x7f1250011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:28.822+0000 7f124e7fc700 10 client.104258.objecter ms_handle_connect 0x7f123c03a2f0 2024-04-20T02:33:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient: get_auth_request con 0x7f125806aac0 auth_method 0 2024-04-20T02:33:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 10 monclient(hunting): handle_auth_done global_id 104223 payload 824 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.022+0000 7f125da6b700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12580a45e0 con 0x7f125806aac0 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1254012930 tx=0x7f1254012cf0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 client.104258.objecter ms_handle_connect 0x7f125806aac0 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 client.104258.objecter resend_mon_ops 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1254010ce0 con 0x7f125806aac0 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 monclient: dump: 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1254010e40 con 0x7f125806aac0 2024-04-20T02:33:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 msgr2=0x7f125812fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1254012930 tx=0x7f1254012cf0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 msgr2=0x7f125812fae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.023+0000 7f124e7fc700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125806aac0 0x7f125812fae0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1254012930 tx=0x7f1254012cf0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.031+0000 7f125fccf700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.031+0000 7f125fccf700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 0x7f125812c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.031+0000 7f125fccf700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.031+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125806a400 con 0x7f125812c420 2024-04-20T02:33:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 0x7f125812c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient: get_auth_request con 0x7f125812c420 auth_method 0 2024-04-20T02:33:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 10 monclient(hunting): handle_auth_done global_id 104232 payload 824 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 1 -- 172.21.15.47:0/2192958119 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f125806bda0 con 0x7f125812c420 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f125da6b700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 0x7f125812c830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1254012930 tx=0x7f12540047c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f124e7fc700 10 client.104258.objecter ms_handle_connect 0x7f125812c420 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.032+0000 7f124e7fc700 10 client.104258.objecter resend_mon_ops 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1254024070 con 0x7f125812c420 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 10 monclient: dump: 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.035 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f1254006220 con 0x7f125812c420 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 msgr2=0x7f125812c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 0x7f125812c830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1254012930 tx=0x7f12540047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 msgr2=0x7f125812c830 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f124e7fc700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f125812c420 0x7f125812c830 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1254012930 tx=0x7f12540047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:29.036 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:33:29.037 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.038 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.046 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.048 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.049 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:29.050 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.051 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.052 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:29.053 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.054 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.055 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.056 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:29.057 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 10 librados: watch_flush enter 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 10 librados: watch_flush exit 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 20 client.104258.objecter shutdown clearing up homeless session... 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 10 client.104258.objecter successfully canceled tick 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f123c03a2f0 msgr2=0x7f123c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f123c03a2f0 0x7f123c03c7b0 secure :-1 s=READY pgs=3382 cs=0 l=1 rev1=1 crypto rx=0x7f12580683f0 tx=0x7f1250011000 comp rx=0 tx=0).stop 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 10 monclient: shutdown 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12580a0010 msgr2=0x7f12581cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 1 --2- 172.21.15.47:0/2192958119 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f12580a0010 0x7f12581cb460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f124800ebe0 tx=0x7f1248009ec0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125e26c700 1 -- 172.21.15.47:0/2192958119 reap_dead start 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125e26c700 1 -- 172.21.15.47:0/2192958119 reap_dead start 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 shutdown_connections 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.033+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 >> 172.21.15.47:0/2192958119 conn(0x7f12580a3000 msgr2=0x7f12581bd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.034+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 shutdown_connections 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.034+0000 7f125fccf700 1 -- 172.21.15.47:0/2192958119 wait complete. 2024-04-20T02:33:29.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.034+0000 7f125fccf700 1 librados: shutdown 2024-04-20T02:33:29.060 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:29.060 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:33:29.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:33:29.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.204+0000 7f8669980700 10 monclient: get_monmap_and_config 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.204+0000 7f8669980700 10 monclient: build_initial_monmap 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.204+0000 7f8669980700 10 monclient: monmap: 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:29.206009+0000 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:29.206009+0000 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:29.205 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:29.206 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.205+0000 7f8669980700 1 Processor -- start 2024-04-20T02:33:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.205+0000 7f8669980700 1 -- start start 2024-04-20T02:33:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.205+0000 7f8669980700 10 monclient: init 2024-04-20T02:33:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient: _add_conns ranks=[7,2,4,10,9,5,1,8,6,3,0] 2024-04-20T02:33:29.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8664137d90 0x7f86640a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): picked mon.noname-i con 0x7f8664137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 0x7f86640a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): picked mon.noname-b con 0x7f8664138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 0x7f86640a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): picked mon.noname-c con 0x7f86641385b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866409fa40 con 0x7f8664138170 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f866409dfc0 con 0x7f86641385b0 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f86640a7bc0 con 0x7f8664137d90 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8669980700 10 monclient(hunting): authenticate will time out at 2688.607127s 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 0x7f86640a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 0x7f86640a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 0x7f86640a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58112/0 (socket says 172.21.15.47:58112) 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 0x7f86640a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43390/0 (socket says 172.21.15.47:43390) 2024-04-20T02:33:29.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 1 -- 172.21.15.47:0/4247366979 learned_addr learned my addr 172.21.15.47:0/4247366979 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): get_auth_request con 0x7f86641385b0 auth_method 0 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): get_auth_request con 0x7f8664138170 auth_method 0 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8664137d90 0x7f86640a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): get_auth_request con 0x7f8664137d90 auth_method 0 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient(hunting): handle_auth_done global_id 104267 payload 274 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 -- 172.21.15.47:0/4247366979 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8664137d90 msgr2=0x7f86640a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8664137d90 0x7f86640a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 -- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 msgr2=0x7f86640a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:29.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 0x7f86640a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient: _finish_hunting 0 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 monclient: found mon.noname-b 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 -- 172.21.15.47:0/4247366979 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640751e0 con 0x7f8664138170 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86637fe700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 0x7f86640a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.206+0000 7f86627fc700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 0x7f86640a5c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f864c009870 tx=0x7f864c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f215145ac39c8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 1 -- 172.21.15.47:0/4247366979 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f864c011aa0 con 0x7f8664138170 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: dump: 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.224 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 1 -- 172.21.15.47:0/4247366979 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c011c00 con 0x7f8664138170 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 5 monclient: authenticate success, global_id 104267 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 1 -- 172.21.15.47:0/4247366979 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f864c011d70 con 0x7f8664138170 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 4 set_mon_vals no callback set 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8661ffb700 10 monclient: dump: 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.227 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 10 monclient: shutdown 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 -- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 msgr2=0x7f86640a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 0x7f86640a5c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f864c009870 tx=0x7f864c009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 -- 172.21.15.47:0/4247366979 shutdown_connections 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86641385b0 0x7f86640a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f8664137d90 0x7f86640a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 --2- 172.21.15.47:0/4247366979 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8664138170 0x7f86640a5c90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 -- 172.21.15.47:0/4247366979 >> 172.21.15.47:0/4247366979 conn(0x7f86640aa8b0 msgr2=0x7f86640a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 -- 172.21.15.47:0/4247366979 shutdown_connections 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.207+0000 7f8669980700 1 -- 172.21.15.47:0/4247366979 wait complete. 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 10 monclient: build_initial_monmap 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 10 monclient: monmap: 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:29.229 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:29.230 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 1 librados: starting msgr at 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 1 librados: starting objecter 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 1 Processor -- start 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 1 -- start start 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 1 librados: setting wanted keys 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 1 librados: calling monclient init 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.208+0000 7f8669980700 10 monclient: init 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient: _add_conns ranks=[5,1,6,2,8,7,4,0,3] 2024-04-20T02:33:29.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8664137d90 0x7f86641c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): picked mon.noname-h con 0x7f8664137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8664138170 0x7f8664136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): picked mon.noname-f con 0x7f8664138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f86641385b0 0x7f8664137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): picked mon.noname-d con 0x7f86641385b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641243f0 con 0x7f8664138170 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f866409ea50 con 0x7f8664137d90 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f86641c97c0 con 0x7f86641385b0 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8669980700 10 monclient(hunting): authenticate will time out at 2688.610586s 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f86641385b0 0x7f8664137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f86641385b0 0x7f8664137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50918/0 (socket says 172.21.15.47:50918) 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 1 -- 172.21.15.47:0/818242734 learned_addr learned my addr 172.21.15.47:0/818242734 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): get_auth_request con 0x7f86641385b0 auth_method 0 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8664138170 0x7f8664136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86637fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8662ffd700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8664137d90 0x7f86641c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): get_auth_request con 0x7f8664138170 auth_method 0 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8662ffd700 10 monclient(hunting): get_auth_request con 0x7f8664137d90 auth_method 0 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8662ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8662ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f8662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.209+0000 7f86627fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 10 monclient(hunting): handle_auth_done global_id 55446 payload 824 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8664137d90 msgr2=0x7f86641c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8664137d90 0x7f86641c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8664138170 msgr2=0x7f8664136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8664138170 0x7f8664136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 10 monclient: _finish_hunting 0 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 monclient: found mon.noname-d 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c009550 con 0x7f86641385b0 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8662ffd700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f8664137d90 0x7f86641c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86637fe700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f86641385b0 0x7f8664137530 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f865c006e90 tx=0x7f865c00ec00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 client.?.objecter ms_handle_connect 0x7f86641385b0 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f865c0044a0 con 0x7f86641385b0 2024-04-20T02:33:29.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: dump: 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.239 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: _finish_auth 0 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:29.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 5 monclient: authenticate success, global_id 55446 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c0036a0 con 0x7f86641385b0 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 10 monclient: _renew_subs 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c0094a0 con 0x7f86641385b0 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f865c003c80 con 0x7f86641385b0 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f86617fa700 4 set_mon_vals no callback set 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 10 client.55446.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f865bfff700 10 monclient: dump: 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.241 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.242 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 10 monclient: _renew_subs 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8664124110 con 0x7f86641385b0 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.210+0000 7f8669980700 1 librados: init done 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8669980700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8669980700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 0x7f86641cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8664068180 con 0x7f866406aac0 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f865c003810 con 0x7f86641385b0 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 0x7f86641cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient: get_auth_request con 0x7f866406aac0 auth_method 0 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f864403a320 0x7f864403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(687..687 src has 266..687) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f865c04a8a0 con 0x7f86641385b0 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 10 client.55446.objecter ms_dispatch 0x7f8664001550 osd_map(687..687 src has 266..687) v4 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 3 client.55446.objecter handle_osd_map got epochs [687,687] > 0 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 3 client.55446.objecter handle_osd_map decoding full epoch 687 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 10 monclient(hunting): handle_auth_done global_id 84596 payload 824 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f865bfff700 20 client.55446.objecter dump_active .. 0 homeless 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f86627fc700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f864403a320 0x7f864403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f8662ffd700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f866409f760 con 0x7f866406aac0 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.211+0000 7f86627fc700 10 monclient: get_auth_request con 0x7f864403a320 auth_method 0 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 client.55446.objecter ms_handle_connect 0x7f866406aac0 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 client.55446.objecter resend_mon_ops 2024-04-20T02:33:29.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f8662ffd700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 0x7f86641cc250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8654007ec0 tx=0x7f8654011430 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 client.55446.objecter ms_handle_connect 0x7f864403a320 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f86627fc700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f864403a320 0x7f864403c7e0 secure :-1 s=READY pgs=3383 cs=0 l=1 rev1=1 crypto rx=0x7f864c0090b0 tx=0x7f864c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8654011c90 con 0x7f866406aac0 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 monclient: dump: 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.245 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8654011df0 con 0x7f866406aac0 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:29.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 msgr2=0x7f86641cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 0x7f86641cc250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8654007ec0 tx=0x7f8654011430 comp rx=0 tx=0).stop 2024-04-20T02:33:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 msgr2=0x7f86641cc250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.212+0000 7f865bfff700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f866406aac0 0x7f86641cc250 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8654007ec0 tx=0x7f8654011430 comp rx=0 tx=0).stop 2024-04-20T02:33:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8669980700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8669980700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 0x7f86641ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8669980700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86640a78f0 con 0x7f86641cc790 2024-04-20T02:33:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 0x7f86641ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient: get_auth_request con 0x7f86641cc790 auth_method 0 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.220+0000 7f8662ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f8662ffd700 10 monclient(hunting): handle_auth_done global_id 84605 payload 824 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f8662ffd700 1 -- 172.21.15.47:0/818242734 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f86641cd0e0 con 0x7f86641cc790 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f8662ffd700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 0x7f86641ccba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8654007ec0 tx=0x7f8654004ad0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 client.55446.objecter ms_handle_connect 0x7f86641cc790 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 client.55446.objecter resend_mon_ops 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8654016a10 con 0x7f86641cc790 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 monclient: dump: 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9524 (secure 0 0 0) 0x7f8654016b70 con 0x7f86641cc790 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 msgr2=0x7f86641ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.221+0000 7f865bfff700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 0x7f86641ccba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8654007ec0 tx=0x7f8654004ad0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f865bfff700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 msgr2=0x7f86641ccba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f865bfff700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f86641cc790 0x7f86641ccba0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8654007ec0 tx=0x7f8654004ad0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:29.253 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:33:29.254 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.255 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.256 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:29.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.258 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.259 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:29.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.261 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.263 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.266 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.267 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.269 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:29.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.274 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:29.275 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 10 librados: watch_flush enter 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 10 librados: watch_flush exit 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 20 client.55446.objecter shutdown clearing up homeless session... 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 10 client.55446.objecter successfully canceled tick 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f864403a320 msgr2=0x7f864403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f864403a320 0x7f864403c7e0 secure :-1 s=READY pgs=3383 cs=0 l=1 rev1=1 crypto rx=0x7f864c0090b0 tx=0x7f864c008030 comp rx=0 tx=0).stop 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 10 monclient: shutdown 2024-04-20T02:33:29.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f86641385b0 msgr2=0x7f8664137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 --2- 172.21.15.47:0/818242734 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f86641385b0 0x7f8664137530 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f865c006e90 tx=0x7f865c00ec00 comp rx=0 tx=0).stop 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f86637fe700 1 -- 172.21.15.47:0/818242734 reap_dead start 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f86637fe700 1 -- 172.21.15.47:0/818242734 reap_dead start 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 shutdown_connections 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 >> 172.21.15.47:0/818242734 conn(0x7f86640aa8b0 msgr2=0x7f86641bf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 shutdown_connections 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 -- 172.21.15.47:0/818242734 wait complete. 2024-04-20T02:33:29.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.222+0000 7f8669980700 1 librados: shutdown 2024-04-20T02:33:29.278 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:29.279 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:33:29.279 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:33:29.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.417+0000 7f35d7b49700 10 monclient: get_monmap_and_config 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.417+0000 7f35d7b49700 10 monclient: build_initial_monmap 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.417+0000 7f35d7b49700 10 monclient: monmap: 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:29.418958+0000 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:29.418958+0000 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:29.418 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:29.419 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.418+0000 7f35d7b49700 1 Processor -- start 2024-04-20T02:33:29.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.418+0000 7f35d7b49700 1 -- start start 2024-04-20T02:33:29.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.418+0000 7f35d7b49700 10 monclient: init 2024-04-20T02:33:29.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient: _add_conns ranks=[1,7,6,3,5,9,4,0,8,10,2] 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f35d00a2950 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d00a3440 0x7f35d013cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): picked mon.noname-i con 0x7f35d00a3440 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 0x7f35d013f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): picked mon.noname-d con 0x7f35d013d490 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00751e0 con 0x7f35d00a2950 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0072830 con 0x7f35d013d490 2024-04-20T02:33:29.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35d009f150 con 0x7f35d00a3440 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d7b49700 10 monclient(hunting): authenticate will time out at 2688.820265s 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 0x7f35d013f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 0x7f35d013f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50928/0 (socket says 172.21.15.47:50928) 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 -- 172.21.15.47:0/1247807930 learned_addr learned my addr 172.21.15.47:0/1247807930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): get_auth_request con 0x7f35d013d490 auth_method 0 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 1 --2- 172.21.15.47:0/1247807930 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d00a3440 0x7f35d013cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): get_auth_request con 0x7f35d00a3440 auth_method 0 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient(hunting): handle_auth_done global_id 55455 payload 274 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 -- 172.21.15.47:0/1247807930 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d00a3440 msgr2=0x7f35d013cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 --2- 172.21.15.47:0/1247807930 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d00a3440 0x7f35d013cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 -- 172.21.15.47:0/1247807930 >> v1:172.21.15.179:6789/0 conn(0x7f35d00a2950 legacy=0x7f35d00a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient: _finish_hunting 0 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 monclient: found mon.noname-d 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 -- 172.21.15.47:0/1247807930 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d009f710 con 0x7f35d013d490 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.419+0000 7f35d60e6700 1 --2- 172.21.15.47:0/1247807930 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 0x7f35d013f880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009e80 tx=0x7f35c800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=af5accab26fd6b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 1 -- 172.21.15.47:0/1247807930 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35c800e070 con 0x7f35d013d490 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: dump: 2024-04-20T02:33:29.434 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.435 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.439 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: _finish_auth 0 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: _check_auth_tickets 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 1 -- 172.21.15.47:0/1247807930 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800bd80 con 0x7f35d013d490 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 5 monclient: authenticate success, global_id 55455 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:29.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 1 -- 172.21.15.47:0/1247807930 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35c8012860 con 0x7f35d013d490 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 4 set_mon_vals no callback set 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d48e3700 10 monclient: dump: 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.441 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 10 monclient: shutdown 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 -- 172.21.15.47:0/1247807930 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 msgr2=0x7f35d013f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 --2- 172.21.15.47:0/1247807930 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 0x7f35d013f880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009e80 tx=0x7f35c800b330 comp rx=0 tx=0).stop 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 -- 172.21.15.47:0/1247807930 shutdown_connections 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 --2- 172.21.15.47:0/1247807930 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f35d013d490 0x7f35d013f880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 --2- 172.21.15.47:0/1247807930 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d00a3440 0x7f35d013cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 -- 172.21.15.47:0/1247807930 >> 172.21.15.47:0/1247807930 conn(0x7f35d00a7570 msgr2=0x7f35d00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 -- 172.21.15.47:0/1247807930 shutdown_connections 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.420+0000 7f35d7b49700 1 -- 172.21.15.47:0/1247807930 wait complete. 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.421+0000 7f35d7b49700 10 monclient: build_initial_monmap 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.421+0000 7f35d7b49700 10 monclient: monmap: 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:29.443 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 librados: starting msgr at 2024-04-20T02:33:29.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 librados: starting objecter 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 Processor -- start 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 -- start start 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 librados: setting wanted keys 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 librados: calling monclient init 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 10 monclient: init 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 10 monclient: _add_conns ranks=[2,4,8,6,7,3,1,5,0] 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35d00a2950 0x7f35d01219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 10 monclient(hunting): picked mon.noname-b con 0x7f35d00a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.422+0000 7f35d7b49700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f35d00a3440 0x7f35d0121ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): picked mon.noname-c con 0x7f35d00a3440 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f35d013d490 0x7f35d0122420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): picked mon.noname-e con 0x7f35d013d490 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01246f0 con 0x7f35d00a2950 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0124cb0 con 0x7f35d00a3440 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f35d009dd70 con 0x7f35d013d490 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 10 monclient(hunting): authenticate will time out at 2688.824046s 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35d00a2950 0x7f35d01219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f35d00a3440 0x7f35d0121ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35d00a2950 0x7f35d01219a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:43398/0 (socket says 172.21.15.47:43398) 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f35d013d490 0x7f35d0122420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f35d00a3440 0x7f35d0121ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58122/0 (socket says 172.21.15.47:58122) 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 -- 172.21.15.47:0/2544165909 learned_addr learned my addr 172.21.15.47:0/2544165909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): get_auth_request con 0x7f35d00a2950 auth_method 0 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): get_auth_request con 0x7f35d00a3440 auth_method 0 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): get_auth_request con 0x7f35d013d490 auth_method 0 2024-04-20T02:33:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d60e6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient(hunting): handle_auth_done global_id 104276 payload 824 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f35d013d490 msgr2=0x7f35d0122420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f35d013d490 0x7f35d0122420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f35d00a3440 msgr2=0x7f35d0121ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f35d00a3440 0x7f35d0121ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient: _finish_hunting 0 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 monclient: found mon.noname-b 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009b30 con 0x7f35d00a2950 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d50e4700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f35d00a3440 0x7f35d0121ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d58e5700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35d00a2950 0x7f35d01219a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c430 tx=0x7f35cc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 client.?.objecter ms_handle_connect 0x7f35d00a2950 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35cc005540 con 0x7f35d00a2950 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 monclient: dump: 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.451 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:29.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35d7b49700 5 monclient: authenticate success, global_id 104276 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.423+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc00ce00 con 0x7f35d00a2950 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c67fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 monclient: _renew_subs 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35cc003af0 con 0x7f35d00a2950 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d009d960 con 0x7f35d00a2950 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c7fff700 4 set_mon_vals no callback set 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c67fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c67fc700 10 monclient: dump: 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.453 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 client.104276.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 monclient: _renew_subs 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:33:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d0124f90 con 0x7f35d00a2950 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 1 librados: init done 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f35cc003c50 con 0x7f35d00a2950 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 0x7f35d01cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.424+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0068310 con 0x7f35d006aac0 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f35b403a2f0 0x7f35b403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(687..687 src has 266..687) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f35cc049a50 con 0x7f35d00a2950 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 client.104276.objecter ms_dispatch 0x7f35d0001550 osd_map(687..687 src has 266..687) v4 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 3 client.104276.objecter handle_osd_map got epochs [687,687] > 0 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 3 client.104276.objecter handle_osd_map decoding full epoch 687 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 0x7f35d01cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient: get_auth_request con 0x7f35d006aac0 auth_method 0 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d60e6700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f35b403a2f0 0x7f35b403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 20 client.104276.objecter dump_active .. 0 homeless 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d60e6700 10 monclient: get_auth_request con 0x7f35b403a2f0 auth_method 0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 10 monclient(hunting): handle_auth_done global_id 84688 payload 824 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35d009ee70 con 0x7f35d006aac0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 client.104276.objecter ms_handle_connect 0x7f35d006aac0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d50e4700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 0x7f35d01cc620 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35c0011d80 tx=0x7f35c000f520 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 client.104276.objecter resend_mon_ops 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 client.104276.objecter ms_handle_connect 0x7f35b403a2f0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35d60e6700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f35b403a2f0 0x7f35b403c7b0 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009e80 tx=0x7f35c8009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35c00044a0 con 0x7f35d006aac0 2024-04-20T02:33:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.425+0000 7f35c67fc700 10 monclient: dump: 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f35c000fcc0 con 0x7f35d006aac0 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 msgr2=0x7f35d01cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 0x7f35d01cc620 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35c0011d80 tx=0x7f35c000f520 comp rx=0 tx=0).stop 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 msgr2=0x7f35d01cc620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.426+0000 7f35c67fc700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d006aac0 0x7f35d01cc620 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.433+0000 7f35d7b49700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.433+0000 7f35d7b49700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 0x7f35d01ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.433+0000 7f35d7b49700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.433+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35d006a400 con 0x7f35d01ccb60 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 0x7f35d01ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient: get_auth_request con 0x7f35d01ccb60 auth_method 0 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 10 monclient(hunting): handle_auth_done global_id 84697 payload 824 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.434+0000 7f35d50e4700 1 -- 172.21.15.47:0/2544165909 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f35d0067730 con 0x7f35d01ccb60 2024-04-20T02:33:29.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d50e4700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 0x7f35d01ccf70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35c001b000 tx=0x7f35c0004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 client.104276.objecter ms_handle_connect 0x7f35d01ccb60 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 client.104276.objecter resend_mon_ops 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35c0014b20 con 0x7f35d01ccb60 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 monclient: dump: 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.462 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f35c00146b0 con 0x7f35d01ccb60 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 msgr2=0x7f35d01ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 0x7f35d01ccf70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35c001b000 tx=0x7f35c0004750 comp rx=0 tx=0).stop 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 msgr2=0x7f35d01ccf70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35c67fc700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f35d01ccb60 0x7f35d01ccf70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 108, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:29.464 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 6, 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.465 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.466 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:29.467 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:29.468 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:29.469 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.470 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.471 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:29.474 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.477 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:29.479 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.481 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:29.481 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:29.481 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.481 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.481 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.482 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.483 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.484 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:29.485 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.485 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.485 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.485 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.486 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:29.487 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:29.488 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.489 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.490 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:29.491 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:29.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 10 librados: watch_flush enter 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 10 librados: watch_flush exit 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 20 client.104276.objecter shutdown clearing up homeless session... 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 10 client.104276.objecter successfully canceled tick 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f35b403a2f0 msgr2=0x7f35b403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f35b403a2f0 0x7f35b403c7b0 secure :-1 s=READY pgs=3384 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009e80 tx=0x7f35c8009000 comp rx=0 tx=0).stop 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 10 monclient: shutdown 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35d00a2950 msgr2=0x7f35d01219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 --2- 172.21.15.47:0/2544165909 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f35d00a2950 0x7f35d01219a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c430 tx=0x7f35cc00c740 comp rx=0 tx=0).stop 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d60e6700 1 -- 172.21.15.47:0/2544165909 reap_dead start 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d60e6700 1 -- 172.21.15.47:0/2544165909 reap_dead start 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 shutdown_connections 2024-04-20T02:33:29.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 >> 172.21.15.47:0/2544165909 conn(0x7f35d00a7570 msgr2=0x7f35d009d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.435+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 shutdown_connections 2024-04-20T02:33:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.436+0000 7f35d7b49700 1 -- 172.21.15.47:0/2544165909 wait complete. 2024-04-20T02:33:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.436+0000 7f35d7b49700 1 librados: shutdown 2024-04-20T02:33:29.495 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:33:29.496 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'i'] 2024-04-20T02:33:29.496 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['h'] 2024-04-20T02:33:29.496 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-20T02:33:29.496 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-20T02:33:29.496 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T02:33:29.496 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:33:29.509 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T02:33:29.510 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-04-20T02:33:29.510 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i store 2024-04-20T02:33:29.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i sync_force --yes-i-really-mean-it 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.693+0000 7f755f7cc700 10 monclient: get_monmap_and_config 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.693+0000 7f755f7cc700 10 monclient: build_initial_monmap 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.693+0000 7f755f7cc700 10 monclient: monmap: 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:29.694986+0000 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:29.694986+0000 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:29.694 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.694+0000 7f755f7cc700 1 Processor -- start 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.694+0000 7f755f7cc700 1 -- start start 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.694+0000 7f755f7cc700 10 monclient: init 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient: _add_conns ranks=[6,4,8,1,10,2,0,3,7,9,5] 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 0x7f75580a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): picked mon.noname-d con 0x7f75580a2870 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558137a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): picked mon.noname-c con 0x7f75580a3190 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): picked mon.noname-e con 0x7f7558137f60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f755809f740 con 0x7f75580a3190 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7558067390 con 0x7f75580a2870 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f755809ee30 con 0x7f7558137f60 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755f7cc700 10 monclient(hunting): authenticate will time out at 2689.096323s 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 0x7f75580a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558137a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58130/0 (socket says 172.21.15.47:58130) 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 0x7f75580a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:50930/0 (socket says 172.21.15.47:50930) 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581383e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:42904/0 (socket says 172.21.15.47:42904) 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 1 -- 172.21.15.47:0/1308611525 learned_addr learned my addr 172.21.15.47:0/1308611525 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): get_auth_request con 0x7f75580a3190 auth_method 0 2024-04-20T02:33:29.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): get_auth_request con 0x7f7558137f60 auth_method 0 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): get_auth_request con 0x7f75580a2870 auth_method 0 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient(hunting): handle_auth_done global_id 36278 payload 274 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 -- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 msgr2=0x7f75580a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 0x7f75580a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 -- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 msgr2=0x7f7558137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558137a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient: _finish_hunting 0 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 monclient: found mon.noname-e 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755dd69700 1 -- 172.21.15.47:0/1308611525 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580751e0 con 0x7f7558137f60 2024-04-20T02:33:29.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755d568700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 0x7f75580a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.695+0000 7f755cd67700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755dd69700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581383e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7550009df0 tx=0x7f755000b330 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e08524f6086a498c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 1 -- 172.21.15.47:0/1308611525 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f755000e070 con 0x7f7558137f60 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: dump: 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.714 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.715 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: _finish_auth 0 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 5 monclient: authenticate success, global_id 36278 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 1 -- 172.21.15.47:0/1308611525 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550012430 con 0x7f7558137f60 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 1 -- 172.21.15.47:0/1308611525 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7550012a10 con 0x7f7558137f60 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f754bfff700 10 monclient: dump: 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 4 set_mon_vals no callback set 2024-04-20T02:33:29.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 10 monclient: shutdown 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 -- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 msgr2=0x7f75581383e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581383e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7550009df0 tx=0x7f755000b330 comp rx=0 tx=0).stop 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 -- 172.21.15.47:0/1308611525 shutdown_connections 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581383e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f75580a2870 0x7f75580a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 --2- 172.21.15.47:0/1308611525 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.696+0000 7f755f7cc700 1 -- 172.21.15.47:0/1308611525 >> 172.21.15.47:0/1308611525 conn(0x7f75580a3f90 msgr2=0x7f755806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.697+0000 7f755f7cc700 1 -- 172.21.15.47:0/1308611525 shutdown_connections 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.697+0000 7f755f7cc700 1 -- 172.21.15.47:0/1308611525 wait complete. 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.697+0000 7f755f7cc700 10 monclient: build_initial_monmap 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.697+0000 7f755f7cc700 10 monclient: monmap: 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:29.718 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:29.719 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 1 librados: starting msgr at 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 1 librados: starting objecter 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 1 Processor -- start 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 1 -- start start 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 1 librados: setting wanted keys 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 1 librados: calling monclient init 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.698+0000 7f755f7cc700 10 monclient: init 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient: _add_conns ranks=[5,4,8,0,1,3,2,6,7] 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f75580a2870 0x7f75581cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): picked mon.noname-h con 0x7f75580a2870 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): picked mon.noname-c con 0x7f75580a3190 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): picked mon.noname-e con 0x7f7558137f60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7558124ba0 con 0x7f75580a3190 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75581cd280 con 0x7f75580a2870 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7558071fe0 con 0x7f7558137f60 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755f7cc700 10 monclient(hunting): authenticate will time out at 2689.100341s 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:58142/0 (socket says 172.21.15.47:58142) 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 -- 172.21.15.47:0/3048794474 learned_addr learned my addr 172.21.15.47:0/3048794474 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): get_auth_request con 0x7f75580a3190 auth_method 0 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): get_auth_request con 0x7f7558137f60 auth_method 0 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f75580a2870 0x7f75581cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): get_auth_request con 0x7f75580a2870 auth_method 0 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient(hunting): handle_auth_done global_id 84658 payload 824 2024-04-20T02:33:29.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 msgr2=0x7f75581341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f75580a2870 msgr2=0x7f75581cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f75580a2870 0x7f75581cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient: _finish_hunting 0 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 monclient: found mon.noname-c 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755cd67700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75540094a0 con 0x7f75580a3190 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.699+0000 7f755dd69700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f7558137f60 0x7f75581341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755cd67700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558133c70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ebe0 tx=0x7f754c00eef0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 client.?.objecter ms_handle_connect 0x7f75580a3190 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f754c00ccc0 con 0x7f75580a3190 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: dump: 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.725 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c00ce40 con 0x7f75580a3190 2024-04-20T02:33:29.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 5 monclient: authenticate success, global_id 84658 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 10 monclient: _renew_subs 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c009550 con 0x7f75580a3190 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f754c0138e0 con 0x7f75580a3190 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f7549ffb700 10 monclient: dump: 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.727 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f754b7fe700 4 set_mon_vals no callback set 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 10 client.84658.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 10 monclient: _renew_subs 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550009a70 con 0x7f75580a3190 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.700+0000 7f755f7cc700 1 librados: init done 2024-04-20T02:33:29.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f754c013470 con 0x7f75580a3190 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755f7cc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755f7cc700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 0x7f75581218c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7558068280 con 0x7f7558124d10 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f753c03a320 0x7f753c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(688..688 src has 266..688) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f754c04a830 con 0x7f75580a3190 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 10 client.84658.objecter ms_dispatch 0x7f7558001550 osd_map(688..688 src has 266..688) v4 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 3 client.84658.objecter handle_osd_map got epochs [688,688] > 0 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 3 client.84658.objecter handle_osd_map decoding full epoch 688 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 0x7f75581218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient: get_auth_request con 0x7f7558124d10 auth_method 0 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755dd69700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f753c03a320 0x7f753c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f7549ffb700 20 client.84658.objecter dump_active .. 0 homeless 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755dd69700 10 monclient: get_auth_request con 0x7f753c03a320 auth_method 0 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.701+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f755d568700 10 monclient(hunting): handle_auth_done global_id 84706 payload 824 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f755d568700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7558067790 con 0x7f7558124d10 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 client.84658.objecter ms_handle_connect 0x7f753c03a320 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f755dd69700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f753c03a320 0x7f753c03c7e0 secure :-1 s=READY pgs=3385 cs=0 l=1 rev1=1 crypto rx=0x7f7550009df0 tx=0x7f7550011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 client.84658.objecter ms_handle_connect 0x7f7558124d10 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 client.84658.objecter resend_mon_ops 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f755d568700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 0x7f75581218c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7554011b50 tx=0x7f7554011f10 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7554014440 con 0x7f7558124d10 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 monclient: dump: 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.731 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f755400fb10 con 0x7f7558124d10 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 msgr2=0x7f75581218c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 0x7f75581218c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7554011b50 tx=0x7f7554011f10 comp rx=0 tx=0).stop 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 msgr2=0x7f75581218c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.702+0000 7f7549ffb700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558124d10 0x7f75581218c0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7554011b50 tx=0x7f7554011f10 comp rx=0 tx=0).stop 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755f7cc700 10 start_mon_command cmd=[{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755f7cc700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 0x7f75581221e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755f7cc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f75581250f0 con 0x7f7558121e00 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 0x7f75581221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient: get_auth_request con 0x7f7558121e00 auth_method 0 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:29.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 10 monclient(hunting): handle_auth_done global_id 84715 payload 824 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 1 -- 172.21.15.47:0/3048794474 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "sync_force", "yes_i_really_mean_it": true}) v1 -- 0x7f7558067c60 con 0x7f7558121e00 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f755d568700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 0x7f75581221e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7554011b50 tx=0x7f7554004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f7549ffb700 10 client.84658.objecter ms_handle_connect 0x7f7558121e00 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.712+0000 7f7549ffb700 10 client.84658.objecter resend_mon_ops 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7554022320 con 0x7f7558121e00 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:33:29.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 10 monclient: dump: 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:29.737 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+83 (secure 0 0 0) 0x7f7554022580 con 0x7f7558121e00 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 10 monclient: handle_command_reply 2 [{"prefix": "sync_force", "yes_i_really_mean_it": true}] 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 msgr2=0x7f75581221e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 0x7f75581221e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7554011b50 tx=0x7f7554004750 comp rx=0 tx=0).stop 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 msgr2=0x7f75581221e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f7549ffb700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f7558121e00 0x7f75581221e0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7554011b50 tx=0x7f7554004750 comp rx=0 tx=0).stop 2024-04-20T02:33:29.738 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:29.739 INFO:teuthology.orchestra.run.smithi047.stdout: "ret": 0, 2024-04-20T02:33:29.739 INFO:teuthology.orchestra.run.smithi047.stdout: "msg": "forcing store sync the next time the monitor starts" 2024-04-20T02:33:29.739 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 10 librados: watch_flush enter 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 10 librados: watch_flush exit 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 20 client.84658.objecter shutdown clearing up homeless session... 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 10 client.84658.objecter successfully canceled tick 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f753c03a320 msgr2=0x7f753c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f753c03a320 0x7f753c03c7e0 secure :-1 s=READY pgs=3385 cs=0 l=1 rev1=1 crypto rx=0x7f7550009df0 tx=0x7f7550011000 comp rx=0 tx=0).stop 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 10 monclient: shutdown 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:29.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 msgr2=0x7f7558133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755dd69700 1 -- 172.21.15.47:0/3048794474 reap_dead start 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.713+0000 7f755f7cc700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558133c70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ebe0 tx=0x7f754c00eef0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.714+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 shutdown_connections 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.714+0000 7f755f7cc700 1 --2- 172.21.15.47:0/3048794474 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f75580a3190 0x7f7558133c70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.714+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 >> 172.21.15.47:0/3048794474 conn(0x7f75580a3f90 msgr2=0x7f75580a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.714+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 shutdown_connections 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.714+0000 7f755f7cc700 1 -- 172.21.15.47:0/3048794474 wait complete. 2024-04-20T02:33:29.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:29.714+0000 7f755f7cc700 1 librados: shutdown 2024-04-20T02:33:29.742 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:33:29.742 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-04-20T02:33:29.743 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:33:29.757 INFO:tasks.ceph.mon.i:Stopped 2024-04-20T02:33:29.757 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-04-20T02:33:29.757 INFO:tasks.ceph.mon.h:Sent signal 19 2024-04-20T02:33:29.758 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:33:44.759 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-04-20T02:33:44.759 INFO:tasks.ceph.mon.h:Sent signal 18 2024-04-20T02:33:44.759 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-04-20T02:33:44.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:33:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.902+0000 7ff211d0b700 10 monclient: get_monmap_and_config 2024-04-20T02:33:44.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.902+0000 7ff211d0b700 10 monclient: build_initial_monmap 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.902+0000 7ff211d0b700 10 monclient: monmap: 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:44.903724+0000 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:44.903724+0000 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:44.904 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:44.904 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 Processor -- start 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 -- start start 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient: init 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient: _add_conns ranks=[3,9,2,10,6,7,4,0,5,8,1] 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): picked mon.noname-g con 0x7ff20c0a5080 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): picked mon.noname-a con 0x7ff20c0a1a80 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 0x7ff20c0a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): picked mon.noname-b con 0x7ff20c0a3dc0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:44.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c072380 con 0x7ff20c0a1a80 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0a5460 con 0x7ff20c0a3dc0 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c070b10 con 0x7ff20c0a5080 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.903+0000 7ff211d0b700 10 monclient(hunting): authenticate will time out at 2704.304978s 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20bfff700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 msgr2=0x7ff20c0a4270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 0x7ff20c0a4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59920/0 (socket says 172.21.15.47:59920) 2024-04-20T02:33:44.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 learned_addr learned my addr 172.21.15.47:0/2914000429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 10 monclient(hunting): get_auth_request con 0x7ff20c0a1a80 auth_method 0 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request con 0x7ff20c0a5080 auth_method 0 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:44.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:44.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:44.904+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:45.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 msgr2=0x7ff20c0a4270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:45.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 0x7ff20c0a4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:45.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:45.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 10 monclient(hunting): get_auth_request con 0x7ff20c0a1a80 auth_method 0 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request con 0x7ff20c0a5080 auth_method 0 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:45.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:45.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.105+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 msgr2=0x7ff20c0a4270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 0x7ff20c0a4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 10 monclient(hunting): get_auth_request con 0x7ff20c0a1a80 auth_method 0 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request con 0x7ff20c0a5080 auth_method 0 2024-04-20T02:33:45.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:45.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:45.506+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 msgr2=0x7ff20c0a4270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 0x7ff20c0a4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 10 monclient(hunting): get_auth_request con 0x7ff20c0a1a80 auth_method 0 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T02:33:46.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request con 0x7ff20c0a5080 auth_method 0 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:46.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:46.307+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): tick 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): _check_auth_tickets 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 monclient(hunting): continuing hunt 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): _reopen_session rank -1 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 msgr2=0x7ff20c0a1540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff20c0a5080 0x7ff20c0a1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 msgr2=0x7ff20c0a4270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:47.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20c0a3dc0 0x7ff20c0a4270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 msgr2=0x7ff20c0a3880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.47:3300/0 conn(0x7ff20c0a1a80 0x7ff20c0a3880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient: _add_conns ranks=[7,1,10,5,8,4,6,0] 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 0x7ff1f8005cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): picked mon.noname-i con 0x7ff1f8003880 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff1f80062d0 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): picked mon.noname-f con 0x7ff1f8009fe0 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 -- 172.21.15.47:0/2914000429 --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0751e0 con 0x7ff1f80062d0 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 -- 172.21.15.47:0/2914000429 --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c072380 con 0x7ff1f8009fe0 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:47.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 1 -- 172.21.15.47:0/2914000429 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0a5460 con 0x7ff1f8003880 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff2097fa700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 msgr2=0x7ff1f8005cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 0x7ff1f8005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 10 monclient(hunting): get_auth_request con 0x7ff1f8009fe0 auth_method 0 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:47.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:47.904+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 msgr2=0x7ff1f8005cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 0x7ff1f8005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 10 monclient(hunting): get_auth_request con 0x7ff1f8009fe0 auth_method 0 2024-04-20T02:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:48.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:48.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.105+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T02:33:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 msgr2=0x7ff1f8005cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:48.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 0x7ff1f8005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:48.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 10 monclient(hunting): get_auth_request con 0x7ff1f8009fe0 auth_method 0 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:48.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:48.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T02:33:49.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20b7fe700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 msgr2=0x7ff1f8005cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:49.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 0x7ff1f8005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:49.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 10 monclient(hunting): get_auth_request con 0x7ff1f8009fe0 auth_method 0 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:33:49.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.307+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T02:33:49.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.498+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060788749 0 0) 0x7ff20c0751e0 con 0x7ff1f80062d0 2024-04-20T02:33:49.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.498+0000 7ff20a7fc700 10 monclient(hunting): _negotiate 2024-04-20T02:33:49.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.498+0000 7ff20a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:49.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.498+0000 7ff20a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:49.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.498+0000 7ff20a7fc700 10 monclient(hunting): my global_id is 55900 2024-04-20T02:33:49.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.498+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1f40035b0 con 0x7ff1f80062d0 2024-04-20T02:33:49.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016320711 0 0) 0x7ff1f40035b0 con 0x7ff1f80062d0 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 msgr2=0x7ff1f8005cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff1f8003880 0x7ff1f8005cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 msgr2=0x7ff1f800c4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 --2- 172.21.15.47:0/2914000429 >> v2:172.21.15.179:3300/0 conn(0x7ff1f8009fe0 0x7ff1f800c4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 monclient: found mon.noname-f-legacy 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.179:6789/0 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 reap_dead start 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f800dc60 con 0x7ff1f80062d0 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7ff20000c7e0 con 0x7ff1f80062d0 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff211d0b700 5 monclient: authenticate success, global_id 55900 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff211d0b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: dump: 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] but i'm connected to v1:172.21.15.179:6789/0, reconnecting 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 >> v1:172.21.15.179:6789/0 conn(0x7ff1f80062d0 legacy=0x7ff1f8008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient: _add_conns ranks=[0,2,6,8,7,1,4,5,3] 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f4004250 0x7ff1f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): picked mon.a con 0x7ff1f4004250 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004e60 0x7ff1f40052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): picked mon.b con 0x7ff1f4004e60 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff1f4009d20 0x7ff1f400c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): picked mon.d con 0x7ff1f4009d20 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c072380 con 0x7ff1f4004250 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0a5460 con 0x7ff1f4004e60 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f400cac0 con 0x7ff1f4009d20 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20affd700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004e60 msgr2=0x7ff1f40052e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20a7fc700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff211d0b700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20affd700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004e60 0x7ff1f40052e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f4004250 0x7ff1f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff1f4009d20 0x7ff1f400c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request con 0x7ff1f4004250 auth_method 0 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): get_auth_request con 0x7ff1f4009d20 auth_method 0 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.499+0000 7ff20bfff700 10 monclient(hunting): handle_auth_done global_id 55900 payload 293 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004e60 msgr2=0x7ff1f40052e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004e60 0x7ff1f40052e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f4004250 msgr2=0x7ff1f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f4004250 0x7ff1f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 monclient: found mon.d 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 -- 172.21.15.47:0/2914000429 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40035b0 con 0x7ff1f4009d20 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 10 monclient: _finish_auth 0 2024-04-20T02:33:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f4004250 0x7ff1f4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20bfff700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff1f4009d20 0x7ff1f400c1b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0a41a0 tx=0x7ff1f4005880 comp rx=0 tx=0).ready entity=mon.6 client_cookie=560b507f2977e237 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff20400dc80 con 0x7ff1f4009d20 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: dump: 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff211d0b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204013030 con 0x7ff1f4009d20 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff211d0b700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff211d0b700 4 set_mon_vals no callback set 2024-04-20T02:33:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 1 -- 172.21.15.47:0/2914000429 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff20400de00 con 0x7ff1f4009d20 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.500+0000 7ff20a7fc700 10 monclient: dump: 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 10 monclient: shutdown 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 -- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff1f4009d20 msgr2=0x7ff1f400c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff1f4009d20 0x7ff1f400c1b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0a41a0 tx=0x7ff1f4005880 comp rx=0 tx=0).stop 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 -- 172.21.15.47:0/2914000429 shutdown_connections 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff1f4009d20 0x7ff1f400c1b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004e60 0x7ff1f40052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 --2- 172.21.15.47:0/2914000429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f4004250 0x7ff1f4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 -- 172.21.15.47:0/2914000429 >> 172.21.15.47:0/2914000429 conn(0x7ff20c0a84a0 msgr2=0x7ff20c0a4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 -- 172.21.15.47:0/2914000429 shutdown_connections 2024-04-20T02:33:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.501+0000 7ff211d0b700 1 -- 172.21.15.47:0/2914000429 wait complete. 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.502+0000 7ff211d0b700 10 monclient: build_initial_monmap 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.502+0000 7ff211d0b700 10 monclient: monmap: 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.503+0000 7ff211d0b700 1 librados: starting msgr at 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.503+0000 7ff211d0b700 1 librados: starting objecter 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.503+0000 7ff211d0b700 1 Processor -- start 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 -- start start 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 librados: setting wanted keys 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 librados: calling monclient init 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient: init 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient: _add_conns ranks=[2,3,1,0,4,5,7,8,6] 2024-04-20T02:33:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004250 0x7ff20c09dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): picked mon.noname-b con 0x7ff1f4004250 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 0x7ff20c132f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): picked mon.noname-g con 0x7ff1f4004e60 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f4009d20 0x7ff20c1cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): picked mon.noname-f con 0x7ff1f4009d20 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c125100 con 0x7ff1f4009d20 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0a21e0 con 0x7ff1f4004250 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c071260 con 0x7ff1f4004e60 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff211d0b700 10 monclient(hunting): authenticate will time out at 2708.906878s 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff20b7fe700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004250 msgr2=0x7ff20c09dd40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.505+0000 7ff20b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004250 0x7ff20c09dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 0x7ff20c132f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 0x7ff20c132f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:44472/0 (socket says 172.21.15.47:44472) 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 1 -- 172.21.15.47:0/1641094302 learned_addr learned my addr 172.21.15.47:0/1641094302 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f4009d20 0x7ff20c1cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): get_auth_request con 0x7ff1f4004e60 auth_method 0 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): get_auth_request con 0x7ff1f4009d20 auth_method 0 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient(hunting): handle_auth_done global_id 55909 payload 824 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 -- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 msgr2=0x7ff20c132f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 0x7ff20c132f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 -- 172.21.15.47:0/1641094302 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004250 msgr2=0x7ff20c09dd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004250 0x7ff20c09dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 monclient: found mon.noname-f 2024-04-20T02:33:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 -- 172.21.15.47:0/1641094302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c123d20 con 0x7ff1f4009d20 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20affd700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 0x7ff20c132f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff20bfff700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f4009d20 0x7ff20c1cbd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff204003d90 tx=0x7ff204006150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff1f3fff700 10 client.?.objecter ms_handle_connect 0x7ff1f4009d20 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff1f3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2040268b0 con 0x7ff1f4009d20 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff1f3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.506+0000 7ff1f3fff700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: dump: 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: _finish_auth 0 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:33:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204026a10 con 0x7ff1f4009d20 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 5 monclient: authenticate success, global_id 55909 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 10 monclient: _renew_subs 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c09e800 con 0x7ff1f4009d20 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff204026bc0 con 0x7ff1f4009d20 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 10 client.55909.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff1f3fff700 10 monclient: dump: 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 10 monclient: _renew_subs 2024-04-20T02:33:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c1245a0 con 0x7ff1f4009d20 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff209ffb700 4 set_mon_vals no callback set 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.507+0000 7ff211d0b700 1 librados: init done 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff211d0b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff211d0b700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff211d0b700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff204026d20 con 0x7ff1f4009d20 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c0a46a0 con 0x7ff1f4009d20 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff1f803af70 0x7ff1f803d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(688..688 src has 266..688) v4 ==== 6288+0+0 (secure 0 0 0) 0x7ff20404cfd0 con 0x7ff1f4009d20 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 10 client.55909.objecter ms_dispatch 0x7ff20c001550 osd_map(688..688 src has 266..688) v4 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 3 client.55909.objecter handle_osd_map got epochs [688,688] > 0 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 3 client.55909.objecter handle_osd_map decoding full epoch 688 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff1f3fff700 20 client.55909.objecter dump_active .. 0 homeless 2024-04-20T02:33:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff1f803af70 0x7ff1f803d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.508+0000 7ff20b7fe700 10 monclient: get_auth_request con 0x7ff1f803af70 auth_method 0 2024-04-20T02:33:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.509+0000 7ff1f3fff700 10 client.55909.objecter ms_handle_connect 0x7ff1f803af70 2024-04-20T02:33:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.509+0000 7ff20b7fe700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff1f803af70 0x7ff1f803d430 secure :-1 s=READY pgs=3386 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc001eb0 tx=0x7ff1fc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.513+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ff20c0a46a0 con 0x7ff1f4009d20 2024-04-20T02:33:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.513+0000 7ff1f3fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.513+0000 7ff1f3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:49.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.819+0000 7ff211d0b700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:33:49.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.819+0000 7ff211d0b700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:49.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.819+0000 7ff211d0b700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:49.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:49.819+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff20c0a46a0 con 0x7ff1f4009d20 2024-04-20T02:33:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:52.506+0000 7ff1f2ffd700 10 monclient: tick 2024-04-20T02:33:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:52.506+0000 7ff1f2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:52.506+0000 7ff1f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:52.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:52.506+0000 7ff1f2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:53.506+0000 7ff1f2ffd700 10 monclient: tick 2024-04-20T02:33:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:53.506+0000 7ff1f2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:53.506+0000 7ff1f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:54.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.506+0000 7ff1f2ffd700 10 monclient: tick 2024-04-20T02:33:54.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.506+0000 7ff1f2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:54.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.506+0000 7ff1f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:54.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.508+0000 7ff208ff9700 10 client.55909.objecter tick 2024-04-20T02:33:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.779+0000 7ff1f3fff700 1 -- 172.21.15.47:0/1641094302 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3164 (secure 0 0 0) 0x7ff20c0a46a0 con 0x7ff1f4009d20 2024-04-20T02:33:54.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.779+0000 7ff1f3fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:33:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.779+0000 7ff1f3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:54.780 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":116,"quorum":[0,1,3,4,5,6,8],"quorum_names":["a","f","g","c","h","d","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:33:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.782+0000 7ff211d0b700 10 librados: watch_flush enter 2024-04-20T02:33:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.782+0000 7ff211d0b700 10 librados: watch_flush exit 2024-04-20T02:33:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.782+0000 7ff211d0b700 20 client.55909.objecter shutdown clearing up homeless session... 2024-04-20T02:33:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.782+0000 7ff211d0b700 10 client.55909.objecter successfully canceled tick 2024-04-20T02:33:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff1f803af70 msgr2=0x7ff1f803d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:54.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff1f803af70 0x7ff1f803d430 secure :-1 s=READY pgs=3386 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc001eb0 tx=0x7ff1fc008040 comp rx=0 tx=0).stop 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 10 monclient: shutdown 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f4009d20 msgr2=0x7ff20c1cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f4009d20 0x7ff20c1cbd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff204003d90 tx=0x7ff204006150 comp rx=0 tx=0).stop 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 shutdown_connections 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff1f803af70 0x7ff1f803d430 unknown :-1 s=CLOSED pgs=3386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1f4009d20 0x7ff20c1cbd10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1f4004250 0x7ff20c09dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 --2- 172.21.15.47:0/1641094302 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff1f4004e60 0x7ff20c132f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 >> 172.21.15.47:0/1641094302 conn(0x7ff20c0a84a0 msgr2=0x7ff20c133bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 shutdown_connections 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 -- 172.21.15.47:0/1641094302 wait complete. 2024-04-20T02:33:54.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.783+0000 7ff211d0b700 1 librados: shutdown 2024-04-20T02:33:54.799 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-04-20T02:33:54.799 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-04-20T02:33:54.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:33:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.948+0000 7fb3dbb63700 10 monclient: get_monmap_and_config 2024-04-20T02:33:54.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.948+0000 7fb3dbb63700 10 monclient: build_initial_monmap 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.948+0000 7fb3dbb63700 10 monclient: monmap: 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:54.949952+0000 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:54.949952+0000 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:54.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.949+0000 7fb3dbb63700 1 Processor -- start 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.949+0000 7fb3dbb63700 1 -- start start 2024-04-20T02:33:54.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.949+0000 7fb3dbb63700 10 monclient: init 2024-04-20T02:33:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:54.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient: _add_conns ranks=[0,10,2,6,7,8,4,5,9,1,3] 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb3d4066610 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb3d409d2d0 0x7fb3d40a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): picked mon.noname-f con 0x7fb3d409d2d0 addr v2:172.21.15.179:3300/0 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3d4141700 0x7fb3d4143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): picked mon.noname-b con 0x7fb3d4141700 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d40751e0 con 0x7fb3d4066610 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d409a9a0 con 0x7fb3d409d2d0 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40669f0 con 0x7fb3d4141700 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3dbb63700 10 monclient(hunting): authenticate will time out at 2714.351356s 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3da100700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3d4141700 msgr2=0x7fb3d4143af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3da100700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3d4141700 0x7fb3d4143af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d98ff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb3d4066610 0x7fb3d40a1f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57720/0 (socket says 172.21.15.47:57720) 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d98ff700 1 -- 172.21.15.47:0/3051652621 learned_addr learned my addr 172.21.15.47:0/3051652621 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3051652621 >> v2:172.21.15.179:3300/0 conn(0x7fb3d409d2d0 0x7fb3d40a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request con 0x7fb3d409d2d0 auth_method 0 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577877662 0 0) 0x7fb3d40751e0 con 0x7fb3d4066610 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 10 monclient(hunting): _negotiate 2024-04-20T02:33:54.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 10 monclient(hunting): my global_id is 104256 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c00035b0 con 0x7fb3d4066610 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2496872963 0 0) 0x7fb3c00035b0 con 0x7fb3d4066610 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3d4141700 msgr2=0x7fb3d4143af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3d4141700 0x7fb3d4143af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 >> v2:172.21.15.179:3300/0 conn(0x7fb3d409d2d0 msgr2=0x7fb3d40a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 --2- 172.21.15.47:0/3051652621 >> v2:172.21.15.179:3300/0 conn(0x7fb3d409d2d0 0x7fb3d40a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 10 monclient: _finish_hunting 0 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 1 monclient: found mon.noname-a-legacy 2024-04-20T02:33:54.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.950+0000 7fb3d88fd700 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.47:6789/0 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4099ff0 con 0x7fb3d4066610 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: _finish_auth 0 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 466391792 0 0) 0x7fb3d00018e0 con 0x7fb3d4066610 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3dbb63700 5 monclient: authenticate success, global_id 104256 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3051652621 >> v2:172.21.15.179:3300/0 conn(0x7fb3d409d2d0 0x7fb3d40a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3dbb63700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: dump: 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:54.974 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:54.975 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 monclient: mon.a has (v2) addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] but i'm connected to v1:172.21.15.47:6789/0, reconnecting 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 >> v1:172.21.15.47:6789/0 conn(0x7fb3d4066610 legacy=0x7fb3d40a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: _add_conns ranks=[3,6,0,7,8,2,1,4,5] 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3c0004250 0x7fb3c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): picked mon.g con 0x7fb3c0004250 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0004de0 0x7fb3c0005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): picked mon.d con 0x7fb3c0004de0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:54.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c0009c80 0x7fb3c000c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): picked mon.a con 0x7fb3c0009c80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d409a9a0 con 0x7fb3c0009c80 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d40669f0 con 0x7fb3c0004250 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c000ca20 con 0x7fb3c0004de0 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient(hunting): _finish_auth 0 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3dbb63700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0004de0 0x7fb3c0005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c0009c80 0x7fb3c000c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3c0004250 0x7fb3c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): get_auth_request con 0x7fb3c0009c80 auth_method 0 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): get_auth_request con 0x7fb3c0004250 auth_method 0 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request con 0x7fb3c0004de0 auth_method 0 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): _init_auth already have auth, reseting 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:54.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient(hunting): handle_auth_done global_id 104256 payload 293 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 -- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0004de0 msgr2=0x7fb3c0005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0004de0 0x7fb3c0005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 -- 172.21.15.47:0/3051652621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3c0004250 msgr2=0x7fb3c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb3c0004250 0x7fb3c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient: _finish_hunting 0 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 monclient: found mon.a 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:54.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 -- 172.21.15.47:0/3051652621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c00035b0 con 0x7fb3c0009c80 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient: _finish_auth 0 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 10 monclient: _check_auth_tickets 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 -- 172.21.15.47:0/3051652621 reap_dead start 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3da100700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c0009c80 0x7fb3c000c110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb3d409c6f0 tx=0x7fb3c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f4774d47b78c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3cc009780 con 0x7fb3c0009c80 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 10 monclient: dump: 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.981 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.951+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc00e420 con 0x7fb3c0009c80 2024-04-20T02:33:54.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3d88fd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 4 set_mon_vals no callback set 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3d88fd700 1 -- 172.21.15.47:0/3051652621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3cc00ea00 con 0x7fb3c0009c80 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3d88fd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3d88fd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3d88fd700 10 monclient: dump: 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:54.983 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 10 monclient: shutdown 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c0009c80 msgr2=0x7fb3c000c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c0009c80 0x7fb3c000c110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb3d409c6f0 tx=0x7fb3c0005880 comp rx=0 tx=0).stop 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3051652621 shutdown_connections 2024-04-20T02:33:54.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3051652621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3c0009c80 0x7fb3c000c110 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3051652621 >> 172.21.15.47:0/3051652621 conn(0x7fb3d40a3000 msgr2=0x7fb3d40a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3051652621 shutdown_connections 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.952+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3051652621 wait complete. 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 10 monclient: build_initial_monmap 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 10 monclient: monmap: 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:54.985 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:54.986 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 1 librados: starting msgr at 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 1 librados: starting objecter 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 1 Processor -- start 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 1 -- start start 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 1 librados: setting wanted keys 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 1 librados: calling monclient init 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.953+0000 7fb3dbb63700 10 monclient: init 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient: _add_conns ranks=[6,8,7,3,2,4,5,1,0] 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 0x7fb3d41d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): picked mon.noname-d con 0x7fb3c0009c80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:54.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb3d41d1150 0x7fb3d408c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): picked mon.noname-e con 0x7fb3d41d1150 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb3d408cac0 0x7fb3d408cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): picked mon.noname-i con 0x7fb3d408cac0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d408f380 con 0x7fb3c0009c80 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d408fc90 con 0x7fb3d408cac0 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d408e690 con 0x7fb3d41d1150 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3dbb63700 10 monclient(hunting): authenticate will time out at 2714.355281s 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 0x7fb3d41d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 0x7fb3d41d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48810/0 (socket says 172.21.15.47:48810) 2024-04-20T02:33:54.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3da100700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb3d408cac0 msgr2=0x7fb3d408cf40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 -- 172.21.15.47:0/3497789947 learned_addr learned my addr 172.21.15.47:0/3497789947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): get_auth_request con 0x7fb3c0009c80 auth_method 0 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3da100700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb3d408cac0 0x7fb3d408cf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb3d41d1150 0x7fb3d408c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request con 0x7fb3d41d1150 auth_method 0 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient(hunting): handle_auth_done global_id 55473 payload 824 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb3d41d1150 msgr2=0x7fb3d408c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb3d41d1150 0x7fb3d408c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb3d408cac0 msgr2=0x7fb3d408cf40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fb3d408cac0 0x7fb3d408cf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 monclient: found mon.noname-d 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d409c720 con 0x7fb3c0009c80 2024-04-20T02:33:54.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3d98ff700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 0x7fb3d41d0c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0007340 tx=0x7fb3d0007650 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3ca7fc700 10 client.?.objecter ms_handle_connect 0x7fb3c0009c80 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.954+0000 7fb3ca7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3d00119f0 con 0x7fb3c0009c80 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: dump: 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:54.991 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0011b50 con 0x7fb3c0009c80 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 5 monclient: authenticate success, global_id 55473 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 monclient: _renew_subs 2024-04-20T02:33:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3d0011cc0 con 0x7fb3c0009c80 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d4099410 con 0x7fb3c0009c80 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3cbfff700 4 set_mon_vals no callback set 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 10 monclient: dump: 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 client.55473.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 monclient: _renew_subs 2024-04-20T02:33:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d408e160 con 0x7fb3c0009c80 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 1 librados: init done 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb3d0011580 con 0x7fb3c0009c80 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 0x7fb3d402cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.955+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d4068180 con 0x7fb3d406aac0 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 0x7fb3d402cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient: get_auth_request con 0x7fb3d406aac0 auth_method 0 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb3b803a2f0 0x7fb3b803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(689..689 src has 266..689) v4 ==== 6661+0+0 (secure 0 0 0) 0x7fb3d004a910 con 0x7fb3c0009c80 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 10 client.55473.objecter ms_dispatch 0x7fb3d4001550 osd_map(689..689 src has 266..689) v4 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 3 client.55473.objecter handle_osd_map got epochs [689,689] > 0 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 3 client.55473.objecter handle_osd_map decoding full epoch 689 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 20 client.55473.objecter dump_active .. 0 homeless 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_done global_id 104265 payload 824 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3d409c230 con 0x7fb3d406aac0 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3da100700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb3b803a2f0 0x7fb3b803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 10 client.55473.objecter ms_handle_connect 0x7fb3d406aac0 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 10 client.55473.objecter resend_mon_ops 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 0x7fb3d402cd40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009470 tx=0x7fb3c4014d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3da100700 10 monclient: get_auth_request con 0x7fb3b803a2f0 auth_method 0 2024-04-20T02:33:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3c40127a0 con 0x7fb3d406aac0 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.956+0000 7fb3ca7fc700 10 monclient: dump: 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb3c4004240 con 0x7fb3d406aac0 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 msgr2=0x7fb3d402cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 0x7fb3d402cd40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009470 tx=0x7fb3c4014d70 comp rx=0 tx=0).stop 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 msgr2=0x7fb3d402cd40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d406aac0 0x7fb3d402cd40 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009470 tx=0x7fb3c4014d70 comp rx=0 tx=0).stop 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3ca7fc700 10 client.55473.objecter ms_handle_connect 0x7fb3b803a2f0 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.957+0000 7fb3da100700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb3b803a2f0 0x7fb3b803c7b0 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c9c0 tx=0x7fb3cc005000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3dbb63700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:54.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 0x7fb3d408d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3dbb63700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d406a400 con 0x7fb3d41d1530 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 0x7fb3d408d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3d90fe700 10 monclient: get_auth_request con 0x7fb3d41d1530 auth_method 0 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3d90fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3d90fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.964+0000 7fb3d90fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3d90fe700 10 monclient(hunting): handle_auth_done global_id 104274 payload 824 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3d90fe700 1 -- 172.21.15.47:0/3497789947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb3d4099d10 con 0x7fb3d41d1530 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3d90fe700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 0x7fb3d408d480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4015010 tx=0x7fb3c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:54.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 client.55473.objecter ms_handle_connect 0x7fb3d41d1530 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 client.55473.objecter resend_mon_ops 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3c4005c30 con 0x7fb3d41d1530 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 monclient: dump: 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.000 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9162 (secure 0 0 0) 0x7fb3c4005d90 con 0x7fb3d41d1530 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 msgr2=0x7fb3d408d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 0x7fb3d408d480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4015010 tx=0x7fb3c40047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 msgr2=0x7fb3d408d480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.965+0000 7fb3ca7fc700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d41d1530 0x7fb3d408d480 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4015010 tx=0x7fb3c40047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:55.002 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.003 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.004 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:55.005 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.011 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.012 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.012 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:55.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.013 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.014 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.015 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.016 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.018 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.020 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.021 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:55.022 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:55.023 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:55.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 10 librados: watch_flush enter 2024-04-20T02:33:55.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 10 librados: watch_flush exit 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 20 client.55473.objecter shutdown clearing up homeless session... 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 10 client.55473.objecter successfully canceled tick 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb3b803a2f0 msgr2=0x7fb3b803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb3b803a2f0 0x7fb3b803c7b0 secure :-1 s=READY pgs=3387 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c9c0 tx=0x7fb3cc005000 comp rx=0 tx=0).stop 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 10 monclient: shutdown 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 msgr2=0x7fb3d41d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 0x7fb3d41d0c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0007340 tx=0x7fb3d0007650 comp rx=0 tx=0).stop 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3da100700 1 -- 172.21.15.47:0/3497789947 reap_dead start 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 shutdown_connections 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 --2- 172.21.15.47:0/3497789947 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fb3c0009c80 0x7fb3d41d0c10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 >> 172.21.15.47:0/3497789947 conn(0x7fb3d40a3000 msgr2=0x7fb3d409fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 shutdown_connections 2024-04-20T02:33:55.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 -- 172.21.15.47:0/3497789947 wait complete. 2024-04-20T02:33:55.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:54.966+0000 7fb3dbb63700 1 librados: shutdown 2024-04-20T02:33:55.027 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:55.027 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-04-20T02:33:55.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.168+0000 7f5e36464700 10 monclient: get_monmap_and_config 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.168+0000 7f5e36464700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.168+0000 7f5e36464700 10 monclient: monmap: 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:55.169228+0000 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:55.169228+0000 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.168 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:55.169 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.168+0000 7f5e36464700 1 Processor -- start 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.168+0000 7f5e36464700 1 -- start start 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.168+0000 7f5e36464700 10 monclient: init 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient: _add_conns ranks=[5,3,9,4,7,2,1,6,0,10,8] 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 0x7f5e30135d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): picked mon.noname-h con 0x7f5e3009fc10 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 0x7f5e300a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): picked mon.noname-g con 0x7f5e3009fff0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e300a03d0 0x7f5e300a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): picked mon.noname-a con 0x7f5e300a03d0 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:55.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3009def0 con 0x7f5e300a03d0 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300a3dd0 con 0x7f5e3009fff0 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3009dc10 con 0x7f5e3009fc10 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e36464700 10 monclient(hunting): authenticate will time out at 2714.570537s 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e34a01700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e300a03d0 0x7f5e300a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 0x7f5e300a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 0x7f5e30135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 0x7f5e30135d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:47014/0 (socket says 172.21.15.47:47014) 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 0x7f5e300a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:41556/0 (socket says 172.21.15.47:41556) 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 1 -- 172.21.15.47:0/3623081163 learned_addr learned my addr 172.21.15.47:0/3623081163 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 10 monclient(hunting): get_auth_request con 0x7f5e3009fc10 auth_method 0 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e34a01700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e300a03d0 0x7f5e300a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49542/0 (socket says 172.21.15.47:49542) 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2f7fe700 10 monclient(hunting): get_auth_request con 0x7f5e3009fff0 auth_method 0 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e34a01700 10 monclient(hunting): get_auth_request con 0x7f5e300a03d0 auth_method 0 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e34a01700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e34a01700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.169+0000 7f5e34a01700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e34a01700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e34a01700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e34a01700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_done global_id 84623 payload 274 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 -- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 msgr2=0x7f5e300a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 --2- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 0x7f5e300a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 -- 172.21.15.47:0/3623081163 >> v2:172.21.15.47:3300/0 conn(0x7f5e300a03d0 msgr2=0x7f5e300a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 --2- 172.21.15.47:0/3623081163 >> v2:172.21.15.47:3300/0 conn(0x7f5e300a03d0 0x7f5e300a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 monclient: found mon.noname-h 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 -- 172.21.15.47:0/3623081163 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300751e0 con 0x7f5e3009fc10 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2f7fe700 1 --2- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 0x7f5e300a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2ffff700 1 --2- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 0x7f5e30135d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e200098f0 tx=0x7f5e20009c00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e354604263a95465 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 1 -- 172.21.15.47:0/3623081163 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e20011aa0 con 0x7f5e3009fc10 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: dump: 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.185 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 1 -- 172.21.15.47:0/3623081163 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e20011c00 con 0x7f5e3009fc10 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 1 -- 172.21.15.47:0/3623081163 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e20011d70 con 0x7f5e3009fc10 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e36464700 5 monclient: authenticate success, global_id 84623 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e36464700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e36464700 4 set_mon_vals no callback set 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.170+0000 7f5e2effd700 10 monclient: dump: 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.187 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 10 monclient: shutdown 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 -- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 msgr2=0x7f5e30135d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 --2- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 0x7f5e30135d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e200098f0 tx=0x7f5e20009c00 comp rx=0 tx=0).stop 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 -- 172.21.15.47:0/3623081163 shutdown_connections 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 --2- 172.21.15.47:0/3623081163 >> v2:172.21.15.47:3300/0 conn(0x7f5e300a03d0 0x7f5e300a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 --2- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f5e3009fc10 0x7f5e30135d00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 --2- 172.21.15.47:0/3623081163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5e3009fff0 0x7f5e300a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 -- 172.21.15.47:0/3623081163 >> 172.21.15.47:0/3623081163 conn(0x7f5e300a3000 msgr2=0x7f5e300a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 -- 172.21.15.47:0/3623081163 shutdown_connections 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.171+0000 7f5e36464700 1 -- 172.21.15.47:0/3623081163 wait complete. 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 10 monclient: monmap: 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:55.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 1 librados: starting msgr at 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 1 librados: starting objecter 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 1 Processor -- start 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 1 -- start start 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 1 librados: setting wanted keys 2024-04-20T02:33:55.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 1 librados: calling monclient init 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.172+0000 7f5e36464700 10 monclient: init 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient: _add_conns ranks=[6,1,4,7,0,2,5,3,8] 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e3009fc10 0x7f5e301ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): picked mon.noname-d con 0x7f5e3009fc10 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e3009fff0 0x7f5e30134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): picked mon.noname-f con 0x7f5e3009fff0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e300a03d0 0x7f5e30134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): picked mon.noname-c con 0x7f5e300a03d0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3009cae0 con 0x7f5e3009fff0 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3012eda0 con 0x7f5e300a03d0 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3012f2f0 con 0x7f5e3009fc10 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e36464700 10 monclient(hunting): authenticate will time out at 2714.574312s 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e3009fc10 0x7f5e301ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e300a03d0 0x7f5e30134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e3009fc10 0x7f5e301ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48812/0 (socket says 172.21.15.47:48812) 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 -- 172.21.15.47:0/2664071000 learned_addr learned my addr 172.21.15.47:0/2664071000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e300a03d0 0x7f5e30134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36274/0 (socket says 172.21.15.47:36274) 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): get_auth_request con 0x7f5e3009fc10 auth_method 0 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): get_auth_request con 0x7f5e300a03d0 auth_method 0 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e3009fff0 0x7f5e30134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): get_auth_request con 0x7f5e3009fff0 auth_method 0 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e34a01700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient(hunting): handle_auth_done global_id 55482 payload 824 2024-04-20T02:33:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e300a03d0 msgr2=0x7f5e30134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e300a03d0 0x7f5e30134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e3009fff0 msgr2=0x7f5e30134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e3009fff0 0x7f5e30134a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 monclient: found mon.noname-d 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240094a0 con 0x7f5e3009fc10 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2ffff700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e3009fc10 0x7f5e301ca210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009c40 tx=0x7f5e20004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2cff9700 10 client.?.objecter ms_handle_connect 0x7f5e3009fc10 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.173+0000 7f5e2cff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e200036a0 con 0x7f5e3009fc10 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: dump: 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e20003820 con 0x7f5e3009fc10 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 5 monclient: authenticate success, global_id 55482 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e20003e00 con 0x7f5e3009fc10 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2cff9700 10 monclient: dump: 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.197 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e2e7fc700 4 set_mon_vals no callback set 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 10 monclient: _renew_subs 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e20009550 con 0x7f5e3009fc10 2024-04-20T02:33:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 10 client.55482.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 10 monclient: _renew_subs 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3012ff50 con 0x7f5e3009fc10 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 1 librados: init done 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.174+0000 7f5e36464700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 0x7f5e3012c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301300e0 con 0x7f5e3006aac0 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 0x7f5e3012c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient: get_auth_request con 0x7f5e3006aac0 auth_method 0 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_done global_id 84676 payload 824 2024-04-20T02:33:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e30066910 con 0x7f5e3006aac0 2024-04-20T02:33:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5e20021070 con 0x7f5e3009fc10 2024-04-20T02:33:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2f7fe700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 0x7f5e3012c8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5e300a64d0 tx=0x7f5e24011f60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e1c03a320 0x7f5e1c03c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 10 client.55482.objecter ms_handle_connect 0x7f5e3006aac0 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 10 client.55482.objecter resend_mon_ops 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(689..689 src has 266..689) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f5e2004bcb0 con 0x7f5e3009fc10 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 10 client.55482.objecter ms_dispatch 0x7f5e30001550 osd_map(689..689 src has 266..689) v4 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 3 client.55482.objecter handle_osd_map got epochs [689,689] > 0 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 3 client.55482.objecter handle_osd_map decoding full epoch 689 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 20 client.55482.objecter dump_active .. 0 homeless 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.175+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e24014440 con 0x7f5e3006aac0 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 10 monclient: dump: 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5e2400fb10 con 0x7f5e3006aac0 2024-04-20T02:33:55.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 msgr2=0x7f5e3012c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 0x7f5e3012c8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5e300a64d0 tx=0x7f5e24011f60 comp rx=0 tx=0).stop 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e34a01700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e1c03a320 0x7f5e1c03c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 msgr2=0x7f5e3012c8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3006aac0 0x7f5e3012c8a0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5e300a64d0 tx=0x7f5e24011f60 comp rx=0 tx=0).stop 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e34a01700 10 monclient: get_auth_request con 0x7f5e1c03a320 auth_method 0 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e2cff9700 10 client.55482.objecter ms_handle_connect 0x7f5e1c03a320 2024-04-20T02:33:55.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.176+0000 7f5e34a01700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e1c03a320 0x7f5e1c03c7e0 secure :-1 s=READY pgs=3388 cs=0 l=1 rev1=1 crypto rx=0x7f5e28011340 tx=0x7f5e2800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.183+0000 7f5e36464700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.183+0000 7f5e36464700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 0x7f5e3012d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.183+0000 7f5e36464700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.183+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30068230 con 0x7f5e3012cde0 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 0x7f5e3012d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient: get_auth_request con 0x7f5e3012cde0 auth_method 0 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 10 monclient(hunting): handle_auth_done global_id 84685 payload 824 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 1 -- 172.21.15.47:0/2664071000 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5e300a08b0 con 0x7f5e3012cde0 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2cff9700 10 client.55482.objecter ms_handle_connect 0x7f5e3012cde0 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2cff9700 10 client.55482.objecter resend_mon_ops 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2f7fe700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 0x7f5e3012d1f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e300a64d0 tx=0x7f5e24004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e240222a0 con 0x7f5e3012cde0 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2cff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.184+0000 7f5e2cff9700 10 monclient: dump: 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.207 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9162 (secure 0 0 0) 0x7f5e24004020 con 0x7f5e3012cde0 2024-04-20T02:33:55.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 msgr2=0x7f5e3012d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 0x7f5e3012d1f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e300a64d0 tx=0x7f5e24004750 comp rx=0 tx=0).stop 2024-04-20T02:33:55.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 msgr2=0x7f5e3012d1f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e2cff9700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e3012cde0 0x7f5e3012d1f0 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e300a64d0 tx=0x7f5e24004750 comp rx=0 tx=0).stop 2024-04-20T02:33:55.212 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:55.212 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:55.213 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.214 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.215 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:55.216 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.217 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.218 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.219 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.220 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.221 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.222 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.223 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.224 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.225 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.226 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:55.227 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.228 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.229 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.229 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.229 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.229 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.229 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.231 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:55.231 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.232 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.234 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:55.235 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.237 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.238 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:55.238 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 10 librados: watch_flush enter 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 10 librados: watch_flush exit 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 20 client.55482.objecter shutdown clearing up homeless session... 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 10 client.55482.objecter successfully canceled tick 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e1c03a320 msgr2=0x7f5e1c03c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e1c03a320 0x7f5e1c03c7e0 secure :-1 s=READY pgs=3388 cs=0 l=1 rev1=1 crypto rx=0x7f5e28011340 tx=0x7f5e2800b040 comp rx=0 tx=0).stop 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 10 monclient: shutdown 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e3009fc10 msgr2=0x7f5e301ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 --2- 172.21.15.47:0/2664071000 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e3009fc10 0x7f5e301ca210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009c40 tx=0x7f5e20004750 comp rx=0 tx=0).stop 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e34a01700 1 -- 172.21.15.47:0/2664071000 reap_dead start 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e34a01700 1 -- 172.21.15.47:0/2664071000 reap_dead start 2024-04-20T02:33:55.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 shutdown_connections 2024-04-20T02:33:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 >> 172.21.15.47:0/2664071000 conn(0x7f5e300a3000 msgr2=0x7f5e300a5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 shutdown_connections 2024-04-20T02:33:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 -- 172.21.15.47:0/2664071000 wait complete. 2024-04-20T02:33:55.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.185+0000 7f5e36464700 1 librados: shutdown 2024-04-20T02:33:55.241 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:55.241 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-04-20T02:33:55.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:33:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.385+0000 7f18a7162700 10 monclient: get_monmap_and_config 2024-04-20T02:33:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.385+0000 7f18a7162700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.385+0000 7f18a7162700 10 monclient: monmap: 2024-04-20T02:33:55.385 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:55.386690+0000 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:55.386690+0000 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.386 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:55.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 Processor -- start 2024-04-20T02:33:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 -- start start 2024-04-20T02:33:55.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient: init 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient: _add_conns ranks=[3,8,0,4,7,5,6,1,10,2,9] 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18a0137d90 0x7f18a00a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): picked mon.noname-g con 0x7f18a0137d90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f18a0138170 0x7f18a00a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): picked mon.noname-e con 0x7f18a0138170 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18a01385b0 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a00751e0 con 0x7f18a01385b0 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a009fa40 con 0x7f18a0137d90 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f18a009dfc0 con 0x7f18a0138170 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.386+0000 7f18a7162700 10 monclient(hunting): authenticate will time out at 2714.787953s 2024-04-20T02:33:55.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f18a0138170 0x7f18a00a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a56ff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f18a01385b0 0x7f18a00a61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57730/0 (socket says 172.21.15.47:57730) 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a56ff700 1 -- 172.21.15.47:0/2478509217 learned_addr learned my addr 172.21.15.47:0/2478509217 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): get_auth_request con 0x7f18a0138170 auth_method 0 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a4efe700 1 --2- 172.21.15.47:0/2478509217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18a0137d90 0x7f18a00a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a4efe700 10 monclient(hunting): get_auth_request con 0x7f18a0137d90 auth_method 0 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a4efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a4efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a4efe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 1 -- 172.21.15.47:0/2478509217 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226479727 0 0) 0x7f18a00751e0 con 0x7f18a01385b0 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient(hunting): handle_auth_done global_id 36305 payload 274 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 -- 172.21.15.47:0/2478509217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18a0137d90 msgr2=0x7f18a00a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 --2- 172.21.15.47:0/2478509217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18a0137d90 0x7f18a00a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 -- 172.21.15.47:0/2478509217 >> v1:172.21.15.47:6789/0 conn(0x7f18a01385b0 legacy=0x7f18a00a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 monclient: found mon.noname-e 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 -- 172.21.15.47:0/2478509217 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00a7bc0 con 0x7f18a0138170 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189ffff700 1 --2- 172.21.15.47:0/2478509217 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f18a0138170 0x7f18a00a5c90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c009b80 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b37802abca118f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 1 -- 172.21.15.47:0/2478509217 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f188c011aa0 con 0x7f18a0138170 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:33:55.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: dump: 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.402 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 1 -- 172.21.15.47:0/2478509217 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c011c00 con 0x7f18a0138170 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a7162700 5 monclient: authenticate success, global_id 36305 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a7162700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a7162700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 1 -- 172.21.15.47:0/2478509217 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f188c011d70 con 0x7f18a0138170 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f18a7162700 4 set_mon_vals no callback set 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:55.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.387+0000 7f189f7fe700 10 monclient: dump: 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.404 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 10 monclient: shutdown 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 -- 172.21.15.47:0/2478509217 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f18a0138170 msgr2=0x7f18a00a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 --2- 172.21.15.47:0/2478509217 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f18a0138170 0x7f18a00a5c90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f188c009870 tx=0x7f188c009b80 comp rx=0 tx=0).stop 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 -- 172.21.15.47:0/2478509217 shutdown_connections 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 --2- 172.21.15.47:0/2478509217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18a0137d90 0x7f18a00a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 --2- 172.21.15.47:0/2478509217 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f18a0138170 0x7f18a00a5c90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 -- 172.21.15.47:0/2478509217 >> 172.21.15.47:0/2478509217 conn(0x7f18a00aa8b0 msgr2=0x7f18a00a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 -- 172.21.15.47:0/2478509217 shutdown_connections 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.388+0000 7f18a7162700 1 -- 172.21.15.47:0/2478509217 wait complete. 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 10 monclient: monmap: 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:55.405 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.406 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 1 librados: starting msgr at 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 1 librados: starting objecter 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 1 Processor -- start 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 1 -- start start 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 1 librados: setting wanted keys 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 1 librados: calling monclient init 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.389+0000 7f18a7162700 10 monclient: init 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient: _add_conns ranks=[7,2,5,6,3,4,8,1,0] 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f18a0137d90 0x7f18a01cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): picked mon.noname-i con 0x7f18a0137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18a0138170 0x7f18a0136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): picked mon.noname-b con 0x7f18a0138170 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 0x7f18a0137250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): picked mon.noname-h con 0x7f18a01385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0122ad0 con 0x7f18a0138170 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18a009e9e0 con 0x7f18a01385b0 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f18a009dc20 con 0x7f18a0137d90 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a7162700 10 monclient(hunting): authenticate will time out at 2714.791271s 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f189ffff700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18a0138170 msgr2=0x7f18a0136d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f189ffff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18a0138170 0x7f18a0136d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a4efe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f18a0137d90 msgr2=0x7f18a01cbe10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a4efe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f18a0137d90 0x7f18a01cbe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:55.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 0x7f18a0137250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 0x7f18a0137250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:47024/0 (socket says 172.21.15.47:47024) 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 -- 172.21.15.47:0/2545642730 learned_addr learned my addr 172.21.15.47:0/2545642730 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): get_auth_request con 0x7f18a01385b0 auth_method 0 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient(hunting): handle_auth_done global_id 84632 payload 824 2024-04-20T02:33:55.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f18a0137d90 msgr2=0x7f18a01cbe10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f18a0137d90 0x7f18a01cbe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18a0138170 msgr2=0x7f18a0136d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f18a0138170 0x7f18a0136d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 monclient: found mon.noname-h 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.390+0000 7f18a56ff700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0094a0 con 0x7f18a01385b0 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a56ff700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 0x7f18a0137250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f189800cb30 tx=0x7f189800cef0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 client.?.objecter ms_handle_connect 0x7f18a01385b0 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f189800adc0 con 0x7f18a01385b0 2024-04-20T02:33:55.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: dump: 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.411 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18980070d0 con 0x7f18a01385b0 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f18980076b0 con 0x7f18a01385b0 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189d7fa700 10 monclient: dump: 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.412 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 5 monclient: authenticate success, global_id 84632 2024-04-20T02:33:55.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 10 monclient: _renew_subs 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f189effd700 4 set_mon_vals no callback set 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a01cc4c0 con 0x7f18a01385b0 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 10 client.84632.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 10 monclient: _renew_subs 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.179:3302/0 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a0123c00 con 0x7f18a01385b0 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.391+0000 7f18a7162700 1 librados: init done 2024-04-20T02:33:55.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a7162700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a7162700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 0x7f18a01208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0068180 con 0x7f18a006aac0 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 0x7f18a01208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.5 v2:172.21.15.179:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1898007810 con 0x7f18a01385b0 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient: get_auth_request con 0x7f18a006aac0 auth_method 0 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f188403a320 0x7f188403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 10 monclient(hunting): handle_auth_done global_id 55491 payload 824 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.5 v2:172.21.15.179:3302/0 5 ==== osd_map(689..689 src has 266..689) v4 ==== 6661+0+0 (secure 0 0 0) 0x7f189804a2a0 con 0x7f18a01385b0 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 10 client.84632.objecter ms_dispatch 0x7f18a0001550 osd_map(689..689 src has 266..689) v4 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 3 client.84632.objecter handle_osd_map got epochs [689,689] > 0 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 3 client.84632.objecter handle_osd_map decoding full epoch 689 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18a009f760 con 0x7f18a006aac0 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189ffff700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f188403a320 0x7f188403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 20 client.84632.objecter dump_active .. 0 homeless 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189ffff700 10 monclient: get_auth_request con 0x7f188403a320 auth_method 0 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 10 client.84632.objecter ms_handle_connect 0x7f18a006aac0 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f189d7fa700 10 client.84632.objecter resend_mon_ops 2024-04-20T02:33:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.392+0000 7f18a4efe700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 0x7f18a01208d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f189400e740 tx=0x7f189400eb00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f189400c070 con 0x7f18a006aac0 2024-04-20T02:33:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 10 monclient: dump: 2024-04-20T02:33:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.417 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.418 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 10 client.84632.objecter ms_handle_connect 0x7f188403a320 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189ffff700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f188403a320 0x7f188403c7e0 secure :-1 s=READY pgs=3389 cs=0 l=1 rev1=1 crypto rx=0x7f18a00a4500 tx=0x7f188c008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1894004030 con 0x7f18a006aac0 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 msgr2=0x7f18a01208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 0x7f18a01208d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f189400e740 tx=0x7f189400eb00 comp rx=0 tx=0).stop 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 msgr2=0x7f18a01208d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.393+0000 7f189d7fa700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a006aac0 0x7f18a01208d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a7162700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a7162700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 0x7f18a0121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a7162700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f18a006a400 con 0x7f18a0120e10 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 0x7f18a0121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient: get_auth_request con 0x7f18a0120e10 auth_method 0 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 10 monclient(hunting): handle_auth_done global_id 55500 payload 824 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 1 -- 172.21.15.47:0/2545642730 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f18a0124120 con 0x7f18a0120e10 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f18a4efe700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 0x7f18a0121220 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f18940047c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f189d7fa700 10 client.84632.objecter ms_handle_connect 0x7f18a0120e10 2024-04-20T02:33:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f189d7fa700 10 client.84632.objecter resend_mon_ops 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1894022070 con 0x7f18a0120e10 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f189d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f189d7fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.401+0000 7f189d7fa700 10 monclient: dump: 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.423 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9162 (secure 0 0 0) 0x7f1894003d00 con 0x7f18a0120e10 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 msgr2=0x7f18a0121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 0x7f18a0121220 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f18940047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 msgr2=0x7f18a0121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f189d7fa700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f18a0120e10 0x7f18a0121220 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f18940047c0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:55.425 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.426 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:55.428 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.429 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:55.430 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.432 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:55.433 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.434 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:55.436 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.437 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:55.439 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.440 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:55.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.443 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.445 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:55.446 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 10 librados: watch_flush enter 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 10 librados: watch_flush exit 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 20 client.84632.objecter shutdown clearing up homeless session... 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 10 client.84632.objecter successfully canceled tick 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f188403a320 msgr2=0x7f188403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f188403a320 0x7f188403c7e0 secure :-1 s=READY pgs=3389 cs=0 l=1 rev1=1 crypto rx=0x7f18a00a4500 tx=0x7f188c008030 comp rx=0 tx=0).stop 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 10 monclient: shutdown 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 msgr2=0x7f18a0137250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a56ff700 1 -- 172.21.15.47:0/2545642730 reap_dead start 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 0x7f18a0137250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f189800cb30 tx=0x7f189800cef0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 shutdown_connections 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 --2- 172.21.15.47:0/2545642730 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f18a01385b0 0x7f18a0137250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 >> 172.21.15.47:0/2545642730 conn(0x7f18a00aa8b0 msgr2=0x7f18a01bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 shutdown_connections 2024-04-20T02:33:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 -- 172.21.15.47:0/2545642730 wait complete. 2024-04-20T02:33:55.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.402+0000 7f18a7162700 1 librados: shutdown 2024-04-20T02:33:55.450 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:55.450 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-04-20T02:33:55.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:33:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.589+0000 7feba3e5b700 10 monclient: get_monmap_and_config 2024-04-20T02:33:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.589+0000 7feba3e5b700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.589+0000 7feba3e5b700 10 monclient: monmap: 2024-04-20T02:33:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.601 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:55.590617+0000 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:55.590617+0000 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.602 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:55.603 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 Processor -- start 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 -- start start 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient: init 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient: _add_conns ranks=[6,1,4,8,9,5,3,2,0,10,7] 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 0x7feb9c0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient(hunting): picked mon.noname-d con 0x7feb9c137d90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7feb9c138170 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 0x7feb9c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient(hunting): picked mon.noname-c con 0x7feb9c1385b0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9c0751e0 con 0x7feb9c138170 2024-04-20T02:33:55.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c09fa40 con 0x7feb9c1385b0 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.590+0000 7feba3e5b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c09dfc0 con 0x7feb9c137d90 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba3e5b700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba3e5b700 10 monclient(hunting): authenticate will time out at 2714.992018s 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 0x7feb9c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 0x7feb9c0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 0x7feb9c0a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36302/0 (socket says 172.21.15.47:36302) 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 -- 172.21.15.47:0/121252590 learned_addr learned my addr 172.21.15.47:0/121252590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 0x7feb9c0a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48840/0 (socket says 172.21.15.47:48840) 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): get_auth_request con 0x7feb9c1385b0 auth_method 0 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): get_auth_request con 0x7feb9c137d90 auth_method 0 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient(hunting): handle_auth_done global_id 84694 payload 274 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 -- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 msgr2=0x7feb9c0a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 --2- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 0x7feb9c0a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 -- 172.21.15.47:0/121252590 >> v1:172.21.15.179:6789/0 conn(0x7feb9c138170 legacy=0x7feb9c0a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 monclient: found mon.noname-c 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 -- 172.21.15.47:0/121252590 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c0a7bc0 con 0x7feb9c1385b0 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba1bf7700 1 --2- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 0x7feb9c0a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba23f8700 1 --2- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 0x7feb9c0a63a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb94009e80 tx=0x7feb9400b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a25c697538b00e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 1 -- 172.21.15.47:0/121252590 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb9400e070 con 0x7feb9c1385b0 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: dump: 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.607 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.608 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 1 -- 172.21.15.47:0/121252590 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9400bd80 con 0x7feb9c1385b0 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba3e5b700 5 monclient: authenticate success, global_id 84694 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba3e5b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba3e5b700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 1 -- 172.21.15.47:0/121252590 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb94012860 con 0x7feb9c1385b0 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba3e5b700 4 set_mon_vals no callback set 2024-04-20T02:33:55.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.591+0000 7feba0bf5700 10 monclient: dump: 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.610 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 10 monclient: shutdown 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 -- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 msgr2=0x7feb9c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 --2- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 0x7feb9c0a63a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb94009e80 tx=0x7feb9400b330 comp rx=0 tx=0).stop 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 -- 172.21.15.47:0/121252590 shutdown_connections 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 --2- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c1385b0 0x7feb9c0a63a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 --2- 172.21.15.47:0/121252590 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c137d90 0x7feb9c0a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 -- 172.21.15.47:0/121252590 >> 172.21.15.47:0/121252590 conn(0x7feb9c0aa8b0 msgr2=0x7feb9c0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 -- 172.21.15.47:0/121252590 shutdown_connections 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.592+0000 7feba3e5b700 1 -- 172.21.15.47:0/121252590 wait complete. 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.593+0000 7feba3e5b700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.593+0000 7feba3e5b700 10 monclient: monmap: 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.593+0000 7feba3e5b700 1 librados: starting msgr at 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.593+0000 7feba3e5b700 1 librados: starting objecter 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 Processor -- start 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 -- start start 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 librados: setting wanted keys 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 librados: calling monclient init 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient: init 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient: _add_conns ranks=[4,6,5,2,0,8,7,3,1] 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 0x7feb9c1cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): picked mon.noname-c con 0x7feb9c137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c138170 0x7feb9c136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): picked mon.noname-d con 0x7feb9c138170 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7feb9c1385b0 0x7feb9c137380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): picked mon.noname-h con 0x7feb9c1385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c124080 con 0x7feb9c137d90 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c1ccf20 con 0x7feb9c1385b0 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c09ea70 con 0x7feb9c138170 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba3e5b700 10 monclient(hunting): authenticate will time out at 2714.995870s 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 0x7feb9c1cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba13f6700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c138170 0x7feb9c136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 0x7feb9c1cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36314/0 (socket says 172.21.15.47:36314) 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 1 -- 172.21.15.47:0/4017089750 learned_addr learned my addr 172.21.15.47:0/4017089750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 10 monclient(hunting): get_auth_request con 0x7feb9c137d90 auth_method 0 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.594+0000 7feba1bf7700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): get_auth_request con 0x7feb9c138170 auth_method 0 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba23f8700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7feb9c1385b0 0x7feb9c137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba23f8700 10 monclient(hunting): get_auth_request con 0x7feb9c1385b0 auth_method 0 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba23f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba23f8700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba23f8700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 10 monclient(hunting): handle_auth_done global_id 84703 payload 824 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c138170 msgr2=0x7feb9c136e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7feb9c138170 0x7feb9c136e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7feb9c1385b0 msgr2=0x7feb9c137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7feb9c1385b0 0x7feb9c137380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 monclient: found mon.noname-c 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb980094a0 con 0x7feb9c137d90 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba23f8700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7feb9c1385b0 0x7feb9c137380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba1bf7700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 0x7feb9c1cc260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb9800ec10 tx=0x7feb9800ef20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 client.?.objecter ms_handle_connect 0x7feb9c137d90 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb9800cd80 con 0x7feb9c137d90 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: dump: 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98013470 con 0x7feb9c137d90 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 5 monclient: authenticate success, global_id 84703 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 10 monclient: _renew_subs 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb94009b30 con 0x7feb9c137d90 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb9800cee0 con 0x7feb9c137d90 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 10 client.84703.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb93fff700 4 set_mon_vals no callback set 2024-04-20T02:33:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feb927fc700 10 monclient: dump: 2024-04-20T02:33:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 10 monclient: _renew_subs 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c09dc20 con 0x7feb9c137d90 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.595+0000 7feba3e5b700 1 librados: init done 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7feb9800c910 con 0x7feb9c137d90 2024-04-20T02:33:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba3e5b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba3e5b700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 0x7feb9c1210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c068180 con 0x7feb9c06aac0 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feb927fc700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb8003a320 0x7feb8003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 0x7feb9c1210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 10 monclient: get_auth_request con 0x7feb9c06aac0 auth_method 0 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(689..689 src has 266..689) v4 ==== 6661+0+0 (secure 0 0 0) 0x7feb9804ac20 con 0x7feb9c137d90 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feb927fc700 10 client.84703.objecter ms_dispatch 0x7feb9c001550 osd_map(689..689 src has 266..689) v4 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feb927fc700 3 client.84703.objecter handle_osd_map got epochs [689,689] > 0 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feb927fc700 3 client.84703.objecter handle_osd_map decoding full epoch 689 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.596+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba23f8700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb8003a320 0x7feb8003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 20 client.84703.objecter dump_active .. 0 homeless 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba23f8700 10 monclient: get_auth_request con 0x7feb8003a320 auth_method 0 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba13f6700 10 monclient(hunting): handle_auth_done global_id 36314 payload 824 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba13f6700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb9c09f760 con 0x7feb9c06aac0 2024-04-20T02:33:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 client.84703.objecter ms_handle_connect 0x7feb9c06aac0 2024-04-20T02:33:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba13f6700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 0x7feb9c1210a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb9c067990 tx=0x7feb8c00fb00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 client.84703.objecter resend_mon_ops 2024-04-20T02:33:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb8c0052e0 con 0x7feb9c06aac0 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 monclient: dump: 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 client.84703.objecter ms_handle_connect 0x7feb8003a320 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feba23f8700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb8003a320 0x7feb8003c7e0 secure :-1 s=READY pgs=3390 cs=0 l=1 rev1=1 crypto rx=0x7feb94009e80 tx=0x7feb94009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7feb8c005440 con 0x7feb9c06aac0 2024-04-20T02:33:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 msgr2=0x7feb9c1210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 0x7feb9c1210a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb9c067990 tx=0x7feb8c00fb00 comp rx=0 tx=0).stop 2024-04-20T02:33:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 msgr2=0x7feb9c1210a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.597+0000 7feb927fc700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c06aac0 0x7feb9c1210a0 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb9c067990 tx=0x7feb8c00fb00 comp rx=0 tx=0).stop 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba3e5b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba3e5b700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 0x7feb9c02bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba3e5b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c06a400 con 0x7feb9c02bbc0 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 0x7feb9c02bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 10 monclient: get_auth_request con 0x7feb9c02bbc0 auth_method 0 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.605+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feba13f6700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feba13f6700 10 monclient(hunting): handle_auth_done global_id 36323 payload 824 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feba13f6700 1 -- 172.21.15.47:0/4017089750 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feb9c137920 con 0x7feb9c02bbc0 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 client.84703.objecter ms_handle_connect 0x7feb9c02bbc0 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 client.84703.objecter resend_mon_ops 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feba13f6700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 0x7feb9c02bfd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0a46c0 tx=0x7feb8c001120 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feb8c023070 con 0x7feb9c02bbc0 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 monclient: dump: 2024-04-20T02:33:55.631 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.632 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7feb8c003c80 con 0x7feb9c02bbc0 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 msgr2=0x7feb9c02bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 0x7feb9c02bfd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0a46c0 tx=0x7feb8c001120 comp rx=0 tx=0).stop 2024-04-20T02:33:55.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 msgr2=0x7feb9c02bfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.606+0000 7feb927fc700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7feb9c02bbc0 0x7feb9c02bfd0 secure :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0a46c0 tx=0x7feb8c001120 comp rx=0 tx=0).stop 2024-04-20T02:33:55.634 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:55.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:55.634 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:55.634 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:55.635 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.636 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:55.637 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.638 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:55.639 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.640 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.642 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:55.643 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.644 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.645 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:55.647 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.649 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:55.650 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.651 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:55.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:55.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.655 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.656 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:33:55.657 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.657 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.657 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:55.657 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 10 librados: watch_flush enter 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 10 librados: watch_flush exit 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 20 client.84703.objecter shutdown clearing up homeless session... 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 10 client.84703.objecter successfully canceled tick 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb8003a320 msgr2=0x7feb8003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7feb8003a320 0x7feb8003c7e0 secure :-1 s=READY pgs=3390 cs=0 l=1 rev1=1 crypto rx=0x7feb94009e80 tx=0x7feb94009000 comp rx=0 tx=0).stop 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 10 monclient: shutdown 2024-04-20T02:33:55.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 msgr2=0x7feb9c1cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 0x7feb9c1cc260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb9800ec10 tx=0x7feb9800ef20 comp rx=0 tx=0).stop 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba23f8700 1 -- 172.21.15.47:0/4017089750 reap_dead start 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 shutdown_connections 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 --2- 172.21.15.47:0/4017089750 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb9c137d90 0x7feb9c1cc260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 >> 172.21.15.47:0/4017089750 conn(0x7feb9c0aa8b0 msgr2=0x7feb9c0a98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 shutdown_connections 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 -- 172.21.15.47:0/4017089750 wait complete. 2024-04-20T02:33:55.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.607+0000 7feba3e5b700 1 librados: shutdown 2024-04-20T02:33:55.661 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:55.661 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-04-20T02:33:55.661 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:33:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.800+0000 7fc6e8a25700 10 monclient: get_monmap_and_config 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.800+0000 7fc6e8a25700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.800+0000 7fc6e8a25700 10 monclient: monmap: 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:55.801915+0000 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:55.801915+0000 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:55.802 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.801+0000 7fc6e8a25700 1 Processor -- start 2024-04-20T02:33:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.801+0000 7fc6e8a25700 1 -- start start 2024-04-20T02:33:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.801+0000 7fc6e8a25700 10 monclient: init 2024-04-20T02:33:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient: _add_conns ranks=[1,6,3,9,4,0,10,7,8,2,5] 2024-04-20T02:33:55.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc6e40a0830 addr v1:172.21.15.179:6789/0 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 0x7fc6e413cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): picked mon.noname-d con 0x7fc6e40a01b0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6e413d280 0x7fc6e413f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): picked mon.noname-g con 0x7fc6e413d280 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e40751e0 con 0x7fc6e40a0830 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e409dee0 con 0x7fc6e413d280 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4067820 con 0x7fc6e40a01b0 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e8a25700 10 monclient(hunting): authenticate will time out at 2715.203237s 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 0x7fc6e413cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 0x7fc6e413cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:48854/0 (socket says 172.21.15.47:48854) 2024-04-20T02:33:55.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2392979858 learned_addr learned my addr 172.21.15.47:0/2392979858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): get_auth_request con 0x7fc6e40a01b0 auth_method 0 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e2d9d700 1 --2- 172.21.15.47:0/2392979858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6e413d280 0x7fc6e413f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_done global_id 55527 payload 274 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2392979858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6e413d280 msgr2=0x7fc6e413f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 --2- 172.21.15.47:0/2392979858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6e413d280 0x7fc6e413f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2392979858 >> v1:172.21.15.179:6789/0 conn(0x7fc6e40a0830 legacy=0x7fc6e40a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 monclient: found mon.noname-d 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2392979858 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e409eee0 con 0x7fc6e40a01b0 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e1d9b700 1 --2- 172.21.15.47:0/2392979858 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 0x7fc6e413cd40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009920 tx=0x7fc6d8009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=300dbbcceb13eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.802+0000 7fc6e159a700 1 -- 172.21.15.47:0/2392979858 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d8009e70 con 0x7fc6e40a01b0 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: dump: 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 1 -- 172.21.15.47:0/2392979858 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d80044a0 con 0x7fc6e40a01b0 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 5 monclient: authenticate success, global_id 55527 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 1 -- 172.21.15.47:0/2392979858 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d800bc90 con 0x7fc6e40a01b0 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e159a700 10 monclient: dump: 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 4 set_mon_vals no callback set 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 10 monclient: shutdown 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2392979858 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 msgr2=0x7fc6e413cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2392979858 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 0x7fc6e413cd40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009920 tx=0x7fc6d8009c30 comp rx=0 tx=0).stop 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2392979858 shutdown_connections 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2392979858 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc6e413d280 0x7fc6e413f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2392979858 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fc6e40a01b0 0x7fc6e413cd40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2392979858 >> 172.21.15.47:0/2392979858 conn(0x7fc6e40a7570 msgr2=0x7fc6e40a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2392979858 shutdown_connections 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.803+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2392979858 wait complete. 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.804+0000 7fc6e8a25700 10 monclient: build_initial_monmap 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.804+0000 7fc6e8a25700 10 monclient: monmap: 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.804+0000 7fc6e8a25700 1 librados: starting msgr at 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.804+0000 7fc6e8a25700 1 librados: starting objecter 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 Processor -- start 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 -- start start 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 librados: setting wanted keys 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 librados: calling monclient init 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient: init 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient: _add_conns ranks=[7,1,2,8,0,5,4,3,6] 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e40a01b0 0x7fc6e41cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): picked mon.noname-i con 0x7fc6e40a01b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:55.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e40a0830 0x7fc6e4138d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): picked mon.noname-f con 0x7fc6e40a0830 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc6e413d280 0x7fc6e4139260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): picked mon.noname-b con 0x7fc6e413d280 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4124e30 con 0x7fc6e40a0830 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4123d00 con 0x7fc6e413d280 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e409e900 con 0x7fc6e40a01b0 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e8a25700 10 monclient(hunting): authenticate will time out at 2715.206716s 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e259c700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e40a01b0 msgr2=0x7fc6e41cc700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e2d9d700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc6e413d280 msgr2=0x7fc6e4139260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e259c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e40a01b0 0x7fc6e41cc700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e2d9d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc6e413d280 0x7fc6e4139260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e40a0830 0x7fc6e4138d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e40a0830 0x7fc6e4138d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:59860/0 (socket says 172.21.15.47:59860) 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2215860280 learned_addr learned my addr 172.21.15.47:0/2215860280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 10 monclient(hunting): get_auth_request con 0x7fc6e40a0830 auth_method 0 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.805+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 10 monclient(hunting): handle_auth_done global_id 55945 payload 824 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e40a01b0 msgr2=0x7fc6e41cc700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fc6e40a01b0 0x7fc6e41cc700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc6e413d280 msgr2=0x7fc6e4139260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:55.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc6e413d280 0x7fc6e4139260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 monclient: found mon.noname-f 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8009550 con 0x7fc6e40a0830 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e1d9b700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e40a0830 0x7fc6e4138d20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009840 tx=0x7fc6d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 client.?.objecter ms_handle_connect 0x7fc6e40a0830 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d8009e70 con 0x7fc6e40a0830 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: dump: 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.824 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:55.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8004330 con 0x7fc6e40a0830 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d801b440 con 0x7fc6e40a0830 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 5 monclient: authenticate success, global_id 55945 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 10 monclient: _renew_subs 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e0d99700 4 set_mon_vals no callback set 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e4124190 con 0x7fc6e40a0830 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6d2ffd700 10 monclient: dump: 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.826 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 10 client.55945.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:55.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 10 monclient: _renew_subs 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e4123320 con 0x7fc6e40a0830 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.806+0000 7fc6e8a25700 1 librados: init done 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e8a25700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 0x7fc6e41219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4068310 con 0x7fc6e406aac0 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 0x7fc6e41219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient: get_auth_request con 0x7fc6e406aac0 auth_method 0 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc6d8021020 con 0x7fc6e40a0830 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.807+0000 7fc6e259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6e259c700 10 monclient(hunting): handle_auth_done global_id 55954 payload 824 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c403a2f0 0x7fc6c403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6e259c700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6e40672f0 con 0x7fc6e406aac0 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(689..689 src has 266..689) v4 ==== 6661+0+0 (secure 0 0 0) 0x7fc6d804ba00 con 0x7fc6e40a0830 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 client.55945.objecter ms_dispatch 0x7fc6e4001550 osd_map(689..689 src has 266..689) v4 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 3 client.55945.objecter handle_osd_map got epochs [689,689] > 0 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 3 client.55945.objecter handle_osd_map decoding full epoch 689 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 20 client.55945.objecter dump_active .. 0 homeless 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 client.55945.objecter ms_handle_connect 0x7fc6e406aac0 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 client.55945.objecter resend_mon_ops 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6e259c700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 0x7fc6e41219a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40037b0 tx=0x7fc6d400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6e2d9d700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c403a2f0 0x7fc6c403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6e2d9d700 10 monclient: get_auth_request con 0x7fc6c403a2f0 auth_method 0 2024-04-20T02:33:55.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d400ce60 con 0x7fc6e406aac0 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 monclient: dump: 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.830 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc6d40072e0 con 0x7fc6e406aac0 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 msgr2=0x7fc6e41219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 0x7fc6e41219a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40037b0 tx=0x7fc6d400c730 comp rx=0 tx=0).stop 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 msgr2=0x7fc6e41219a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e406aac0 0x7fc6e41219a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6e2d9d700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c403a2f0 0x7fc6c403c7b0 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40a07b0 tx=0x7fc6dc007270 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.808+0000 7fc6d2ffd700 10 client.55945.objecter ms_handle_connect 0x7fc6c403a2f0 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e8a25700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 0x7fc6e41222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e8a25700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e406a400 con 0x7fc6e4121ee0 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e259c700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 0x7fc6e41222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e259c700 10 monclient: get_auth_request con 0x7fc6e4121ee0 auth_method 0 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e259c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.816+0000 7fc6e259c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6e259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6e259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:55.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6e259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6e259c700 10 monclient(hunting): handle_auth_done global_id 55963 payload 824 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6e259c700 1 -- 172.21.15.47:0/2215860280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc6e409f1c0 con 0x7fc6e4121ee0 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6e259c700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 0x7fc6e41222f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40037b0 tx=0x7fc6d4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 client.55945.objecter ms_handle_connect 0x7fc6e4121ee0 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 client.55945.objecter resend_mon_ops 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6d401f070 con 0x7fc6e4121ee0 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 monclient: dump: 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:55.836 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9334 (secure 0 0 0) 0x7fc6d401a2e0 con 0x7fc6e4121ee0 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 msgr2=0x7fc6e41222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 0x7fc6e41222f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40037b0 tx=0x7fc6d4003820 comp rx=0 tx=0).stop 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 msgr2=0x7fc6e41222f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.817+0000 7fc6d2ffd700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e4121ee0 0x7fc6e41222f0 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40037b0 tx=0x7fc6d4003820 comp rx=0 tx=0).stop 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:55.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:55.838 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.839 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.840 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:55.841 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:55.842 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:55.843 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:55.844 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.845 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.847 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:55.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.849 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.850 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.851 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.852 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.853 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:55.854 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.855 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:55.856 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.858 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:55.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:55.860 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:55.860 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 10 librados: watch_flush enter 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 10 librados: watch_flush exit 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 20 client.55945.objecter shutdown clearing up homeless session... 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 10 client.55945.objecter successfully canceled tick 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c403a2f0 msgr2=0x7fc6c403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fc6c403a2f0 0x7fc6c403c7b0 secure :-1 s=READY pgs=3391 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40a07b0 tx=0x7fc6dc007270 comp rx=0 tx=0).stop 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 10 monclient: shutdown 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e40a0830 msgr2=0x7fc6e4138d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 --2- 172.21.15.47:0/2215860280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6e40a0830 0x7fc6e4138d20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009840 tx=0x7fc6d8004750 comp rx=0 tx=0).stop 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e2d9d700 1 -- 172.21.15.47:0/2215860280 reap_dead start 2024-04-20T02:33:55.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e2d9d700 1 -- 172.21.15.47:0/2215860280 reap_dead start 2024-04-20T02:33:55.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 shutdown_connections 2024-04-20T02:33:55.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 >> 172.21.15.47:0/2215860280 conn(0x7fc6e40a7570 msgr2=0x7fc6e409a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:55.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 shutdown_connections 2024-04-20T02:33:55.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 -- 172.21.15.47:0/2215860280 wait complete. 2024-04-20T02:33:55.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:55.818+0000 7fc6e8a25700 1 librados: shutdown 2024-04-20T02:33:55.863 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:55.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-04-20T02:33:55.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.014+0000 7f37012bb700 10 monclient: get_monmap_and_config 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.014+0000 7f37012bb700 10 monclient: build_initial_monmap 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.014+0000 7f37012bb700 10 monclient: monmap: 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:56.016036+0000 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:56.016036+0000 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:56.015 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:56.016 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.015+0000 7f37012bb700 1 Processor -- start 2024-04-20T02:33:56.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.015+0000 7f37012bb700 1 -- start start 2024-04-20T02:33:56.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.015+0000 7f37012bb700 10 monclient: init 2024-04-20T02:33:56.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient: _add_conns ranks=[4,7,3,0,2,9,6,5,10,1,8] 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 0x7f36fc0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): picked mon.noname-c con 0x7f36fc137d90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc138170 0x7f36fc0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): picked mon.noname-i con 0x7f36fc138170 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1385b0 0x7f36fc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): picked mon.noname-g con 0x7f36fc1385b0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc09fa40 con 0x7f36fc1385b0 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc09dfc0 con 0x7f36fc137d90 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0a7bc0 con 0x7f36fc138170 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f37012bb700 10 monclient(hunting): authenticate will time out at 2715.417161s 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 0x7f36fc0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fa7fc700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc138170 msgr2=0x7f36fc0a5c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fa7fc700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc138170 0x7f36fc0a5c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 0x7f36fc0a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36324/0 (socket says 172.21.15.47:36324) 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 -- 172.21.15.47:0/3356649990 learned_addr learned my addr 172.21.15.47:0/3356649990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): get_auth_request con 0x7f36fc137d90 auth_method 0 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fb7fe700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1385b0 0x7f36fc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fb7fe700 10 monclient(hunting): get_auth_request con 0x7f36fc1385b0 auth_method 0 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient(hunting): handle_auth_done global_id 84712 payload 274 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 -- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc138170 msgr2=0x7f36fc0a5c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc138170 0x7f36fc0a5c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 -- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1385b0 msgr2=0x7f36fc0a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1385b0 0x7f36fc0a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient: _finish_hunting 0 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 monclient: found mon.noname-c 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 -- 172.21.15.47:0/3356649990 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc0751e0 con 0x7f36fc137d90 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36fb7fe700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1385b0 0x7f36fc0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36faffd700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 0x7f36fc0a09e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009950 tx=0x7f36ec009c60 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ddf60a92908bbba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.016+0000 7f36f9ffb700 1 -- 172.21.15.47:0/3356649990 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36ec009ea0 con 0x7f36fc137d90 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: dump: 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.031 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.032 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: _finish_auth 0 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 1 -- 172.21.15.47:0/3356649990 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec00bc90 con 0x7f36fc137d90 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 5 monclient: authenticate success, global_id 84712 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:56.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 1 -- 172.21.15.47:0/3356649990 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36ec00be00 con 0x7f36fc137d90 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 4 set_mon_vals no callback set 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f36f9ffb700 10 monclient: dump: 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.035 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 10 monclient: shutdown 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 -- 172.21.15.47:0/3356649990 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 msgr2=0x7f36fc0a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 0x7f36fc0a09e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009950 tx=0x7f36ec009c60 comp rx=0 tx=0).stop 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 -- 172.21.15.47:0/3356649990 shutdown_connections 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1385b0 0x7f36fc0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36fc137d90 0x7f36fc0a09e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 --2- 172.21.15.47:0/3356649990 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc138170 0x7f36fc0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 -- 172.21.15.47:0/3356649990 >> 172.21.15.47:0/3356649990 conn(0x7f36fc0aa8b0 msgr2=0x7f36fc0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 -- 172.21.15.47:0/3356649990 shutdown_connections 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.017+0000 7f37012bb700 1 -- 172.21.15.47:0/3356649990 wait complete. 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.018+0000 7f37012bb700 10 monclient: build_initial_monmap 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.018+0000 7f37012bb700 10 monclient: monmap: 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:56.037 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.018+0000 7f37012bb700 1 librados: starting msgr at 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.018+0000 7f37012bb700 1 librados: starting objecter 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.019+0000 7f37012bb700 1 Processor -- start 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.019+0000 7f37012bb700 1 -- start start 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.019+0000 7f37012bb700 1 librados: setting wanted keys 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.019+0000 7f37012bb700 1 librados: calling monclient init 2024-04-20T02:33:56.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.019+0000 7f37012bb700 10 monclient: init 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient: _add_conns ranks=[0,1,7,2,8,5,4,6,3] 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36fc137d90 0x7f36fc1c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): picked mon.noname-a con 0x7f36fc137d90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36fc138170 0x7f36fc136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): picked mon.noname-f con 0x7f36fc138170 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc1385b0 0x7f36fc137520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): picked mon.noname-i con 0x7f36fc1385b0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc1247c0 con 0x7f36fc137d90 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc1c99d0 con 0x7f36fc138170 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc09dc20 con 0x7f36fc1385b0 2024-04-20T02:33:56.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f37012bb700 10 monclient(hunting): authenticate will time out at 2715.421226s 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36fc137d90 0x7f36fc1c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36fb7fe700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc1385b0 msgr2=0x7f36fc137520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36fc137d90 0x7f36fc1c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49556/0 (socket says 172.21.15.47:49556) 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 -- 172.21.15.47:0/2507152318 learned_addr learned my addr 172.21.15.47:0/2507152318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): get_auth_request con 0x7f36fc137d90 auth_method 0 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36fb7fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc1385b0 0x7f36fc137520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36fa7fc700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36fc138170 0x7f36fc136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient(hunting): handle_auth_done global_id 104301 payload 824 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc1385b0 msgr2=0x7f36fc137520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f36fc1385b0 0x7f36fc137520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36fc138170 msgr2=0x7f36fc136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36fc138170 0x7f36fc136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient: _finish_hunting 0 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 monclient: found mon.noname-a 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec009550 con 0x7f36fc137d90 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36fa7fc700 10 monclient: get_auth_request con 0x7f36fc138170 auth_method 0 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36fa7fc700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36fc138170 0x7f36fc136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:33:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36faffd700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36fc137d90 0x7f36fc1c8da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f36fc09e100 tx=0x7f36ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36f37fe700 10 client.?.objecter ms_handle_connect 0x7f36fc137d90 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36f37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36ec004350 con 0x7f36fc137d90 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36f37fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.020+0000 7f36f37fe700 10 monclient: dump: 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.042 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 10 monclient: _finish_auth 0 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec0044f0 con 0x7f36fc137d90 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 5 monclient: authenticate success, global_id 104301 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 monclient: _renew_subs 2024-04-20T02:33:56.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc122640 con 0x7f36fc137d90 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36ec024470 con 0x7f36fc137d90 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f37fe700 10 monclient: dump: 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.044 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f36f97fa700 4 set_mon_vals no callback set 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 client.104301.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 monclient: _renew_subs 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc1243e0 con 0x7f36fc137d90 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 1 librados: init done 2024-04-20T02:33:56.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 0x7f36fc1cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.021+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc068180 con 0x7f36fc06aaa0 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f36ec021070 con 0x7f36fc137d90 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f36dc03a2f0 0x7f36dc03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 0x7f36fc1cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient: get_auth_request con 0x7f36fc06aaa0 auth_method 0 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(690..690 src has 266..690) v4 ==== 6674+0+0 (secure 0 0 0) 0x7f36ec04bbc0 con 0x7f36fc137d90 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 10 client.104301.objecter ms_dispatch 0x7f36fc001550 osd_map(690..690 src has 266..690) v4 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 3 client.104301.objecter handle_osd_map got epochs [690,690] > 0 2024-04-20T02:33:56.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 3 client.104301.objecter handle_osd_map decoding full epoch 690 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 20 client.104301.objecter dump_active .. 0 homeless 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fb7fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f36dc03a2f0 0x7f36dc03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fb7fe700 10 monclient: get_auth_request con 0x7f36dc03a2f0 auth_method 0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_done global_id 104277 payload 824 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36fc09f760 con 0x7f36fc06aaa0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 10 client.104301.objecter ms_handle_connect 0x7f36fc06aaa0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 10 client.104301.objecter resend_mon_ops 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fa7fc700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 0x7f36fc1cc2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36fc120b40 tx=0x7f36e400ca80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36f37fe700 10 client.104301.objecter ms_handle_connect 0x7f36dc03a2f0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.022+0000 7f36fb7fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f36dc03a2f0 0x7f36dc03c7b0 secure :-1 s=READY pgs=3392 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009540 tx=0x7f36f4009430 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36e4003c40 con 0x7f36fc06aaa0 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 10 monclient: dump: 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f36e4003da0 con 0x7f36fc06aaa0 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 msgr2=0x7f36fc1cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 0x7f36fc1cc2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36fc120b40 tx=0x7f36e400ca80 comp rx=0 tx=0).stop 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 msgr2=0x7f36fc1cc2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.023+0000 7f36f37fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc06aaa0 0x7f36fc1cc2e0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36fc120b40 tx=0x7f36e400ca80 comp rx=0 tx=0).stop 2024-04-20T02:33:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f37012bb700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f37012bb700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 0x7f36fc1ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f37012bb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc06b9b0 con 0x7f36fc1cc820 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 0x7f36fc1ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient: get_auth_request con 0x7f36fc1cc820 auth_method 0 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 10 monclient(hunting): handle_auth_done global_id 104286 payload 824 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.031+0000 7f36fa7fc700 1 -- 172.21.15.47:0/2507152318 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f36fc1cd170 con 0x7f36fc1cc820 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 client.104301.objecter ms_handle_connect 0x7f36fc1cc820 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 client.104301.objecter resend_mon_ops 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36fa7fc700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 0x7f36fc1ccc30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36fc06aec0 tx=0x7f36e4005a50 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36e40058a0 con 0x7f36fc1cc820 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 monclient: dump: 2024-04-20T02:33:56.053 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9162 (secure 0 0 0) 0x7f36e4011410 con 0x7f36fc1cc820 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 msgr2=0x7f36fc1ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 0x7f36fc1ccc30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36fc06aec0 tx=0x7f36e4005a50 comp rx=0 tx=0).stop 2024-04-20T02:33:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 msgr2=0x7f36fc1ccc30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:56.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36f37fe700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36fc1cc820 0x7f36fc1ccc30 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36fc06aec0 tx=0x7f36e4005a50 comp rx=0 tx=0).stop 2024-04-20T02:33:56.056 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:56.056 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:56.057 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:56.058 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.059 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:56.060 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:56.061 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:56.062 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.063 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:56.064 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.065 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.066 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.068 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.069 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.070 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.072 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.073 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:56.075 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.076 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.078 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:56.079 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.080 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:56.080 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 10 librados: watch_flush enter 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 10 librados: watch_flush exit 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 20 client.104301.objecter shutdown clearing up homeless session... 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 10 client.104301.objecter successfully canceled tick 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f36dc03a2f0 msgr2=0x7f36dc03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f36dc03a2f0 0x7f36dc03c7b0 secure :-1 s=READY pgs=3392 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009540 tx=0x7f36f4009430 comp rx=0 tx=0).stop 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 10 monclient: shutdown 2024-04-20T02:33:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36fc137d90 msgr2=0x7f36fc1c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 --2- 172.21.15.47:0/2507152318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36fc137d90 0x7f36fc1c8da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f36fc09e100 tx=0x7f36ec004750 comp rx=0 tx=0).stop 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36fb7fe700 1 -- 172.21.15.47:0/2507152318 reap_dead start 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f36fb7fe700 1 -- 172.21.15.47:0/2507152318 reap_dead start 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 shutdown_connections 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 >> 172.21.15.47:0/2507152318 conn(0x7f36fc0aa8b0 msgr2=0x7f36fc1bf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.032+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 shutdown_connections 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.033+0000 7f37012bb700 1 -- 172.21.15.47:0/2507152318 wait complete. 2024-04-20T02:33:56.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.033+0000 7f37012bb700 1 librados: shutdown 2024-04-20T02:33:56.083 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:56.083 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-04-20T02:33:56.084 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:33:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.224+0000 7fbbc4b68700 10 monclient: get_monmap_and_config 2024-04-20T02:33:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.224+0000 7fbbc4b68700 10 monclient: build_initial_monmap 2024-04-20T02:33:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.224+0000 7fbbc4b68700 10 monclient: monmap: 2024-04-20T02:33:56.224 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:33:56.225691+0000 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:33:56.225691+0000 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:33:56.225 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:33:56.226 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.225+0000 7fbbc4b68700 1 Processor -- start 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.225+0000 7fbbc4b68700 1 -- start start 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.225+0000 7fbbc4b68700 10 monclient: init 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient: _add_conns ranks=[2,9,0,8,7,3,4,10,5,6,1] 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbc0071710 0x7fbbc009e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): picked mon.noname-b con 0x7fbbc0071710 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbc006f090 0x7fbbc0071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): picked mon.noname-a con 0x7fbbc006f090 addr v2:172.21.15.47:3300/0 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbbc0072030 addr v1:172.21.15.47:6789/0 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00751e0 con 0x7fbbc0072030 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc009ee80 con 0x7fbbc006f090 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00a0680 con 0x7fbbc0071710 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbc4b68700 10 monclient(hunting): authenticate will time out at 2715.627389s 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbe59c700 1 -- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbc0071710 msgr2=0x7fbbc009e940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.47:3301/0 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbe59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbc0071710 0x7fbbc009e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbc006f090 0x7fbbc0071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbed9d700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbbc0072030 0x7fbbc00a2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57740/0 (socket says 172.21.15.47:57740) 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbed9d700 1 -- 172.21.15.47:0/2790054048 learned_addr learned my addr 172.21.15.47:0/2790054048 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbbc006f090 0x7fbbc0071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49572/0 (socket says 172.21.15.47:49572) 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): get_auth_request con 0x7fbbc006f090 auth_method 0 2024-04-20T02:33:56.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_done global_id 104310 payload 274 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 -- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbc0071710 msgr2=0x7fbbc009e940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 --2- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbc0071710 0x7fbbc009e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 -- 172.21.15.47:0/2790054048 >> v1:172.21.15.47:6789/0 conn(0x7fbbc0072030 legacy=0x7fbbc00a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient: _finish_hunting 0 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 monclient: found mon.noname-a 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbdd9b700 1 -- 172.21.15.47:0/2790054048 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00a0030 con 0x7fbbc006f090 2024-04-20T02:33:56.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbd59a700 1 -- 172.21.15.47:0/2790054048 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977925926 0 0) 0x7fbbc00751e0 con 0x7fbbc0072030 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.226+0000 7fbbbd59a700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbdd9b700 1 --2- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbc006f090 0x7fbbc0071af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400bb10 tx=0x7fbbb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25611362fafbed42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 1 -- 172.21.15.47:0/2790054048 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbb4010470 con 0x7fbbc006f090 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: dump: 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.239 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: _finish_auth 0 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: _check_auth_tickets 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 1 -- 172.21.15.47:0/2790054048 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb40105d0 con 0x7fbbc006f090 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 5 monclient: authenticate success, global_id 104310 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:56.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 1 -- 172.21.15.47:0/2790054048 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbb4010bb0 con 0x7fbbc006f090 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 10 monclient: get_monmap_and_config success 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 4 set_mon_vals no callback set 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbbd59a700 10 monclient: dump: 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.241 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 10 monclient: shutdown 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 -- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbc006f090 msgr2=0x7fbbc0071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbc006f090 0x7fbbc0071af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400bb10 tx=0x7fbbb400be20 comp rx=0 tx=0).stop 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 -- 172.21.15.47:0/2790054048 shutdown_connections 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbbc0071710 0x7fbbc009e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/2790054048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbc006f090 0x7fbbc0071af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 -- 172.21.15.47:0/2790054048 >> 172.21.15.47:0/2790054048 conn(0x7fbbc00a7570 msgr2=0x7fbbc00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 -- 172.21.15.47:0/2790054048 shutdown_connections 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.227+0000 7fbbc4b68700 1 -- 172.21.15.47:0/2790054048 wait complete. 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.228+0000 7fbbc4b68700 10 monclient: build_initial_monmap 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.228+0000 7fbbc4b68700 10 monclient: monmap: 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:33:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 librados: starting msgr at 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 librados: starting objecter 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 Processor -- start 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 -- start start 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 librados: setting wanted keys 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 librados: calling monclient init 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient: init 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient: _reopen_session rank -1 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient: _add_conns ranks=[3,5,7,2,1,8,4,0,6] 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbc006f090 0x7fbbc01c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): picked mon.noname-g con 0x7fbbc006f090 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:33:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc0071710 0x7fbbc0133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): picked mon.noname-h con 0x7fbbc0071710 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbbc0072030 0x7fbbc0134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): picked mon.noname-i con 0x7fbbc0072030 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc01213e0 con 0x7fbbc006f090 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc011f3f0 con 0x7fbbc0071710 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00724a0 con 0x7fbbc0072030 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): _renew_subs 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbc4b68700 10 monclient(hunting): authenticate will time out at 2715.630715s 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbbed9d700 1 -- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbbc0072030 msgr2=0x7fbbc0134220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.179:3303/0 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbbed9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbbc0072030 0x7fbbc0134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:33:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbbe59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbc006f090 0x7fbbc01c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.229+0000 7fbbbdd9b700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc0071710 0x7fbbc0133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbc006f090 0x7fbbc01c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:41612/0 (socket says 172.21.15.47:41612) 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 -- 172.21.15.47:0/3369269047 learned_addr learned my addr 172.21.15.47:0/3369269047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): get_auth_request con 0x7fbbc006f090 auth_method 0 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): get_auth_request con 0x7fbbc0071710 auth_method 0 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient(hunting): handle_auth_done global_id 104295 payload 824 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbbc0072030 msgr2=0x7fbbc0134220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fbbc0072030 0x7fbbc0134220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc0071710 msgr2=0x7fbbc0133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc0071710 0x7fbbc0133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient: _finish_hunting 0 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 monclient: found mon.noname-g 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:33:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb400b740 con 0x7fbbc006f090 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbe59c700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbc006f090 0x7fbbc01c7d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b550 tx=0x7fbbb000b860 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 client.?.objecter ms_handle_connect 0x7fbbc006f090 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbb0003c50 con 0x7fbbc006f090 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: dump: 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: _finish_auth 0 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: _check_auth_tickets 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 5 monclient: authenticate success, global_id 104295 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0003df0 con 0x7fbbc006f090 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 10 monclient: _renew_subs 2024-04-20T02:33:56.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc01c8800 con 0x7fbbc006f090 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbb00103a0 con 0x7fbbc006f090 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbbcd99700 4 set_mon_vals no callback set 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 10 client.104295.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbaeffd700 10 monclient: dump: 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 10 monclient: _renew_subs 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc0121100 con 0x7fbbc006f090 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.230+0000 7fbbc4b68700 1 librados: init done 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbc4b68700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 0x7fbbc01c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0068310 con 0x7fbbc006aac0 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbbb0005070 con 0x7fbbc006f090 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 0x7fbbc01c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient: get_auth_request con 0x7fbbc006aac0 auth_method 0 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbaeffd700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbba003a2f0 0x7fbba003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(690..690 src has 266..690) v4 ==== 6674+0+0 (secure 0 0 0) 0x7fbbb0049900 con 0x7fbbc006f090 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbaeffd700 10 client.104295.objecter ms_dispatch 0x7fbbc0001550 osd_map(690..690 src has 266..690) v4 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbaeffd700 3 client.104295.objecter handle_osd_map got epochs [690,690] > 0 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.231+0000 7fbbaeffd700 3 client.104295.objecter handle_osd_map decoding full epoch 690 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 20 client.104295.objecter dump_active .. 0 homeless 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbbed9d700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbba003a2f0 0x7fbba003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_done global_id 84659 payload 824 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbbdd9b700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbc01350d0 con 0x7fbbc006aac0 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbbed9d700 10 monclient: get_auth_request con 0x7fbba003a2f0 auth_method 0 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 client.104295.objecter ms_handle_connect 0x7fbbc006aac0 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 client.104295.objecter resend_mon_ops 2024-04-20T02:33:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbbdd9b700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 0x7fbbc01c4ff0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00a7f00 tx=0x7fbbb4004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbb401e040 con 0x7fbbc006aac0 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbbed9d700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbba003a2f0 0x7fbba003c7b0 secure :-1 s=READY pgs=3393 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8001460 tx=0x7fbbb80016c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 monclient: dump: 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.255 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 client.104295.objecter ms_handle_connect 0x7fbba003a2f0 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbbb4024640 con 0x7fbbc006aac0 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 msgr2=0x7fbbc01c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 0x7fbbc01c4ff0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00a7f00 tx=0x7fbbb4004750 comp rx=0 tx=0).stop 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 msgr2=0x7fbbc01c4ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.232+0000 7fbbaeffd700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc006aac0 0x7fbbc01c4ff0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00a7f00 tx=0x7fbbb4004750 comp rx=0 tx=0).stop 2024-04-20T02:33:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.240+0000 7fbbc4b68700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.240+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 0x7fbbc002bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.240+0000 7fbbc4b68700 10 monclient(hunting): start opening mon connection 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.240+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc006a400 con 0x7fbbc01c8260 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 0x7fbbc002bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient: get_auth_request con 0x7fbbc01c8260 auth_method 0 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 10 monclient(hunting): handle_auth_done global_id 84668 payload 824 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 1 -- 172.21.15.47:0/3369269047 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbbc0070c10 con 0x7fbbc01c8260 2024-04-20T02:33:56.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbbdd9b700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 0x7fbbc002bbc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400b2c0 tx=0x7fbbb400b420 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbaeffd700 10 client.104295.objecter ms_handle_connect 0x7fbbc01c8260 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbaeffd700 10 client.104295.objecter resend_mon_ops 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbb4004540 con 0x7fbbc01c8260 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbaeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbaeffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.241+0000 7fbbaeffd700 10 monclient: dump: 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:33:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9502 (secure 0 0 0) 0x7fbbb4034070 con 0x7fbbc01c8260 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 msgr2=0x7fbbc002bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 0x7fbbc002bbc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400b2c0 tx=0x7fbbb400b420 comp rx=0 tx=0).stop 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 msgr2=0x7fbbc002bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:33:56.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbaeffd700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fbbc01c8260 0x7fbbc002bbc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 116, 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:33:56.262 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:56.263 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:56.264 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:33:56.265 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:33:56.266 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.267 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:33:56.268 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:33:56.269 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:33:56.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.272 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.273 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:33:56.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.275 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.277 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:33:56.279 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.281 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:33:56.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:33:56.283 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:33:56.283 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 10 librados: watch_flush enter 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 10 librados: watch_flush exit 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 20 client.104295.objecter shutdown clearing up homeless session... 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 10 client.104295.objecter successfully canceled tick 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbba003a2f0 msgr2=0x7fbba003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fbba003a2f0 0x7fbba003c7b0 secure :-1 s=READY pgs=3393 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8001460 tx=0x7fbbb80016c0 comp rx=0 tx=0).stop 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 10 monclient: shutdown 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbc006f090 msgr2=0x7fbbc01c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 --2- 172.21.15.47:0/3369269047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbbc006f090 0x7fbbc01c7d20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000b550 tx=0x7fbbb000b860 comp rx=0 tx=0).stop 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbbed9d700 1 -- 172.21.15.47:0/3369269047 reap_dead start 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbbed9d700 1 -- 172.21.15.47:0/3369269047 reap_dead start 2024-04-20T02:33:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 shutdown_connections 2024-04-20T02:33:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 >> 172.21.15.47:0/3369269047 conn(0x7fbbc00a7570 msgr2=0x7fbbc00a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:33:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 shutdown_connections 2024-04-20T02:33:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 -- 172.21.15.47:0/3369269047 wait complete. 2024-04-20T02:33:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:33:56.242+0000 7fbbc4b68700 1 librados: shutdown 2024-04-20T02:33:56.286 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-04-20T02:33:56.286 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-04-20T02:34:02.474 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (8062 ms) 2024-04-20T02:34:02.474 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-04-20T02:34:02.474 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (11915 ms) 2024-04-20T02:34:02.474 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-04-20T02:34:02.474 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (19467 ms) 2024-04-20T02:34:02.474 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (30801 ms) 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7990 ms) 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8095 ms) 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: hmm, no HitSet yet 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: ok, hit_set contains 252:602f83fe:::foo:head 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (8943 ms) 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-04-20T02:34:02.475 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: first is 1713580333 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,1713580338,1713580339,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,1713580338,1713580339,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,1713580338,1713580339,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,1713580338,1713580339,1713580341,1713580342,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,1713580338,1713580339,1713580341,1713580342,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580333,1713580335,1713580336,1713580338,1713580339,1713580341,1713580342,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: got ls 1713580336,1713580338,1713580339,1713580341,1713580342,1713580344,1713580345,0 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: first now 1713580336, trimmed 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20191 ms) 2024-04-20T02:34:02.476 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: foo0 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (13837 ms) 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17178 ms) 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (24935 ms) 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-04-20T02:34:02.477 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5270 ms) 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (2581 ms) 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (27443 ms) 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (2565 ms) 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (302320 ms total) 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (1040344 ms total) 2024-04-20T02:34:02.478 INFO:tasks.workunit.client.0.smithi179.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-04-20T02:34:02.480 INFO:tasks.workunit.client.0.smithi179.stderr:+ exit 0 2024-04-20T02:34:02.480 INFO:tasks.workunit.client.0.smithi179.stderr:+ cleanup 2024-04-20T02:34:02.480 INFO:tasks.workunit.client.0.smithi179.stderr:+ pkill -P 46793 2024-04-20T02:34:02.499 INFO:tasks.workunit.client.0.smithi179.stderr:+ true 2024-04-20T02:34:02.499 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T02:34:02.500 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T02:34:02.562 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2024-04-20T02:34:02.562 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T02:34:02.896 DEBUG:teuthology.parallel:result is None 2024-04-20T02:34:02.896 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T02:34:02.927 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T02:34:02.927 DEBUG:teuthology.orchestra.run.smithi179:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-20T02:34:02.993 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T02:34:02.993 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-04-20T02:34:03.039 INFO:tasks.mon_thrash:joining mon_thrasher 2024-04-20T02:35:26.287 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-20T02:35:26.287 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-20T02:35:26.287 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T02:35:26.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T02:35:26.291 INFO:tasks.ceph.mon.b:Started 2024-04-20T02:35:26.291 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-04-20T02:35:26.292 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-04-20T02:35:26.292 INFO:tasks.ceph.mon.i:Restarting daemon 2024-04-20T02:35:26.292 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-04-20T02:35:26.295 INFO:tasks.ceph.mon.i:Started 2024-04-20T02:35:26.296 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-04-20T02:35:26.296 INFO:tasks.ceph.mon.h:Sent signal 19 2024-04-20T02:35:26.296 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-04-20T02:35:26.414 INFO:tasks.ceph.mon.i.smithi179.stderr:2024-04-20T02:35:26.415+0000 7fc5657bbcc0 -1 mon.i@-1(???).log v0 log_external_backlog rewinding external_log_to from 719 -> 0 (sync_force? mon rebuild?) 2024-04-20T02:35:41.297 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-04-20T02:35:41.301 INFO:tasks.ceph.mon.h:Sent signal 18 2024-04-20T02:35:41.301 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:35:41.301 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:35:41.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.456+0000 7fb04f460700 10 monclient: get_monmap_and_config 2024-04-20T02:35:41.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.456+0000 7fb04f460700 10 monclient: build_initial_monmap 2024-04-20T02:35:41.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.456+0000 7fb04f460700 10 monclient: monmap: 2024-04-20T02:35:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:41.457667+0000 2024-04-20T02:35:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:41.457667+0000 2024-04-20T02:35:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:41.473 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:41.474 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:41.475 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:41.475 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 1 Processor -- start 2024-04-20T02:35:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 1 -- start start 2024-04-20T02:35:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 10 monclient: init 2024-04-20T02:35:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 10 monclient: _add_conns ranks=[3,0,9,8,5,4,6,10,7,1,2] 2024-04-20T02:35:41.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0480a4c80 0x7fb0480a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 10 monclient(hunting): picked mon.noname-g con 0x7fb0480a4c80 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb0480a0d20 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb0480a5770 0x7fb04813ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.457+0000 7fb04f460700 10 monclient(hunting): picked mon.noname-a con 0x7fb0480a5770 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0480751e0 con 0x7fb0480a0d20 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0480a0000 con 0x7fb0480a5770 2024-04-20T02:35:41.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480a6a10 con 0x7fb0480a4c80 2024-04-20T02:35:41.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:41.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 10 monclient(hunting): authenticate will time out at 2820.859052s 2024-04-20T02:35:41.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb0480a5770 0x7fb04813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04c9fb700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb0480a0d20 0x7fb0480a5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47842/0 (socket says 172.21.15.47:47842) 2024-04-20T02:35:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04c9fb700 1 -- 172.21.15.47:0/944964250 learned_addr learned my addr 172.21.15.47:0/944964250 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): get_auth_request con 0x7fb0480a5770 auth_method 0 2024-04-20T02:35:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 1 --2- 172.21.15.47:0/944964250 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0480a4c80 0x7fb0480a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): get_auth_request con 0x7fb0480a4c80 auth_method 0 2024-04-20T02:35:41.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d1fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient(hunting): handle_auth_done global_id 104328 payload 274 2024-04-20T02:35:41.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 -- 172.21.15.47:0/944964250 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0480a4c80 msgr2=0x7fb0480a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:41.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 --2- 172.21.15.47:0/944964250 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0480a4c80 0x7fb0480a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 -- 172.21.15.47:0/944964250 >> v1:172.21.15.47:6789/0 conn(0x7fb0480a0d20 legacy=0x7fb0480a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:41.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 monclient: found mon.noname-a 2024-04-20T02:35:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 -- 172.21.15.47:0/944964250 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04809e440 con 0x7fb0480a5770 2024-04-20T02:35:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 1 -- 172.21.15.47:0/944964250 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388557609 0 0) 0x7fb0480751e0 con 0x7fb0480a0d20 2024-04-20T02:35:41.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:35:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04d9fd700 1 --2- 172.21.15.47:0/944964250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0480a5770 0x7fb04813ef10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb040009d90 tx=0x7fb04000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4e68aa5a3c7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 1 -- 172.21.15.47:0/944964250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb04000e070 con 0x7fb0480a5770 2024-04-20T02:35:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: dump: 2024-04-20T02:35:41.484 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.485 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.485 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.485 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.485 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.485 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.485 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.486 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.487 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: _finish_auth 0 2024-04-20T02:35:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:41.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 5 monclient: authenticate success, global_id 104328 2024-04-20T02:35:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 1 -- 172.21.15.47:0/944964250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0400098f0 con 0x7fb0480a5770 2024-04-20T02:35:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:41.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 1 -- 172.21.15.47:0/944964250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0400128a0 con 0x7fb0480a5770 2024-04-20T02:35:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb04f460700 4 set_mon_vals no callback set 2024-04-20T02:35:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.458+0000 7fb03ffff700 10 monclient: dump: 2024-04-20T02:35:41.489 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.490 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.491 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 10 monclient: shutdown 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 -- 172.21.15.47:0/944964250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0480a5770 msgr2=0x7fb04813ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 --2- 172.21.15.47:0/944964250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0480a5770 0x7fb04813ef10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb040009d90 tx=0x7fb04000b330 comp rx=0 tx=0).stop 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 -- 172.21.15.47:0/944964250 shutdown_connections 2024-04-20T02:35:41.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 --2- 172.21.15.47:0/944964250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb0480a5770 0x7fb04813ef10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 --2- 172.21.15.47:0/944964250 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb0480a4c80 0x7fb0480a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 -- 172.21.15.47:0/944964250 >> 172.21.15.47:0/944964250 conn(0x7fb0480aa280 msgr2=0x7fb0480aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 -- 172.21.15.47:0/944964250 shutdown_connections 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.459+0000 7fb04f460700 1 -- 172.21.15.47:0/944964250 wait complete. 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 10 monclient: build_initial_monmap 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 10 monclient: monmap: 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:41.493 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 1 librados: starting msgr at 2024-04-20T02:35:41.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 1 librados: starting objecter 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 1 Processor -- start 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 1 -- start start 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 1 librados: setting wanted keys 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 1 librados: calling monclient init 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.460+0000 7fb04f460700 10 monclient: init 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient: _add_conns ranks=[5,8,1,7,2,3,4,6,0] 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb0480a0d20 0x7fb0481ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): picked mon.noname-h con 0x7fb0480a0d20 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 0x7fb04813ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): picked mon.noname-e con 0x7fb0480a4c80 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0480a5770 0x7fb04813b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): picked mon.noname-f con 0x7fb0480a5770 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04813e3a0 con 0x7fb0480a5770 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb04809ea00 con 0x7fb0480a0d20 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb04809dd20 con 0x7fb0480a4c80 2024-04-20T02:35:41.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04f460700 10 monclient(hunting): authenticate will time out at 2820.862209s 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 0x7fb04813ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 0x7fb04813ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:49628/0 (socket says 172.21.15.47:49628) 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 -- 172.21.15.47:0/536780929 learned_addr learned my addr 172.21.15.47:0/536780929 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): get_auth_request con 0x7fb0480a4c80 auth_method 0 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d1fc700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb0480a0d20 0x7fb0481ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d1fc700 10 monclient(hunting): get_auth_request con 0x7fb0480a0d20 auth_method 0 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d1fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d9fd700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0480a5770 0x7fb04813b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d1fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d1fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d9fd700 10 monclient(hunting): get_auth_request con 0x7fb0480a5770 auth_method 0 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d9fd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d9fd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d9fd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient(hunting): handle_auth_done global_id 36332 payload 824 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 -- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb0480a0d20 msgr2=0x7fb0481ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb0480a0d20 0x7fb0481ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 -- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0480a5770 msgr2=0x7fb04813b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0480a5770 0x7fb04813b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient: _finish_hunting 0 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 monclient: found mon.noname-e 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 -- 172.21.15.47:0/536780929 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009a70 con 0x7fb0480a4c80 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d1fc700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb0480a0d20 0x7fb0481ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04d9fd700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0480a5770 0x7fb04813b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb04c9fb700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 0x7fb04813ace0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb038001eb0 tx=0x7fb03800c8b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb03dffb700 10 client.?.objecter ms_handle_connect 0x7fb0480a4c80 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb03dffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0380070d0 con 0x7fb0480a4c80 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.461+0000 7fb03dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: dump: 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.500 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: _finish_auth 0 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:41.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038007250 con 0x7fb0480a4c80 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 5 monclient: authenticate success, global_id 36332 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 10 monclient: _renew_subs 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0481361d0 con 0x7fb0480a4c80 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0380133c0 con 0x7fb0480a4c80 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 10 monclient: dump: 2024-04-20T02:35:41.502 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.503 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03f7fe700 4 set_mon_vals no callback set 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 10 client.36332.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 10 monclient: _renew_subs 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04813c870 con 0x7fb0480a4c80 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb04f460700 1 librados: init done 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.462+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb0380073c0 con 0x7fb0480a4c80 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb02c03a2f0 0x7fb02c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fb03804a140 con 0x7fb0480a4c80 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 10 client.36332.objecter ms_dispatch 0x7fb048001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 3 client.36332.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 3 client.36332.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 20 client.36332.objecter dump_active .. 0 homeless 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04f460700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04f460700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04f460700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04d1fc700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb02c03a2f0 0x7fb02c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04813e8a0 con 0x7fb0480a4c80 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04d1fc700 10 monclient: get_auth_request con 0x7fb02c03a2f0 auth_method 0 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb03dffb700 10 client.36332.objecter ms_handle_connect 0x7fb02c03a2f0 2024-04-20T02:35:41.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.463+0000 7fb04d1fc700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb02c03a2f0 0x7fb02c03c7b0 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 crypto rx=0x7fb044009c30 tx=0x7fb044008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.468+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fb04813e8a0 con 0x7fb0480a4c80 2024-04-20T02:35:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.468+0000 7fb03dffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:41.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.468+0000 7fb03dffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb04f460700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb04f460700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb04f460700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb04813e8a0 con 0x7fb0480a4c80 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb03dffb700 1 -- 172.21.15.47:0/536780929 <== mon.8 v2:172.21.15.47:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7fb04813e8a0 con 0x7fb0480a4c80 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb03dffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:35:41.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.776+0000 7fb03dffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:41.777 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":120,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 10 librados: watch_flush enter 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 10 librados: watch_flush exit 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 20 client.36332.objecter shutdown clearing up homeless session... 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 10 client.36332.objecter successfully canceled tick 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb02c03a2f0 msgr2=0x7fb02c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb02c03a2f0 0x7fb02c03c7b0 secure :-1 s=READY pgs=3397 cs=0 l=1 rev1=1 crypto rx=0x7fb044009c30 tx=0x7fb044008040 comp rx=0 tx=0).stop 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 10 monclient: shutdown 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 msgr2=0x7fb04813ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 0x7fb04813ace0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb038001eb0 tx=0x7fb03800c8b0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 shutdown_connections 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb02c03a2f0 0x7fb02c03c7b0 unknown :-1 s=CLOSED pgs=3397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0480a5770 0x7fb04813b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fb0480a0d20 0x7fb0481ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 --2- 172.21.15.47:0/536780929 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fb0480a4c80 0x7fb04813ace0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 >> 172.21.15.47:0/536780929 conn(0x7fb0480aa280 msgr2=0x7fb048139640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.778+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 shutdown_connections 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.779+0000 7fb04f460700 1 -- 172.21.15.47:0/536780929 wait complete. 2024-04-20T02:35:41.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.779+0000 7fb04f460700 1 librados: shutdown 2024-04-20T02:35:41.792 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:35:41.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-04-20T02:35:41.793 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.930+0000 7f363b97b700 10 monclient: get_monmap_and_config 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.930+0000 7f363b97b700 10 monclient: build_initial_monmap 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.930+0000 7f363b97b700 10 monclient: monmap: 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:41.931629+0000 2024-04-20T02:35:41.930 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:41.931629+0000 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:41.931 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 Processor -- start 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 -- start start 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient: init 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient: _add_conns ranks=[7,10,4,6,9,1,0,2,3,5,8] 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3634071630 0x7f3634071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): picked mon.noname-i con 0x7f3634071630 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3634071f50 0x7f36340a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): picked mon.noname-f con 0x7f3634071f50 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 0x7f36340a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): picked mon.noname-c con 0x7f36340a2e90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:41.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3634072720 con 0x7f3634071f50 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36340a0820 con 0x7f36340a2e90 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f363409e890 con 0x7f3634071630 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.931+0000 7f363b97b700 10 monclient(hunting): authenticate will time out at 2821.332876s 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 0x7f36340a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 0x7f36340a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37652/0 (socket says 172.21.15.47:37652) 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 -- 172.21.15.47:0/2269471888 learned_addr learned my addr 172.21.15.47:0/2269471888 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): get_auth_request con 0x7f36340a2e90 auth_method 0 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639717700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3634071630 0x7f3634071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3638f16700 1 --2- 172.21.15.47:0/2269471888 >> v2:172.21.15.179:3300/0 conn(0x7f3634071f50 0x7f36340a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639717700 10 monclient(hunting): get_auth_request con 0x7f3634071630 auth_method 0 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639717700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639717700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639717700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3638f16700 10 monclient(hunting): get_auth_request con 0x7f3634071f50 auth_method 0 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3638f16700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3638f16700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3638f16700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient(hunting): handle_auth_done global_id 84721 payload 274 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 -- 172.21.15.47:0/2269471888 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3634071630 msgr2=0x7f3634071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3634071630 0x7f3634071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 -- 172.21.15.47:0/2269471888 >> v2:172.21.15.179:3300/0 conn(0x7f3634071f50 msgr2=0x7f36340a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 --2- 172.21.15.47:0/2269471888 >> v2:172.21.15.179:3300/0 conn(0x7f3634071f50 0x7f36340a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient: _finish_hunting 0 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 monclient: found mon.noname-c 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 -- 172.21.15.47:0/2269471888 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340751e0 con 0x7f36340a2e90 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639717700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3634071630 0x7f3634071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3638f16700 1 --2- 172.21.15.47:0/2269471888 >> v2:172.21.15.179:3300/0 conn(0x7f3634071f50 0x7f36340a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f3639f18700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 0x7f36340a3320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f362c009e80 tx=0x7f362c00b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=58010110f3d762c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 1 -- 172.21.15.47:0/2269471888 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f362c00e070 con 0x7f36340a2e90 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: dump: 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.945 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: _finish_auth 0 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 1 -- 172.21.15.47:0/2269471888 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c0099b0 con 0x7f36340a2e90 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f363b97b700 5 monclient: authenticate success, global_id 84721 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f363b97b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 1 -- 172.21.15.47:0/2269471888 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f362c00bdf0 con 0x7f36340a2e90 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f363b97b700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f363b97b700 4 set_mon_vals no callback set 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.932+0000 7f362bfff700 10 monclient: dump: 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.947 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 10 monclient: shutdown 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:41.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 -- 172.21.15.47:0/2269471888 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 msgr2=0x7f36340a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 0x7f36340a3320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f362c009e80 tx=0x7f362c00b330 comp rx=0 tx=0).stop 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 -- 172.21.15.47:0/2269471888 shutdown_connections 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f36340a2e90 0x7f36340a3320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 --2- 172.21.15.47:0/2269471888 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f3634071630 0x7f3634071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 --2- 172.21.15.47:0/2269471888 >> v2:172.21.15.179:3300/0 conn(0x7f3634071f50 0x7f36340a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 -- 172.21.15.47:0/2269471888 >> 172.21.15.47:0/2269471888 conn(0x7f36340a7570 msgr2=0x7f36340a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 -- 172.21.15.47:0/2269471888 shutdown_connections 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.933+0000 7f363b97b700 1 -- 172.21.15.47:0/2269471888 wait complete. 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 10 monclient: build_initial_monmap 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 10 monclient: monmap: 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:41.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 1 librados: starting msgr at 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 1 librados: starting objecter 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 1 Processor -- start 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 1 -- start start 2024-04-20T02:35:41.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 1 librados: setting wanted keys 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 1 librados: calling monclient init 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.934+0000 7f363b97b700 10 monclient: init 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient: _add_conns ranks=[6,8,3,4,0,5,7,1,2] 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3634071630 0x7f36341c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): picked mon.noname-d con 0x7f3634071630 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3634071f50 0x7f36341b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): picked mon.noname-e con 0x7f3634071f50 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36340a2e90 0x7f36341b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): picked mon.noname-g con 0x7f36340a2e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36341a5780 con 0x7f36340a2e90 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36341a4650 con 0x7f3634071630 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f363409f200 con 0x7f3634071f50 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f363b97b700 10 monclient(hunting): authenticate will time out at 2821.336160s 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3634071630 0x7f36341c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3634071f50 0x7f36341b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3634071f50 0x7f36341b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:49632/0 (socket says 172.21.15.47:49632) 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3634071630 0x7f36341c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:51430/0 (socket says 172.21.15.47:51430) 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 -- 172.21.15.47:0/4063472889 learned_addr learned my addr 172.21.15.47:0/4063472889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): get_auth_request con 0x7f3634071f50 auth_method 0 2024-04-20T02:35:41.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): get_auth_request con 0x7f3634071630 auth_method 0 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36340a2e90 0x7f36341b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): get_auth_request con 0x7f36340a2e90 auth_method 0 2024-04-20T02:35:41.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639f18700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient(hunting): handle_auth_done global_id 36341 payload 824 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3634071630 msgr2=0x7f36341c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3634071630 0x7f36341c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36340a2e90 msgr2=0x7f36341b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36340a2e90 0x7f36341b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient: _finish_hunting 0 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 monclient: found mon.noname-e 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c009b30 con 0x7f3634071f50 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3639717700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3634071630 0x7f36341c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3638f16700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3634071f50 0x7f36341b8890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3624009fd0 tx=0x7f362400eac0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3629ffb700 10 client.?.objecter ms_handle_connect 0x7f3634071f50 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3629ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.935+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3624013880 con 0x7f3634071f50 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: dump: 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: _finish_auth 0 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624013a00 con 0x7f3634071f50 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3624012370 con 0x7f3634071f50 2024-04-20T02:35:41.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 5 monclient: authenticate success, global_id 36341 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f362b7fe700 4 set_mon_vals no callback set 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f3629ffb700 10 monclient: dump: 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 10 monclient: _renew_subs 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3634072380 con 0x7f3634071f50 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 10 client.36341.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 10 monclient: _renew_subs 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36341a6020 con 0x7f3634071f50 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.936+0000 7f363b97b700 1 librados: init done 2024-04-20T02:35:41.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f363b97b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f363b97b700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 0x7f36341a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634067d30 con 0x7f363406aac0 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3624013410 con 0x7f3634071f50 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 0x7f36341a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient: get_auth_request con 0x7f363406aac0 auth_method 0 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f361803a320 0x7f361803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f3624049ee0 con 0x7f3634071f50 2024-04-20T02:35:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 10 client.36341.objecter ms_dispatch 0x7f3634001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 3 client.36341.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 3 client.36341.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 20 client.36341.objecter dump_active .. 0 homeless 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 10 monclient(hunting): handle_auth_done global_id 104346 payload 824 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f363406f090 con 0x7f363406aac0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639f18700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f361803a320 0x7f361803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639f18700 10 monclient: get_auth_request con 0x7f361803a320 auth_method 0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 10 client.36341.objecter ms_handle_connect 0x7f363406aac0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3629ffb700 10 client.36341.objecter resend_mon_ops 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.937+0000 7f3639717700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 0x7f36341a22a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3630011ab0 tx=0x7f3630011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36300043a0 con 0x7f363406aac0 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 10 monclient: dump: 2024-04-20T02:35:41.960 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.961 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 10 client.36341.objecter ms_handle_connect 0x7f361803a320 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3639f18700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f361803a320 0x7f361803c7e0 secure :-1 s=READY pgs=3398 cs=0 l=1 rev1=1 crypto rx=0x7f362c009e80 tx=0x7f362c009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3630004500 con 0x7f363406aac0 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 msgr2=0x7f36341a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 0x7f36341a22a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3630011ab0 tx=0x7f3630011e70 comp rx=0 tx=0).stop 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 msgr2=0x7f36341a22a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.938+0000 7f3629ffb700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363406aac0 0x7f36341a22a0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3630011ab0 tx=0x7f3630011e70 comp rx=0 tx=0).stop 2024-04-20T02:35:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f363b97b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f363b97b700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 0x7f36341a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:41.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f363b97b700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634068310 con 0x7f36341a27e0 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 0x7f36341a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient: get_auth_request con 0x7f36341a27e0 auth_method 0 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 10 monclient(hunting): handle_auth_done global_id 104355 payload 824 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 1 -- 172.21.15.47:0/4063472889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f36341a3130 con 0x7f36341a27e0 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3639717700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 0x7f36341a2bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3630000c00 tx=0x7f3630004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3629ffb700 10 client.36341.objecter ms_handle_connect 0x7f36341a27e0 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.946+0000 7f3629ffb700 10 client.36341.objecter resend_mon_ops 2024-04-20T02:35:41.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3630003b10 con 0x7f36341a27e0 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 10 monclient: dump: 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:41.966 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f3630003c70 con 0x7f36341a27e0 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 msgr2=0x7f36341a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 0x7f36341a2bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3630000c00 tx=0x7f3630004750 comp rx=0 tx=0).stop 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 msgr2=0x7f36341a2bf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3629ffb700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36341a27e0 0x7f36341a2bf0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3630000c00 tx=0x7f3630004750 comp rx=0 tx=0).stop 2024-04-20T02:35:41.967 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:41.968 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:41.969 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:41.970 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:41.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:41.972 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.973 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.974 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.975 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.976 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.977 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.978 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.979 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.980 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:41.981 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.983 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:41.984 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:41.985 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:41.986 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:41.987 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:41.988 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:41.988 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:41.988 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:41.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:41.988 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:41.988 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 10 librados: watch_flush enter 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 10 librados: watch_flush exit 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 20 client.36341.objecter shutdown clearing up homeless session... 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 10 client.36341.objecter successfully canceled tick 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f361803a320 msgr2=0x7f361803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f361803a320 0x7f361803c7e0 secure :-1 s=READY pgs=3398 cs=0 l=1 rev1=1 crypto rx=0x7f362c009e80 tx=0x7f362c009000 comp rx=0 tx=0).stop 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 10 monclient: shutdown 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3634071f50 msgr2=0x7f36341b8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 --2- 172.21.15.47:0/4063472889 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3634071f50 0x7f36341b8890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3624009fd0 tx=0x7f362400eac0 comp rx=0 tx=0).stop 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3639f18700 1 -- 172.21.15.47:0/4063472889 reap_dead start 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f3639f18700 1 -- 172.21.15.47:0/4063472889 reap_dead start 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 shutdown_connections 2024-04-20T02:35:41.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 >> 172.21.15.47:0/4063472889 conn(0x7f36340a7570 msgr2=0x7f36340a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:41.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 shutdown_connections 2024-04-20T02:35:41.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 -- 172.21.15.47:0/4063472889 wait complete. 2024-04-20T02:35:41.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:41.947+0000 7f363b97b700 1 librados: shutdown 2024-04-20T02:35:41.991 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:41.991 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-04-20T02:35:41.991 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-20T02:35:42.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.129+0000 7f719d1e0700 10 monclient: get_monmap_and_config 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.129+0000 7f719d1e0700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.129+0000 7f719d1e0700 10 monclient: monmap: 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:42.131003+0000 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:42.131003+0000 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.130 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:42.131 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:42.131 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.130+0000 7f719d1e0700 1 Processor -- start 2024-04-20T02:35:42.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.130+0000 7f719d1e0700 1 -- start start 2024-04-20T02:35:42.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.130+0000 7f719d1e0700 10 monclient: init 2024-04-20T02:35:42.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient: _add_conns ranks=[9,3,5,10,2,0,6,1,8,4,7] 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71980a18a0 0x7f7198135e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): picked mon.noname-a con 0x7f71980a18a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71980a1c80 0x7f71980a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): picked mon.noname-g con 0x7f71980a1c80 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f71980a2100 0x7f71980a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): picked mon.noname-h con 0x7f71980a2100 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7198135bd0 con 0x7f71980a18a0 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7198071eb0 con 0x7f71980a1c80 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7198072190 con 0x7f71980a2100 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient(hunting): authenticate will time out at 2821.532142s 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71980a18a0 0x7f7198135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71980a18a0 0x7f7198135e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57434/0 (socket says 172.21.15.47:57434) 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 -- 172.21.15.47:0/245040917 learned_addr learned my addr 172.21.15.47:0/245040917 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): get_auth_request con 0x7f71980a18a0 auth_method 0 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719659c700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71980a1c80 0x7f71980a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719759e700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f71980a2100 0x7f71980a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719659c700 10 monclient(hunting): get_auth_request con 0x7f71980a1c80 auth_method 0 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719759e700 10 monclient(hunting): get_auth_request con 0x7f71980a2100 auth_method 0 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient(hunting): handle_auth_done global_id 104364 payload 274 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 -- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f71980a2100 msgr2=0x7f71980a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f71980a2100 0x7f71980a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 -- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71980a1c80 msgr2=0x7f71980a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71980a1c80 0x7f71980a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 monclient: found mon.noname-a 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 -- 172.21.15.47:0/245040917 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980751e0 con 0x7f71980a18a0 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7196d9d700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71980a18a0 0x7f7198135e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71880098f0 tx=0x7f7188009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bb99f71007a849 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 1 -- 172.21.15.47:0/245040917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f718800bb10 con 0x7f71980a18a0 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: dump: 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.147 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 1 -- 172.21.15.47:0/245040917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718800bc70 con 0x7f71980a18a0 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 5 monclient: authenticate success, global_id 104364 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 1 -- 172.21.15.47:0/245040917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f718800bde0 con 0x7f71980a18a0 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f719d1e0700 4 set_mon_vals no callback set 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.131+0000 7f7195d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f7195d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f7195d9b700 10 monclient: dump: 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.149 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 10 monclient: shutdown 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 -- 172.21.15.47:0/245040917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71980a18a0 msgr2=0x7f7198135e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71980a18a0 0x7f7198135e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71880098f0 tx=0x7f7188009c00 comp rx=0 tx=0).stop 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 -- 172.21.15.47:0/245040917 shutdown_connections 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71980a18a0 0x7f7198135e20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f71980a2100 0x7f71980a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 --2- 172.21.15.47:0/245040917 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f71980a1c80 0x7f71980a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 -- 172.21.15.47:0/245040917 >> 172.21.15.47:0/245040917 conn(0x7f71980a50b0 msgr2=0x7f7198066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 -- 172.21.15.47:0/245040917 shutdown_connections 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.132+0000 7f719d1e0700 1 -- 172.21.15.47:0/245040917 wait complete. 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.133+0000 7f719d1e0700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.133+0000 7f719d1e0700 10 monclient: monmap: 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.151 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.133+0000 7f719d1e0700 1 librados: starting msgr at 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.133+0000 7f719d1e0700 1 librados: starting objecter 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.134+0000 7f719d1e0700 1 Processor -- start 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.134+0000 7f719d1e0700 1 -- start start 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.134+0000 7f719d1e0700 1 librados: setting wanted keys 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.134+0000 7f719d1e0700 1 librados: calling monclient init 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.134+0000 7f719d1e0700 10 monclient: init 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient: _add_conns ranks=[7,4,2,6,5,0,3,1,8] 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f71980a18a0 0x7f71981cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): picked mon.noname-i con 0x7f71980a18a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71980a1c80 0x7f7198131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): picked mon.noname-c con 0x7f71980a1c80 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 0x7f7198132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): picked mon.noname-b con 0x7f71980a2100 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71980714c0 con 0x7f71980a2100 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71981cd0f0 con 0x7f71980a1c80 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f719809ee50 con 0x7f71980a18a0 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719d1e0700 10 monclient(hunting): authenticate will time out at 2821.536250s 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 0x7f7198132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71980a1c80 0x7f7198131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 0x7f7198132100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48340/0 (socket says 172.21.15.47:48340) 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71980a1c80 0x7f7198131bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37668/0 (socket says 172.21.15.47:37668) 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 -- 172.21.15.47:0/992732056 learned_addr learned my addr 172.21.15.47:0/992732056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): get_auth_request con 0x7f71980a2100 auth_method 0 2024-04-20T02:35:42.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): get_auth_request con 0x7f71980a1c80 auth_method 0 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f71980a18a0 0x7f71981cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): get_auth_request con 0x7f71980a18a0 auth_method 0 2024-04-20T02:35:42.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient(hunting): handle_auth_done global_id 114104 payload 824 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f71980a18a0 msgr2=0x7f71981cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f71980a18a0 0x7f71981cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71980a1c80 msgr2=0x7f7198131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71980a1c80 0x7f7198131bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 monclient: found mon.noname-b 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188009550 con 0x7f71980a2100 2024-04-20T02:35:42.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719659c700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71980a1c80 0x7f7198131bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f719759e700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 0x7f7198132100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7190009fd0 tx=0x7f719000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.135+0000 7f717f7fe700 10 client.?.objecter ms_handle_connect 0x7f71980a2100 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7190007ea0 con 0x7f71980a2100 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: dump: 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.156 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7190014420 con 0x7f71980a2100 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 5 monclient: authenticate success, global_id 114104 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 10 monclient: _renew_subs 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7198124b10 con 0x7f71980a2100 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7190014a00 con 0x7f71980a2100 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f717f7fe700 10 monclient: dump: 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.158 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 10 client.114104.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 10 monclient: _renew_subs 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719559a700 4 set_mon_vals no callback set 2024-04-20T02:35:42.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719806e330 con 0x7f71980a2100 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.136+0000 7f719d1e0700 1 librados: init done 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f719d1e0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f719d1e0700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 0x7f719806c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7198068430 con 0x7f719806aac0 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 0x7f719806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7190014590 con 0x7f71980a2100 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient: get_auth_request con 0x7f719806aac0 auth_method 0 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 10 monclient(hunting): handle_auth_done global_id 114113 payload 824 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f718403a320 0x7f718403c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7198070bb0 con 0x7f719806aac0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f719004a930 con 0x7f71980a2100 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 10 client.114104.objecter ms_dispatch 0x7f7198001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 3 client.114104.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 3 client.114104.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f7196d9d700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 0x7f719806c760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71880098f0 tx=0x7f7188004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f719659c700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f718403a320 0x7f718403c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 20 client.114104.objecter dump_active .. 0 homeless 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 10 client.114104.objecter ms_handle_connect 0x7f719806aac0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 10 client.114104.objecter resend_mon_ops 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f719659c700 10 monclient: get_auth_request con 0x7f718403a320 auth_method 0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7188020070 con 0x7f719806aac0 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:42.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.137+0000 7f717f7fe700 10 monclient: dump: 2024-04-20T02:35:42.162 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.162 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.163 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7188004250 con 0x7f719806aac0 2024-04-20T02:35:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 msgr2=0x7f719806c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 0x7f719806c760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71880098f0 tx=0x7f7188004750 comp rx=0 tx=0).stop 2024-04-20T02:35:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 msgr2=0x7f719806c760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f719659c700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f718403a320 0x7f718403c7e0 secure :-1 s=READY pgs=3399 cs=0 l=1 rev1=1 crypto rx=0x7f719806df00 tx=0x7f718c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806aac0 0x7f719806c760 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71880098f0 tx=0x7f7188004750 comp rx=0 tx=0).stop 2024-04-20T02:35:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.138+0000 7f717f7fe700 10 client.114104.objecter ms_handle_connect 0x7f718403a320 2024-04-20T02:35:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.145+0000 7f719d1e0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.145+0000 7f719d1e0700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 0x7f719806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.145+0000 7f719d1e0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.145+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719806a400 con 0x7f719806cca0 2024-04-20T02:35:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 0x7f719806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient: get_auth_request con 0x7f719806cca0 auth_method 0 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 10 monclient(hunting): handle_auth_done global_id 114122 payload 824 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 1 -- 172.21.15.47:0/992732056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7198067150 con 0x7f719806cca0 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f7196d9d700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 0x7f719806d0b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7188009840 tx=0x7f7188011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 client.114104.objecter ms_handle_connect 0x7f719806cca0 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 client.114104.objecter resend_mon_ops 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7188003e30 con 0x7f719806cca0 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 monclient: dump: 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 <== mon.2 v2:172.21.15.47:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f718802b620 con 0x7f719806cca0 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 msgr2=0x7f719806d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 0x7f719806d0b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7188009840 tx=0x7f7188011000 comp rx=0 tx=0).stop 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 msgr2=0x7f719806d0b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.146+0000 7f717f7fe700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f719806cca0 0x7f719806d0b0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7188009840 tx=0x7f7188011000 comp rx=0 tx=0).stop 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:42.170 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 0, 2024-04-20T02:35:42.171 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:42.172 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:42.173 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.174 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:42.175 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.176 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.177 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.179 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.183 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.184 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:42.186 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.187 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.188 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.189 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.190 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:42.191 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 10 librados: watch_flush enter 2024-04-20T02:35:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 10 librados: watch_flush exit 2024-04-20T02:35:42.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 20 client.114104.objecter shutdown clearing up homeless session... 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 10 client.114104.objecter successfully canceled tick 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f718403a320 msgr2=0x7f718403c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f718403a320 0x7f718403c7e0 secure :-1 s=READY pgs=3399 cs=0 l=1 rev1=1 crypto rx=0x7f719806df00 tx=0x7f718c008040 comp rx=0 tx=0).stop 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 10 monclient: shutdown 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 msgr2=0x7f7198132100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 0x7f7198132100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7190009fd0 tx=0x7f719000e3c0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719759e700 1 -- 172.21.15.47:0/992732056 reap_dead start 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 shutdown_connections 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 --2- 172.21.15.47:0/992732056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71980a2100 0x7f7198132100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 >> 172.21.15.47:0/992732056 conn(0x7f71980a50b0 msgr2=0x7f71980a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 shutdown_connections 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 -- 172.21.15.47:0/992732056 wait complete. 2024-04-20T02:35:42.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.147+0000 7f719d1e0700 1 librados: shutdown 2024-04-20T02:35:42.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:42.194 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-04-20T02:35:42.194 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-20T02:35:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.334+0000 7f5321d89700 10 monclient: get_monmap_and_config 2024-04-20T02:35:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.334+0000 7f5321d89700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.334+0000 7f5321d89700 10 monclient: monmap: 2024-04-20T02:35:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:42.335659+0000 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:42.335659+0000 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:42.336 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 Processor -- start 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 -- start start 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient: init 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient: _add_conns ranks=[2,9,7,4,10,8,5,1,0,6,3] 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): picked mon.noname-b con 0x7f531c09f690 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f531c0a3ea0 0x7f531c0a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): picked mon.noname-a con 0x7f531c0a3ea0 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f531c1357f0 0x7f531c135c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): picked mon.noname-i con 0x7f531c1357f0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c070ce0 con 0x7f531c0a3ea0 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c072780 con 0x7f531c09f690 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f531c09fa70 con 0x7f531c1357f0 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.335+0000 7f5321d89700 10 monclient(hunting): authenticate will time out at 2821.736896s 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f531c0a3ea0 0x7f531c0a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f531c0a3ea0 0x7f531c0a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57450/0 (socket says 172.21.15.47:57450) 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c0a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48372/0 (socket says 172.21.15.47:48372) 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 -- 172.21.15.47:0/355236020 learned_addr learned my addr 172.21.15.47:0/355236020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): get_auth_request con 0x7f531c0a3ea0 auth_method 0 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): get_auth_request con 0x7f531c09f690 auth_method 0 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531bfff700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f531c1357f0 0x7f531c135c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531bfff700 10 monclient(hunting): get_auth_request con 0x7f531c1357f0 auth_method 0 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient(hunting): handle_auth_done global_id 104373 payload 274 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 -- 172.21.15.47:0/355236020 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f531c1357f0 msgr2=0x7f531c135c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f531c1357f0 0x7f531c135c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 -- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 msgr2=0x7f531c0a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c0a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 monclient: found mon.noname-a 2024-04-20T02:35:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 -- 172.21.15.47:0/355236020 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0751e0 con 0x7f531c0a3ea0 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531b7fe700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c0a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531affd700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f531c0a3ea0 0x7f531c0a4320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53100098f0 tx=0x7f5310009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ed47d74e5d8514 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 1 -- 172.21.15.47:0/355236020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5310011aa0 con 0x7f531c0a3ea0 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: dump: 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 1 -- 172.21.15.47:0/355236020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310011c00 con 0x7f531c0a3ea0 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f5321d89700 5 monclient: authenticate success, global_id 104373 2024-04-20T02:35:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f5321d89700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 1 -- 172.21.15.47:0/355236020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5310011d70 con 0x7f531c0a3ea0 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f531a7fc700 10 monclient: dump: 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.352 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f5321d89700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.336+0000 7f5321d89700 4 set_mon_vals no callback set 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 10 monclient: shutdown 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 -- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f531c0a3ea0 msgr2=0x7f531c0a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f531c0a3ea0 0x7f531c0a4320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53100098f0 tx=0x7f5310009c00 comp rx=0 tx=0).stop 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 -- 172.21.15.47:0/355236020 shutdown_connections 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f531c1357f0 0x7f531c135c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c0a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 --2- 172.21.15.47:0/355236020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f531c0a3ea0 0x7f531c0a4320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 -- 172.21.15.47:0/355236020 >> 172.21.15.47:0/355236020 conn(0x7f531c0a8610 msgr2=0x7f531c0a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 -- 172.21.15.47:0/355236020 shutdown_connections 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.337+0000 7f5321d89700 1 -- 172.21.15.47:0/355236020 wait complete. 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 10 monclient: monmap: 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 1 librados: starting msgr at 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 1 librados: starting objecter 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 1 Processor -- start 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 1 -- start start 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 1 librados: setting wanted keys 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 1 librados: calling monclient init 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.338+0000 7f5321d89700 10 monclient: init 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient: _add_conns ranks=[2,4,5,7,6,3,1,8,0] 2024-04-20T02:35:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c09db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): picked mon.noname-b con 0x7f531c09f690 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c0a3ea0 0x7f531c133a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): picked mon.noname-c con 0x7f531c0a3ea0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f531c1357f0 0x7f531c1ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): picked mon.noname-h con 0x7f531c1357f0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1214c0 con 0x7f531c09f690 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531c11f890 con 0x7f531c0a3ea0 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531c11f540 con 0x7f531c1357f0 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f5321d89700 10 monclient(hunting): authenticate will time out at 2821.740465s 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c09db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c09db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48380/0 (socket says 172.21.15.47:48380) 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 -- 172.21.15.47:0/3322014121 learned_addr learned my addr 172.21.15.47:0/3322014121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): get_auth_request con 0x7f531c09f690 auth_method 0 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c0a3ea0 0x7f531c133a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): get_auth_request con 0x7f531c0a3ea0 auth_method 0 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531bfff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f531c1357f0 0x7f531c1ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531bfff700 10 monclient(hunting): get_auth_request con 0x7f531c1357f0 auth_method 0 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient(hunting): handle_auth_done global_id 114140 payload 824 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f531c1357f0 msgr2=0x7f531c1ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f531c1357f0 0x7f531c1ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c0a3ea0 msgr2=0x7f531c133a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c0a3ea0 0x7f531c133a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 monclient: found mon.noname-b 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531b7fe700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0094a0 con 0x7f531c09f690 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.339+0000 7f531bfff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f531c1357f0 0x7f531c1ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f531b7fe700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c09db10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f530c00ebe0 tx=0x7f530c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 client.?.objecter ms_handle_connect 0x7f531c09f690 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f530c009800 con 0x7f531c09f690 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: dump: 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 5 monclient: authenticate success, global_id 114140 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00cd80 con 0x7f531c09f690 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f530c0138e0 con 0x7f531c09f690 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 10 monclient: _renew_subs 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5310009550 con 0x7f531c09f690 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5319ffb700 4 set_mon_vals no callback set 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5303fff700 10 monclient: dump: 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 10 client.114140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 10 monclient: _renew_subs 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c120e90 con 0x7f531c09f690 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.340+0000 7f5321d89700 1 librados: init done 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5321d89700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f530c013470 con 0x7f531c09f690 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5321d89700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 0x7f531c02bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531c068380 con 0x7f531c11fc50 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 0x7f531c02bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient: get_auth_request con 0x7f531c11fc50 auth_method 0 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f530803a2f0 0x7f530803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f530c04a440 con 0x7f531c09f690 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 10 client.114140.objecter ms_dispatch 0x7f531c001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 3 client.114140.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 3 client.114140.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 10 monclient(hunting): handle_auth_done global_id 84748 payload 824 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531bfff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f530803a2f0 0x7f530803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f531c0721c0 con 0x7f531c11fc50 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 20 client.114140.objecter dump_active .. 0 homeless 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531bfff700 10 monclient: get_auth_request con 0x7f530803a2f0 auth_method 0 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 10 client.114140.objecter ms_handle_connect 0x7f531c11fc50 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f5303fff700 10 client.114140.objecter resend_mon_ops 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.341+0000 7f531affd700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 0x7f531c02bbc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f531c0a5910 tx=0x7f5310004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5310020070 con 0x7f531c11fc50 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 10 monclient: dump: 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.364 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 10 client.114140.objecter ms_handle_connect 0x7f530803a2f0 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f531bfff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f530803a2f0 0x7f530803c7b0 secure :-1 s=READY pgs=3400 cs=0 l=1 rev1=1 crypto rx=0x7f531c04ec70 tx=0x7f531400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5310004430 con 0x7f531c11fc50 2024-04-20T02:35:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 msgr2=0x7f531c02bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 0x7f531c02bbc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f531c0a5910 tx=0x7f5310004750 comp rx=0 tx=0).stop 2024-04-20T02:35:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 msgr2=0x7f531c02bbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.342+0000 7f5303fff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c11fc50 0x7f531c02bbc0 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f531c0a5910 tx=0x7f5310004750 comp rx=0 tx=0).stop 2024-04-20T02:35:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5321d89700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5321d89700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 0x7f531c02c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5321d89700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f531c06a400 con 0x7f531c133fd0 2024-04-20T02:35:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 0x7f531c02c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient: get_auth_request con 0x7f531c133fd0 auth_method 0 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 10 monclient(hunting): handle_auth_done global_id 84757 payload 824 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 1 -- 172.21.15.47:0/3322014121 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f531c0724a0 con 0x7f531c133fd0 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5303fff700 10 client.114140.objecter ms_handle_connect 0x7f531c133fd0 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f531affd700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 0x7f531c02c100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5310009c40 tx=0x7f5310011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5303fff700 10 client.114140.objecter resend_mon_ops 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f531002cd50 con 0x7f531c133fd0 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5303fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5303fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.350+0000 7f5303fff700 10 monclient: dump: 2024-04-20T02:35:42.369 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.370 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 <== mon.4 v2:172.21.15.47:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f531002c8e0 con 0x7f531c133fd0 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 msgr2=0x7f531c02c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 0x7f531c02c100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5310009c40 tx=0x7f5310011000 comp rx=0 tx=0).stop 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 msgr2=0x7f531c02c100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5303fff700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f531c133fd0 0x7f531c02c100 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5310009c40 tx=0x7f5310011000 comp rx=0 tx=0).stop 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:42.371 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.372 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.373 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:42.374 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:42.375 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:42.375 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:42.375 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.375 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.376 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.377 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.381 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.382 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:42.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.384 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.385 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.388 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.390 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.391 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.392 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.393 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.394 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.395 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.396 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:42.396 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 10 librados: watch_flush enter 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 10 librados: watch_flush exit 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 20 client.114140.objecter shutdown clearing up homeless session... 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 10 client.114140.objecter successfully canceled tick 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f530803a2f0 msgr2=0x7f530803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f530803a2f0 0x7f530803c7b0 secure :-1 s=READY pgs=3400 cs=0 l=1 rev1=1 crypto rx=0x7f531c04ec70 tx=0x7f531400b040 comp rx=0 tx=0).stop 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 10 monclient: shutdown 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 msgr2=0x7f531c09db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f531bfff700 1 -- 172.21.15.47:0/3322014121 reap_dead start 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c09db10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f530c00ebe0 tx=0x7f530c00eef0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 shutdown_connections 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 --2- 172.21.15.47:0/3322014121 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f531c09f690 0x7f531c09db10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 >> 172.21.15.47:0/3322014121 conn(0x7f531c0a8610 msgr2=0x7f531c0a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 shutdown_connections 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 -- 172.21.15.47:0/3322014121 wait complete. 2024-04-20T02:35:42.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.351+0000 7f5321d89700 1 librados: shutdown 2024-04-20T02:35:42.399 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:42.400 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-04-20T02:35:42.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-04-20T02:35:42.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.541+0000 7f051c9f8700 10 monclient: get_monmap_and_config 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.541+0000 7f051c9f8700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.541+0000 7f051c9f8700 10 monclient: monmap: 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:42.542710+0000 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:42.542710+0000 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:42.542 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 1 Processor -- start 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 1 -- start start 2024-04-20T02:35:42.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 10 monclient: init 2024-04-20T02:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 10 monclient: _add_conns ranks=[0,3,1,9,5,8,4,7,10,2,6] 2024-04-20T02:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0518137d90 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f05180a09e0 0x7f051813cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 10 monclient(hunting): picked mon.noname-g con 0x7f05180a09e0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f051813d1c0 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.542+0000 7f051c9f8700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05180751e0 con 0x7f0518137d90 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051c9f8700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051809dfc0 con 0x7f051813d1c0 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051c9f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051c9f8700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051809fa40 con 0x7f05180a09e0 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051c9f8700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051c9f8700 10 monclient(hunting): authenticate will time out at 2821.944040s 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051659c700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0518137d90 0x7f0518138170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47846/0 (socket says 172.21.15.47:47846) 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051659c700 1 -- 172.21.15.47:0/1863210745 learned_addr learned my addr 172.21.15.47:0/1863210745 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 1 --2- 172.21.15.47:0/1863210745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f05180a09e0 0x7f051813cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): get_auth_request con 0x7f05180a09e0 auth_method 0 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57227862 0 0) 0x7f05180751e0 con 0x7f0518137d90 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): _negotiate 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): my global_id is 104382 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05000035b0 con 0x7f0518137d90 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608392215 0 0) 0x7f051809dfc0 con 0x7f051813d1c0 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): _negotiate 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 10 monclient(hunting): my global_id is 55990 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05180751e0 con 0x7f051813d1c0 2024-04-20T02:35:42.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient(hunting): handle_auth_done global_id 104331 payload 274 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 1 -- 172.21.15.47:0/1863210745 >> v1:172.21.15.179:6789/0 conn(0x7f051813d1c0 legacy=0x7f051813f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 1 -- 172.21.15.47:0/1863210745 >> v1:172.21.15.47:6789/0 conn(0x7f0518137d90 legacy=0x7f0518138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 1 monclient: found mon.noname-g 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.543+0000 7f0515d9b700 1 -- 172.21.15.47:0/1863210745 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518138880 con 0x7f05180a09e0 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f0515d9b700 1 --2- 172.21.15.47:0/1863210745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f05180a09e0 0x7f051813cc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f050c009870 tx=0x7f050c009b80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=4c598b25e417ffe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f050c011aa0 con 0x7f05180a09e0 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: dump: 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.559 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c011c00 con 0x7f05180a09e0 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 5 monclient: authenticate success, global_id 104331 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 1 -- 172.21.15.47:0/1863210745 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f050c011d70 con 0x7f05180a09e0 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 4 set_mon_vals no callback set 2024-04-20T02:35:42.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051559a700 10 monclient: dump: 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 10 monclient: shutdown 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 -- 172.21.15.47:0/1863210745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f05180a09e0 msgr2=0x7f051813cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1863210745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f05180a09e0 0x7f051813cc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f050c009870 tx=0x7f050c009b80 comp rx=0 tx=0).stop 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 -- 172.21.15.47:0/1863210745 shutdown_connections 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1863210745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f05180a09e0 0x7f051813cc80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 -- 172.21.15.47:0/1863210745 >> 172.21.15.47:0/1863210745 conn(0x7f05180aa8b0 msgr2=0x7f05180a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 -- 172.21.15.47:0/1863210745 shutdown_connections 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.544+0000 7f051c9f8700 1 -- 172.21.15.47:0/1863210745 wait complete. 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.545+0000 7f051c9f8700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.545+0000 7f051c9f8700 10 monclient: monmap: 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.563 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 librados: starting msgr at 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 librados: starting objecter 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 Processor -- start 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 -- start start 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 librados: setting wanted keys 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 librados: calling monclient init 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient: init 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient: _add_conns ranks=[1,5,8,0,4,2,6,3,7] 2024-04-20T02:35:42.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05180a09e0 0x7f05181cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): picked mon.noname-f con 0x7f05180a09e0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0518137d90 0x7f05181ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): picked mon.noname-h con 0x7f0518137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 0x7f05180a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): picked mon.noname-e con 0x7f051813d1c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0518138c40 con 0x7f05180a09e0 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0518138e90 con 0x7f0518137d90 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f051809e9e0 con 0x7f051813d1c0 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f051c9f8700 10 monclient(hunting): authenticate will time out at 2821.947811s 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f0516d9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 0x7f05180a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.546+0000 7f0516d9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 0x7f05180a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:49642/0 (socket says 172.21.15.47:49642) 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 -- 172.21.15.47:0/1582522949 learned_addr learned my addr 172.21.15.47:0/1582522949 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): get_auth_request con 0x7f051813d1c0 auth_method 0 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05180a09e0 0x7f05181cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0518137d90 0x7f05181ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): get_auth_request con 0x7f05180a09e0 auth_method 0 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): get_auth_request con 0x7f0518137d90 auth_method 0 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0515d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient(hunting): handle_auth_done global_id 36350 payload 824 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0518137d90 msgr2=0x7f05181ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0518137d90 0x7f05181ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05180a09e0 msgr2=0x7f05181cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05180a09e0 0x7f05181cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 monclient: found mon.noname-e 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c0094a0 con 0x7f051813d1c0 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0516d9d700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 0x7f05180a1680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f051000cc50 tx=0x7f051000cf60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 client.?.objecter ms_handle_connect 0x7f051813d1c0 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0510003e70 con 0x7f051813d1c0 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: dump: 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.568 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 5 monclient: authenticate success, global_id 36350 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051000a890 con 0x7f051813d1c0 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 10 monclient: _renew_subs 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f051000ae70 con 0x7f051813d1c0 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051809dc20 con 0x7f051813d1c0 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0514d99700 4 set_mon_vals no callback set 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 10 client.36350.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:42.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f0506ffd700 10 monclient: dump: 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.571 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 10 monclient: _renew_subs 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 10 monclient: _send_mon_message to mon.e at v2:172.21.15.47:3304/0 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05180a34b0 con 0x7f051813d1c0 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.547+0000 7f051c9f8700 1 librados: init done 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051c9f8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 0x7f05180a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051c9f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0518139020 con 0x7f051806aac0 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.8 v2:172.21.15.47:3304/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f051000aa00 con 0x7f051813d1c0 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 0x7f05180a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient: get_auth_request con 0x7f051806aac0 auth_method 0 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.548+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f04f803a2f0 0x7f04f803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.8 v2:172.21.15.47:3304/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f051004a430 con 0x7f051813d1c0 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 client.36350.objecter ms_dispatch 0x7f0518001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 3 client.36350.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 3 client.36350.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f051659c700 10 monclient(hunting): handle_auth_done global_id 55545 payload 824 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f051659c700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05180a7bc0 con 0x7f051806aac0 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 20 client.36350.objecter dump_active .. 0 homeless 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 client.36350.objecter ms_handle_connect 0x7f051806aac0 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 client.36350.objecter resend_mon_ops 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f051659c700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 0x7f05180a4b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0508005910 tx=0x7f050800fb00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0515d9b700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f04f803a2f0 0x7f04f803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0515d9b700 10 monclient: get_auth_request con 0x7f04f803a2f0 auth_method 0 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f05080070d0 con 0x7f051806aac0 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 monclient: dump: 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0508005b50 con 0x7f051806aac0 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 msgr2=0x7f05180a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 0x7f05180a4b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0508005910 tx=0x7f050800fb00 comp rx=0 tx=0).stop 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 msgr2=0x7f05180a4b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.549+0000 7f0506ffd700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f051806aac0 0x7f05180a4b80 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0508005910 tx=0x7f050800fb00 comp rx=0 tx=0).stop 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.550+0000 7f0506ffd700 10 client.36350.objecter ms_handle_connect 0x7f04f803a2f0 2024-04-20T02:35:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.550+0000 7f0515d9b700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f04f803a2f0 0x7f04f803c7b0 secure :-1 s=READY pgs=3401 cs=0 l=1 rev1=1 crypto rx=0x7f050c009870 tx=0x7f050c01c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051c9f8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 0x7f05180a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051c9f8700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0518068180 con 0x7f05180a50c0 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051659c700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 0x7f05180a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051659c700 10 monclient: get_auth_request con 0x7f05180a50c0 auth_method 0 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.557+0000 7f051659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f051659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f051659c700 10 monclient(hunting): handle_auth_done global_id 55554 payload 824 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f051659c700 1 -- 172.21.15.47:0/1582522949 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f05180a0000 con 0x7f05180a50c0 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 client.36350.objecter ms_handle_connect 0x7f05180a50c0 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f051659c700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 0x7f05180a54d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0508005910 tx=0x7f05080010a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 client.36350.objecter resend_mon_ops 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0508023070 con 0x7f05180a50c0 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 monclient: dump: 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 <== mon.6 v2:172.21.15.47:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f050801ed20 con 0x7f05180a50c0 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 msgr2=0x7f05180a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 0x7f05180a54d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0508005910 tx=0x7f05080010a0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 msgr2=0x7f05180a54d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.558+0000 7f0506ffd700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f05180a50c0 0x7f05180a54d0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0508005910 tx=0x7f05080010a0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:42.581 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.582 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.583 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.584 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:42.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.586 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:42.588 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.589 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:42.590 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.592 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.593 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.594 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.595 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:42.596 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.597 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:42.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.601 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:42.602 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 10 librados: watch_flush enter 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 10 librados: watch_flush exit 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 20 client.36350.objecter shutdown clearing up homeless session... 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 10 client.36350.objecter successfully canceled tick 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f04f803a2f0 msgr2=0x7f04f803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f04f803a2f0 0x7f04f803c7b0 secure :-1 s=READY pgs=3401 cs=0 l=1 rev1=1 crypto rx=0x7f050c009870 tx=0x7f050c01c040 comp rx=0 tx=0).stop 2024-04-20T02:35:42.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 10 monclient: shutdown 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 msgr2=0x7f05180a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f0516d9d700 1 -- 172.21.15.47:0/1582522949 reap_dead start 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 0x7f05180a1680 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f051000cc50 tx=0x7f051000cf60 comp rx=0 tx=0).stop 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 shutdown_connections 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 --2- 172.21.15.47:0/1582522949 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f051813d1c0 0x7f05180a1680 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 >> 172.21.15.47:0/1582522949 conn(0x7f05180aa8b0 msgr2=0x7f0518135040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 shutdown_connections 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 -- 172.21.15.47:0/1582522949 wait complete. 2024-04-20T02:35:42.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.559+0000 7f051c9f8700 1 librados: shutdown 2024-04-20T02:35:42.605 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:42.606 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-04-20T02:35:42.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-04-20T02:35:42.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.745+0000 7fa0727b6700 10 monclient: get_monmap_and_config 2024-04-20T02:35:42.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.745+0000 7fa0727b6700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.745+0000 7fa0727b6700 10 monclient: monmap: 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:42.746813+0000 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:42.746813+0000 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.746 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.747 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.747 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.747 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:42.747 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:42.747 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.746+0000 7fa0727b6700 1 Processor -- start 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.746+0000 7fa0727b6700 1 -- start start 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.746+0000 7fa0727b6700 10 monclient: init 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.746+0000 7fa0727b6700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient: _add_conns ranks=[2,7,3,5,0,9,8,10,6,1,4] 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 0x7fa06c09c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): picked mon.noname-b con 0x7fa06c09d060 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c0a1f10 0x7fa06c0a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): picked mon.noname-i con 0x7fa06c0a1f10 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa06c0a2870 0x7fa06c13e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): picked mon.noname-g con 0x7fa06c0a2870 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0a3dd0 con 0x7fa06c09d060 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c09d440 con 0x7fa06c0a2870 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c09b730 con 0x7fa06c0a1f10 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa0727b6700 10 monclient(hunting): authenticate will time out at 2822.148137s 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 0x7fa06c09c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 0x7fa06c09c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48384/0 (socket says 172.21.15.47:48384) 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 -- 172.21.15.47:0/3940207218 learned_addr learned my addr 172.21.15.47:0/3940207218 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): get_auth_request con 0x7fa06c09d060 auth_method 0 2024-04-20T02:35:42.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa070d53700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa06c0a2870 0x7fa06c13e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa070d53700 10 monclient(hunting): get_auth_request con 0x7fa06c0a2870 auth_method 0 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa070d53700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa070d53700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa070d53700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06b7fe700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c0a1f10 0x7fa06c0a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06b7fe700 10 monclient(hunting): get_auth_request con 0x7fa06c0a1f10 auth_method 0 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient(hunting): handle_auth_done global_id 114149 payload 274 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 -- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c0a1f10 msgr2=0x7fa06c0a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c0a1f10 0x7fa06c0a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 -- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa06c0a2870 msgr2=0x7fa06c13e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa06c0a2870 0x7fa06c13e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 monclient: found mon.noname-b 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 -- 172.21.15.47:0/3940207218 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0751e0 con 0x7fa06c09d060 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa070d53700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa06c0a2870 0x7fa06c13e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06bfff700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 0x7fa06c09c3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009920 tx=0x7fa05c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9958ac3cb4465a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06affd700 1 -- 172.21.15.47:0/3940207218 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa05c009e70 con 0x7fa06c09d060 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.747+0000 7fa06affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: dump: 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.763 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 1 -- 172.21.15.47:0/3940207218 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c00bcd0 con 0x7fa06c09d060 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 5 monclient: authenticate success, global_id 114149 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 1 -- 172.21.15.47:0/3940207218 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa05c00be40 con 0x7fa06c09d060 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 4 set_mon_vals no callback set 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:42.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa06affd700 10 monclient: dump: 2024-04-20T02:35:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.765 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 10 monclient: shutdown 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 -- 172.21.15.47:0/3940207218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 msgr2=0x7fa06c09c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 0x7fa06c09c3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009920 tx=0x7fa05c009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 -- 172.21.15.47:0/3940207218 shutdown_connections 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa06c0a2870 0x7fa06c13e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c09d060 0x7fa06c09c3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 --2- 172.21.15.47:0/3940207218 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c0a1f10 0x7fa06c0a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 -- 172.21.15.47:0/3940207218 >> 172.21.15.47:0/3940207218 conn(0x7fa06c0a3000 msgr2=0x7fa06c0a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 -- 172.21.15.47:0/3940207218 shutdown_connections 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.748+0000 7fa0727b6700 1 -- 172.21.15.47:0/3940207218 wait complete. 2024-04-20T02:35:42.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 10 monclient: monmap: 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.768 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.769 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 1 librados: starting msgr at 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 1 librados: starting objecter 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 1 Processor -- start 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 1 -- start start 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 1 librados: setting wanted keys 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 1 librados: calling monclient init 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.749+0000 7fa0727b6700 10 monclient: init 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient: _add_conns ranks=[7,2,4,1,0,5,3,8,6] 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c09d060 0x7fa06c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): picked mon.noname-i con 0x7fa06c09d060 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:42.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c0a1f10 0x7fa06c1c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): picked mon.noname-b con 0x7fa06c0a1f10 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa06c0a2870 0x7fa06c098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): picked mon.noname-c con 0x7fa06c0a2870 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0709b0 con 0x7fa06c0a1f10 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c072a40 con 0x7fa06c0a2870 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c071010 con 0x7fa06c09d060 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa0727b6700 10 monclient(hunting): authenticate will time out at 2822.151477s 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa06c0a2870 0x7fa06c098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c0a1f10 0x7fa06c1c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa06c0a2870 0x7fa06c098ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37688/0 (socket says 172.21.15.47:37688) 2024-04-20T02:35:42.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 1 -- 172.21.15.47:0/503112158 learned_addr learned my addr 172.21.15.47:0/503112158 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c0a1f10 0x7fa06c1c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48390/0 (socket says 172.21.15.47:48390) 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): get_auth_request con 0x7fa06c0a2870 auth_method 0 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c09d060 0x7fa06c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): get_auth_request con 0x7fa06c0a1f10 auth_method 0 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): get_auth_request con 0x7fa06c09d060 auth_method 0 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa070d53700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 10 monclient(hunting): handle_auth_done global_id 114158 payload 824 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c09d060 msgr2=0x7fa06c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.750+0000 7fa06b7fe700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fa06c09d060 0x7fa06c071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa06c0a2870 msgr2=0x7fa06c098ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa06c0a2870 0x7fa06c098ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 1 monclient: found mon.noname-b 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009550 con 0x7fa06c0a1f10 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa070d53700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa06c0a2870 0x7fa06c098ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06b7fe700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c0a1f10 0x7fa06c1c9510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa06000e5d0 tx=0x7fa06000e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 client.?.objecter ms_handle_connect 0x7fa06c0a1f10 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa060009380 con 0x7fa06c0a1f10 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: dump: 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.774 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0600044a0 con 0x7fa06c0a1f10 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa060003b10 con 0x7fa06c0a1f10 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa068ff9700 10 monclient: dump: 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.776 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa06a7fc700 4 set_mon_vals no callback set 2024-04-20T02:35:42.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 5 monclient: authenticate success, global_id 114158 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 10 monclient: _renew_subs 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c1c9a50 con 0x7fa06c0a1f10 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 10 client.114158.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 10 monclient: _renew_subs 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c09a940 con 0x7fa06c0a1f10 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.751+0000 7fa0727b6700 1 librados: init done 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa0727b6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa060003c70 con 0x7fa06c0a1f10 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa0727b6700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 0x7fa06c1cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c09c0d0 con 0x7fa06c06aac0 2024-04-20T02:35:42.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 0x7fa06c1cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient: get_auth_request con 0x7fa06c06aac0 auth_method 0 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa05803a2f0 0x7fa05803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fa060049bf0 con 0x7fa06c0a1f10 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 10 client.114158.objecter ms_dispatch 0x7fa06c001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 3 client.114158.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 3 client.114158.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa070d53700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa05803a2f0 0x7fa05803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa068ff9700 20 client.114158.objecter dump_active .. 0 homeless 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa070d53700 10 monclient: get_auth_request con 0x7fa05803a2f0 auth_method 0 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 10 monclient(hunting): handle_auth_done global_id 36359 payload 824 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.752+0000 7fa06bfff700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa06c09c930 con 0x7fa06c06aac0 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa06bfff700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 0x7fa06c1cc500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009fd0 tx=0x7fa05c004750 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 client.114158.objecter ms_handle_connect 0x7fa06c06aac0 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 client.114158.objecter resend_mon_ops 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa05c020070 con 0x7fa06c06aac0 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 monclient: dump: 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.780 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 client.114158.objecter ms_handle_connect 0x7fa05803a2f0 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa070d53700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa05803a2f0 0x7fa05803c7b0 secure :-1 s=READY pgs=3402 cs=0 l=1 rev1=1 crypto rx=0x7fa06c071870 tx=0x7fa064009870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa05c00bb60 con 0x7fa06c06aac0 2024-04-20T02:35:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 msgr2=0x7fa06c1cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 0x7fa06c1cc500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009fd0 tx=0x7fa05c004750 comp rx=0 tx=0).stop 2024-04-20T02:35:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 msgr2=0x7fa06c1cc500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.753+0000 7fa068ff9700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c06aac0 0x7fa06c1cc500 secure :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009fd0 tx=0x7fa05c004750 comp rx=0 tx=0).stop 2024-04-20T02:35:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa0727b6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa0727b6700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 0x7fa06c1cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa0727b6700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c068180 con 0x7fa06c099000 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 0x7fa06c1cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient: get_auth_request con 0x7fa06c099000 auth_method 0 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.761+0000 7fa06bfff700 10 monclient(hunting): handle_auth_done global_id 36368 payload 824 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa06bfff700 1 -- 172.21.15.47:0/503112158 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa06c099f60 con 0x7fa06c099000 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 client.114158.objecter ms_handle_connect 0x7fa06c099000 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa06bfff700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 0x7fa06c1cca40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009130 tx=0x7fa05c0092c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 client.114158.objecter resend_mon_ops 2024-04-20T02:35:42.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa05c02a9f0 con 0x7fa06c099000 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 monclient: dump: 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.786 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 <== mon.8 v2:172.21.15.47:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9524 (secure 0 0 0) 0x7fa05c0045e0 con 0x7fa06c099000 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 msgr2=0x7fa06c1cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 0x7fa06c1cca40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009130 tx=0x7fa05c0092c0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 msgr2=0x7fa06c1cca40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa068ff9700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fa06c099000 0x7fa06c1cca40 secure :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009130 tx=0x7fa05c0092c0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.788 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:42.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:42.788 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:42.788 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:42.788 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.789 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.790 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.791 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:42.792 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.793 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.795 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.796 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.797 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.798 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.799 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:42.801 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:42.802 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.803 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:42.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.805 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.806 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.807 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.809 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.810 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": [ 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.812 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:42.812 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa0727b6700 10 librados: watch_flush enter 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa0727b6700 10 librados: watch_flush exit 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa0727b6700 20 client.114158.objecter shutdown clearing up homeless session... 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.762+0000 7fa0727b6700 10 client.114158.objecter successfully canceled tick 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa05803a2f0 msgr2=0x7fa05803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fa05803a2f0 0x7fa05803c7b0 secure :-1 s=READY pgs=3402 cs=0 l=1 rev1=1 crypto rx=0x7fa06c071870 tx=0x7fa064009870 comp rx=0 tx=0).stop 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 10 monclient: shutdown 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c0a1f10 msgr2=0x7fa06c1c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 --2- 172.21.15.47:0/503112158 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa06c0a1f10 0x7fa06c1c9510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa06000e5d0 tx=0x7fa06000e8e0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa070d53700 1 -- 172.21.15.47:0/503112158 reap_dead start 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa070d53700 1 -- 172.21.15.47:0/503112158 reap_dead start 2024-04-20T02:35:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 shutdown_connections 2024-04-20T02:35:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 >> 172.21.15.47:0/503112158 conn(0x7fa06c0a3000 msgr2=0x7fa06c1bfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 shutdown_connections 2024-04-20T02:35:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 -- 172.21.15.47:0/503112158 wait complete. 2024-04-20T02:35:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.763+0000 7fa0727b6700 1 librados: shutdown 2024-04-20T02:35:42.815 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:42.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-04-20T02:35:42.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-04-20T02:35:42.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.954+0000 7ffb5f22f700 10 monclient: get_monmap_and_config 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.954+0000 7ffb5f22f700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.954+0000 7ffb5f22f700 10 monclient: monmap: 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:42.956064+0000 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:42.956064+0000 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.955 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:42.956 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:42.956 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.955+0000 7ffb5f22f700 1 Processor -- start 2024-04-20T02:35:42.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.955+0000 7ffb5f22f700 1 -- start start 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.955+0000 7ffb5f22f700 10 monclient: init 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient: _add_conns ranks=[6,8,0,7,2,1,10,5,9,3,4] 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fa80 0x7ffb58135df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): picked mon.noname-d con 0x7ffb5809fa80 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb5809fe60 0x7ffb580a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): picked mon.noname-e con 0x7ffb5809fe60 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ffb580a1f10 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580751e0 con 0x7ffb580a1f10 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb580a0780 con 0x7ffb5809fa80 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5809df90 con 0x7ffb5809fe60 2024-04-20T02:35:42.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5f22f700 10 monclient(hunting): authenticate will time out at 2822.357437s 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb5809fe60 0x7ffb580a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5d7cc700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffb580a1f10 0x7ffb580a2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47860/0 (socket says 172.21.15.47:47860) 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5d7cc700 1 -- 172.21.15.47:0/3702207607 learned_addr learned my addr 172.21.15.47:0/3702207607 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fa80 0x7ffb58135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): get_auth_request con 0x7ffb5809fe60 auth_method 0 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): get_auth_request con 0x7ffb5809fa80 auth_method 0 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5d7cc700 1 -- 172.21.15.47:0/3702207607 >> v1:172.21.15.47:6789/0 conn(0x7ffb580a1f10 legacy=0x7ffb580a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-04-20T02:35:42.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5d7cc700 1 -- 172.21.15.47:0/3702207607 >> v1:172.21.15.47:6789/0 conn(0x7ffb580a1f10 legacy=0x7ffb580a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5d7cc700 1 --1- 172.21.15.47:0/3702207607 >> v1:172.21.15.47:6789/0 conn(0x7ffb580a1f10 0x7ffb580a2370 :-1 s=OPENED pgs=140 cs=1 l=1).handle_message read tag failed 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5d7cc700 1 --1- 172.21.15.47:0/3702207607 >> v1:172.21.15.47:6789/0 conn(0x7ffb580a1f10 0x7ffb580a2370 :-1 s=OPENED pgs=140 cs=1 l=1).fault on lossy channel, failing 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb577fe700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.47:6789/0 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_done global_id 55563 payload 274 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 1 -- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb5809fe60 msgr2=0x7ffb580a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb5809fe60 0x7ffb580a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 1 -- 172.21.15.47:0/3702207607 >> v1:172.21.15.47:6789/0 conn(0x7ffb580a1f10 legacy=0x7ffb580a2370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 1 monclient: found mon.noname-d 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb5cfcb700 1 -- 172.21.15.47:0/3702207607 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5809dcb0 con 0x7ffb5809fa80 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.956+0000 7ffb57fff700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb5809fe60 0x7ffb580a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fa80 0x7ffb58135df0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb480097c0 tx=0x7ffb48009ad0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e6d25fd75fde780c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 1 -- 172.21.15.47:0/3702207607 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480044a0 con 0x7ffb5809fa80 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: dump: 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.970 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 5 monclient: authenticate success, global_id 55563 2024-04-20T02:35:42.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 1 -- 172.21.15.47:0/3702207607 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb480036a0 con 0x7ffb5809fa80 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 1 -- 172.21.15.47:0/3702207607 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb48003c80 con 0x7ffb5809fa80 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 4 set_mon_vals no callback set 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb577fe700 10 monclient: dump: 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.972 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 10 monclient: shutdown 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fa80 msgr2=0x7ffb58135df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fa80 0x7ffb58135df0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb480097c0 tx=0x7ffb48009ad0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3702207607 shutdown_connections 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fa80 0x7ffb58135df0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3702207607 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ffb5809fe60 0x7ffb580a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.957+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3702207607 >> 172.21.15.47:0/3702207607 conn(0x7ffb580a3000 msgr2=0x7ffb580a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.958+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3702207607 shutdown_connections 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.958+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3702207607 wait complete. 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.958+0000 7ffb5f22f700 10 monclient: build_initial_monmap 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.958+0000 7ffb5f22f700 10 monclient: monmap: 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:42.974 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.958+0000 7ffb5f22f700 1 librados: starting msgr at 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.958+0000 7ffb5f22f700 1 librados: starting objecter 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 Processor -- start 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 -- start start 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 librados: setting wanted keys 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 librados: calling monclient init 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient: init 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:42.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient: _add_conns ranks=[0,6,5,2,3,7,4,1,8] 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb5809fa80 0x7ffb581ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): picked mon.noname-a con 0x7ffb5809fa80 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fe60 0x7ffb58132ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): picked mon.noname-d con 0x7ffb5809fe60 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb580a1f10 0x7ffb58133420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): picked mon.noname-h con 0x7ffb580a1f10 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5809cbc0 con 0x7ffb5809fa80 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58133960 con 0x7ffb580a1f10 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5812e0e0 con 0x7ffb5809fe60 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5f22f700 10 monclient(hunting): authenticate will time out at 2822.360923s 2024-04-20T02:35:42.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.959+0000 7ffb5cfcb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb5809fa80 0x7ffb581ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb5809fa80 0x7ffb581ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57452/0 (socket says 172.21.15.47:57452) 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 1 -- 172.21.15.47:0/3024686498 learned_addr learned my addr 172.21.15.47:0/3024686498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): get_auth_request con 0x7ffb5809fa80 auth_method 0 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fe60 0x7ffb58132ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): get_auth_request con 0x7ffb5809fe60 auth_method 0 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb580a1f10 0x7ffb58133420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): get_auth_request con 0x7ffb580a1f10 auth_method 0 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5d7cc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient(hunting): handle_auth_done global_id 55572 payload 824 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb580a1f10 msgr2=0x7ffb58133420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7ffb580a1f10 0x7ffb58133420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb5809fa80 msgr2=0x7ffb581ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb5809fa80 0x7ffb581ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 monclient: found mon.noname-d 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400094a0 con 0x7ffb5809fe60 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb5809fa80 0x7ffb581ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb57fff700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fe60 0x7ffb58132ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000e9b0 tx=0x7ffb4000ecc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 client.?.objecter ms_handle_connect 0x7ffb5809fe60 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4000ee50 con 0x7ffb5809fe60 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: dump: 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.979 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: _finish_auth 0 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: _check_auth_tickets 2024-04-20T02:35:42.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 5 monclient: authenticate success, global_id 55572 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4000ccc0 con 0x7ffb5809fe60 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 10 monclient: _renew_subs 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4000ce30 con 0x7ffb5809fe60 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480094a0 con 0x7ffb5809fe60 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb56ffd700 4 set_mon_vals no callback set 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 10 client.55572.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb557fa700 10 monclient: dump: 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.981 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 10 monclient: _renew_subs 2024-04-20T02:35:42.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb580729b0 con 0x7ffb5809fe60 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.960+0000 7ffb5f22f700 1 librados: init done 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5f22f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 0x7ffb5812b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58068180 con 0x7ffb5806aac0 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ffb4000c850 con 0x7ffb5809fe60 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 0x7ffb5812b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient: get_auth_request con 0x7ffb5806aac0 auth_method 0 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.961+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb3803a2f0 0x7ffb3803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_done global_id 56008 payload 824 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7ffb4004a240 con 0x7ffb5809fe60 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 client.55572.objecter ms_dispatch 0x7ffb58001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 3 client.55572.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 3 client.55572.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb5cfcb700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb5809e550 con 0x7ffb5806aac0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb5d7cc700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb3803a2f0 0x7ffb3803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 20 client.55572.objecter dump_active .. 0 homeless 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb5d7cc700 10 monclient: get_auth_request con 0x7ffb3803a2f0 auth_method 0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 client.55572.objecter ms_handle_connect 0x7ffb5806aac0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 0x7ffb5812b5c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009790 tx=0x7ffb480047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 client.55572.objecter resend_mon_ops 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb48004580 con 0x7ffb5806aac0 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 monclient: dump: 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.985 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 client.55572.objecter ms_handle_connect 0x7ffb3803a2f0 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb5d7cc700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb3803a2f0 0x7ffb3803c7b0 secure :-1 s=READY pgs=3403 cs=0 l=1 rev1=1 crypto rx=0x7ffb58072d50 tx=0x7ffb50009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ffb4801a430 con 0x7ffb5806aac0 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 msgr2=0x7ffb5812b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 0x7ffb5812b5c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009790 tx=0x7ffb480047c0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 msgr2=0x7ffb5812b5c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.962+0000 7ffb557fa700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5806aac0 0x7ffb5812b5c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.970+0000 7ffb5f22f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.970+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 0x7ffb5812bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.970+0000 7ffb5f22f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.970+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5806a400 con 0x7ffb5812bb00 2024-04-20T02:35:42.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 0x7ffb5812bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient: get_auth_request con 0x7ffb5812bb00 auth_method 0 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 10 monclient(hunting): handle_auth_done global_id 56017 payload 824 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 1 -- 172.21.15.47:0/3024686498 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffb580a0930 con 0x7ffb5812bb00 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb5cfcb700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 0x7ffb5812bf10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffb58099fa0 tx=0x7ffb48011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb557fa700 10 client.55572.objecter ms_handle_connect 0x7ffb5812bb00 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb557fa700 10 client.55572.objecter resend_mon_ops 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb48009300 con 0x7ffb5812bb00 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb557fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:42.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb557fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.971+0000 7ffb557fa700 10 monclient: dump: 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:42.990 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 <== mon.1 v2:172.21.15.179:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9356 (secure 0 0 0) 0x7ffb4802c6d0 con 0x7ffb5812bb00 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 msgr2=0x7ffb5812bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 0x7ffb5812bf10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffb58099fa0 tx=0x7ffb48011000 comp rx=0 tx=0).stop 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 msgr2=0x7ffb5812bf10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb557fa700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb5812bb00 0x7ffb5812bf10 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffb58099fa0 tx=0x7ffb48011000 comp rx=0 tx=0).stop 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:42.991 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "leader", 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.992 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:42.993 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:42.994 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:42.995 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:42.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.997 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:42.998 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:42.999 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:43.001 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.002 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:43.004 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.005 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:43.007 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.008 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.010 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 3 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.012 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:43.012 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:43.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 10 librados: watch_flush enter 2024-04-20T02:35:43.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 10 librados: watch_flush exit 2024-04-20T02:35:43.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 20 client.55572.objecter shutdown clearing up homeless session... 2024-04-20T02:35:43.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 10 client.55572.objecter successfully canceled tick 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb3803a2f0 msgr2=0x7ffb3803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ffb3803a2f0 0x7ffb3803c7b0 secure :-1 s=READY pgs=3403 cs=0 l=1 rev1=1 crypto rx=0x7ffb58072d50 tx=0x7ffb50009040 comp rx=0 tx=0).stop 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 10 monclient: shutdown 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fe60 msgr2=0x7ffb58132ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fe60 0x7ffb58132ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000e9b0 tx=0x7ffb4000ecc0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5d7cc700 1 -- 172.21.15.47:0/3024686498 reap_dead start 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 shutdown_connections 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 --2- 172.21.15.47:0/3024686498 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ffb5809fe60 0x7ffb58132ee0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 >> 172.21.15.47:0/3024686498 conn(0x7ffb580a3000 msgr2=0x7ffb5809ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 shutdown_connections 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 -- 172.21.15.47:0/3024686498 wait complete. 2024-04-20T02:35:43.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:42.972+0000 7ffb5f22f700 1 librados: shutdown 2024-04-20T02:35:43.015 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:43.015 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-04-20T02:35:43.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.153+0000 7fdee4334700 10 monclient: get_monmap_and_config 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.153+0000 7fdee4334700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.153+0000 7fdee4334700 10 monclient: monmap: 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:43.154801+0000 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:43.154801+0000 2024-04-20T02:35:43.154 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 1 Processor -- start 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 1 -- start start 2024-04-20T02:35:43.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 10 monclient: init 2024-04-20T02:35:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 10 monclient: _add_conns ranks=[4,9,2,7,3,5,1,6,8,0,10] 2024-04-20T02:35:43.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 0x7fdedc09b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.154+0000 7fdee4334700 10 monclient(hunting): picked mon.noname-c con 0x7fdedc135ad0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 0x7fdedc1362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): picked mon.noname-a con 0x7fdedc135eb0 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 0x7fdedc0a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): picked mon.noname-b con 0x7fdedc0a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc09bc20 con 0x7fdedc135eb0 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc067db0 con 0x7fdedc0a2950 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0a0ae0 con 0x7fdedc135ad0 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee4334700 10 monclient(hunting): authenticate will time out at 2822.556048s 2024-04-20T02:35:43.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 0x7fdedc09b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 0x7fdedc0a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 0x7fdedc1362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 0x7fdedc0a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:48392/0 (socket says 172.21.15.47:48392) 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 0x7fdedc09b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37698/0 (socket says 172.21.15.47:37698) 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 0x7fdedc1362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57460/0 (socket says 172.21.15.47:57460) 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 1 -- 172.21.15.47:0/2001968263 learned_addr learned my addr 172.21.15.47:0/2001968263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): get_auth_request con 0x7fdedc135ad0 auth_method 0 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): get_auth_request con 0x7fdedc0a2950 auth_method 0 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): get_auth_request con 0x7fdedc135eb0 auth_method 0 2024-04-20T02:35:43.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient(hunting): handle_auth_done global_id 114167 payload 274 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 -- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 msgr2=0x7fdedc09b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 --2- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 0x7fdedc09b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 -- 172.21.15.47:0/2001968263 >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 msgr2=0x7fdedc1362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 --2- 172.21.15.47:0/2001968263 >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 0x7fdedc1362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 monclient: found mon.noname-b 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 -- 172.21.15.47:0/2001968263 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc0751e0 con 0x7fdedc0a2950 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee20d0700 1 --2- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 0x7fdedc09b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee18cf700 1 --2- 172.21.15.47:0/2001968263 >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 0x7fdedc1362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee28d1700 1 --2- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 0x7fdedc0a2dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fded4009d90 tx=0x7fded400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c157d488d5289fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee10ce700 1 -- 172.21.15.47:0/2001968263 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fded400e070 con 0x7fdedc0a2950 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.155+0000 7fdee10ce700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: dump: 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.171 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 5 monclient: authenticate success, global_id 114167 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 1 -- 172.21.15.47:0/2001968263 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded40098f0 con 0x7fdedc0a2950 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 1 -- 172.21.15.47:0/2001968263 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fded40128a0 con 0x7fdedc0a2950 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 4 set_mon_vals no callback set 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee10ce700 10 monclient: dump: 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.173 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 10 monclient: shutdown 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 -- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 msgr2=0x7fdedc0a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 --2- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 0x7fdedc0a2dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fded4009d90 tx=0x7fded400b330 comp rx=0 tx=0).stop 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 -- 172.21.15.47:0/2001968263 shutdown_connections 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 --2- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdedc0a2950 0x7fdedc0a2dd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 --2- 172.21.15.47:0/2001968263 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc135ad0 0x7fdedc09b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 --2- 172.21.15.47:0/2001968263 >> v2:172.21.15.47:3300/0 conn(0x7fdedc135eb0 0x7fdedc1362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 -- 172.21.15.47:0/2001968263 >> 172.21.15.47:0/2001968263 conn(0x7fdedc0a7570 msgr2=0x7fdedc0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 -- 172.21.15.47:0/2001968263 shutdown_connections 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.156+0000 7fdee4334700 1 -- 172.21.15.47:0/2001968263 wait complete. 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 10 monclient: monmap: 2024-04-20T02:35:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 1 librados: starting msgr at 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 1 librados: starting objecter 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 1 Processor -- start 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 1 -- start start 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 1 librados: setting wanted keys 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 1 librados: calling monclient init 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.157+0000 7fdee4334700 10 monclient: init 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient: _add_conns ranks=[4,8,3,6,7,0,1,5,2] 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc0a2950 0x7fdedc1cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): picked mon.noname-c con 0x7fdedc0a2950 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdedc135ad0 0x7fdedc133a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): picked mon.noname-e con 0x7fdedc135ad0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc135eb0 0x7fdedc133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): picked mon.noname-g con 0x7fdedc135eb0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc127200 con 0x7fdedc135eb0 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc127470 con 0x7fdedc0a2950 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc066ea0 con 0x7fdedc135ad0 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee4334700 10 monclient(hunting): authenticate will time out at 2822.559290s 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc0a2950 0x7fdedc1cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdedc135ad0 0x7fdedc133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc0a2950 0x7fdedc1cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37712/0 (socket says 172.21.15.47:37712) 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 -- 172.21.15.47:0/1203098220 learned_addr learned my addr 172.21.15.47:0/1203098220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): get_auth_request con 0x7fdedc0a2950 auth_method 0 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): get_auth_request con 0x7fdedc135ad0 auth_method 0 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc135eb0 0x7fdedc133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): get_auth_request con 0x7fdedc135eb0 auth_method 0 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee28d1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient(hunting): handle_auth_done global_id 84784 payload 824 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdedc135ad0 msgr2=0x7fdedc133a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdedc135ad0 0x7fdedc133a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc135eb0 msgr2=0x7fdedc133f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc135eb0 0x7fdedc133f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 monclient: found mon.noname-c 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8009550 con 0x7fdedc0a2950 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee18cf700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fdedc135ad0 0x7fdedc133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fdee20d0700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc0a2950 0x7fdedc1cc450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fded800ebe0 tx=0x7fded800eef0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fded2ffd700 10 client.?.objecter ms_handle_connect 0x7fdedc0a2950 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.158+0000 7fded2ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fded80044a0 con 0x7fdedc0a2950 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: dump: 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.181 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 5 monclient: authenticate success, global_id 84784 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 monclient: _renew_subs 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded800c850 con 0x7fdedc0a2950 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc0094a0 con 0x7fdedc0a2950 2024-04-20T02:35:43.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fded800ce30 con 0x7fdedc0a2950 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee08cd700 4 set_mon_vals no callback set 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 client.84784.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fded2ffd700 10 monclient: dump: 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.183 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 monclient: _renew_subs 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded4009a70 con 0x7fdedc0a2950 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 1 librados: init done 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 0x7fdedc123de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.159+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc127600 con 0x7fdedc06aac0 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 0x7fdedc123de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient: get_auth_request con 0x7fdedc06aac0 auth_method 0 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 10 monclient(hunting): handle_auth_done global_id 104358 payload 824 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdedc126bd0 con 0x7fdedc06aac0 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fded800c9c0 con 0x7fdedc0a2950 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee18cf700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 0x7fdedc123de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0a7e80 tx=0x7fdecc00f520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdec003a2f0 0x7fdec003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 10 client.84784.objecter ms_handle_connect 0x7fdedc06aac0 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 10 client.84784.objecter resend_mon_ops 2024-04-20T02:35:43.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fded804a3a0 con 0x7fdedc0a2950 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 10 client.84784.objecter ms_dispatch 0x7fdedc001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 3 client.84784.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 3 client.84784.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 20 client.84784.objecter dump_active .. 0 homeless 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdecc00fd80 con 0x7fdedc06aac0 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fdee28d1700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdec003a2f0 0x7fdec003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.160+0000 7fded2ffd700 10 monclient: dump: 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.186 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fdee28d1700 10 monclient: get_auth_request con 0x7fdec003a2f0 auth_method 0 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdecc014440 con 0x7fdedc06aac0 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 msgr2=0x7fdedc123de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 0x7fdedc123de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0a7e80 tx=0x7fdecc00f520 comp rx=0 tx=0).stop 2024-04-20T02:35:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 msgr2=0x7fdedc123de0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc06aac0 0x7fdedc123de0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0a7e80 tx=0x7fdecc00f520 comp rx=0 tx=0).stop 2024-04-20T02:35:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fded2ffd700 10 client.84784.objecter ms_handle_connect 0x7fdec003a2f0 2024-04-20T02:35:43.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.161+0000 7fdee28d1700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdec003a2f0 0x7fdec003c7b0 secure :-1 s=READY pgs=3404 cs=0 l=1 rev1=1 crypto rx=0x7fded400b370 tx=0x7fded4011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.168+0000 7fdee4334700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee4334700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 0x7fdedc02bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee4334700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc068310 con 0x7fdedc02bbc0 2024-04-20T02:35:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 0x7fdedc02bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient: get_auth_request con 0x7fdedc02bbc0 auth_method 0 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 10 monclient(hunting): handle_auth_done global_id 104367 payload 824 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 1 -- 172.21.15.47:0/1203098220 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdedc067a40 con 0x7fdedc02bbc0 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fded2ffd700 10 client.84784.objecter ms_handle_connect 0x7fdedc02bbc0 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fded2ffd700 10 client.84784.objecter resend_mon_ops 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.169+0000 7fdee18cf700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 0x7fdedc02bfd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0a7e80 tx=0x7fdecc004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdecc02b070 con 0x7fdedc02bbc0 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 10 monclient: dump: 2024-04-20T02:35:43.191 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 <== mon.3 v2:172.21.15.179:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7fdecc023040 con 0x7fdedc02bbc0 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 msgr2=0x7fdedc02bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 0x7fdedc02bfd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0a7e80 tx=0x7fdecc004750 comp rx=0 tx=0).stop 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 msgr2=0x7fdedc02bfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fded2ffd700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdedc02bbc0 0x7fdedc02bfd0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0a7e80 tx=0x7fdecc004750 comp rx=0 tx=0).stop 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:43.193 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 1, 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:43.194 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.195 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:43.196 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.197 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.198 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.201 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.202 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.203 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:43.204 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.205 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.207 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.208 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.209 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.210 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:43.212 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:43.213 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 10 librados: watch_flush enter 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 10 librados: watch_flush exit 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 20 client.84784.objecter shutdown clearing up homeless session... 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 10 client.84784.objecter successfully canceled tick 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdec003a2f0 msgr2=0x7fdec003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fdec003a2f0 0x7fdec003c7b0 secure :-1 s=READY pgs=3404 cs=0 l=1 rev1=1 crypto rx=0x7fded400b370 tx=0x7fded4011000 comp rx=0 tx=0).stop 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 10 monclient: shutdown 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc0a2950 msgr2=0x7fdedc1cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee4334700 1 --2- 172.21.15.47:0/1203098220 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdedc0a2950 0x7fdedc1cc450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fded800ebe0 tx=0x7fded800eef0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee28d1700 1 -- 172.21.15.47:0/1203098220 reap_dead start 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.170+0000 7fdee28d1700 1 -- 172.21.15.47:0/1203098220 reap_dead start 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.171+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 shutdown_connections 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.171+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 >> 172.21.15.47:0/1203098220 conn(0x7fdedc0a7570 msgr2=0x7fdedc0a5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.171+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 shutdown_connections 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.171+0000 7fdee4334700 1 -- 172.21.15.47:0/1203098220 wait complete. 2024-04-20T02:35:43.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.171+0000 7fdee4334700 1 librados: shutdown 2024-04-20T02:35:43.216 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:43.216 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-04-20T02:35:43.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-04-20T02:35:43.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.356+0000 7f866089c700 10 monclient: get_monmap_and_config 2024-04-20T02:35:43.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.356+0000 7f866089c700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.356+0000 7f866089c700 10 monclient: monmap: 2024-04-20T02:35:43.357 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.357 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.357 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:43.357801+0000 2024-04-20T02:35:43.357 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:43.357801+0000 2024-04-20T02:35:43.358 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.358 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.358 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:43.358 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:43.358 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.358 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.359 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.359 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.359 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.359 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.359 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.359 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:43.360 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:43.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 1 Processor -- start 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 1 -- start start 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 10 monclient: init 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 10 monclient: _add_conns ranks=[1,0,3,2,9,4,6,5,7,10,8] 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f865c137d90 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f865c0a09e0 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.357+0000 7f866089c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f865c13e3a0 0x7f865c140790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 10 monclient(hunting): picked mon.noname-g con 0x7f865c13e3a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0751e0 con 0x7f865c0a09e0 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c09dfc0 con 0x7f865c137d90 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c09fa40 con 0x7f865c13e3a0 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f866089c700 10 monclient(hunting): authenticate will time out at 2822.759081s 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f8659d9b700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f865c0a09e0 0x7f865c13cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57892/0 (socket says 172.21.15.47:57892) 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f8659d9b700 1 -- 172.21.15.47:0/4282679684 learned_addr learned my addr 172.21.15.47:0/4282679684 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 1 --2- 172.21.15.47:0/4282679684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f865c13e3a0 0x7f865c140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): get_auth_request con 0x7f865c13e3a0 auth_method 0 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91820154 0 0) 0x7f865c0751e0 con 0x7f865c0a09e0 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): _negotiate 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): my global_id is 104409 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86440035b0 con 0x7f865c0a09e0 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192290696 0 0) 0x7f865c09dfc0 con 0x7f865c137d90 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): _negotiate 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 10 monclient(hunting): my global_id is 56026 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c0751e0 con 0x7f865c137d90 2024-04-20T02:35:43.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient(hunting): handle_auth_done global_id 104376 payload 274 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 1 -- 172.21.15.47:0/4282679684 >> v1:172.21.15.179:6789/0 conn(0x7f865c137d90 legacy=0x7f865c138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 1 -- 172.21.15.47:0/4282679684 >> v1:172.21.15.47:6789/0 conn(0x7f865c0a09e0 legacy=0x7f865c13cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 1 monclient: found mon.noname-g 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 1 -- 172.21.15.47:0/4282679684 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c138880 con 0x7f865c13e3a0 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.358+0000 7f865ad9d700 1 --2- 172.21.15.47:0/4282679684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f865c13e3a0 0x7f865c140790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8654009dc0 tx=0x7f865400b330 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3fa499452003a32c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f865400e070 con 0x7f865c13e3a0 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: dump: 2024-04-20T02:35:43.382 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86540098f0 con 0x7f865c13e3a0 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 5 monclient: authenticate success, global_id 104376 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 1 -- 172.21.15.47:0/4282679684 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f86540128a0 con 0x7f865c13e3a0 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 4 set_mon_vals no callback set 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f865959a700 10 monclient: dump: 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.384 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 10 monclient: shutdown 2024-04-20T02:35:43.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 -- 172.21.15.47:0/4282679684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f865c13e3a0 msgr2=0x7f865c140790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 --2- 172.21.15.47:0/4282679684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f865c13e3a0 0x7f865c140790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8654009dc0 tx=0x7f865400b330 comp rx=0 tx=0).stop 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 -- 172.21.15.47:0/4282679684 shutdown_connections 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 --2- 172.21.15.47:0/4282679684 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f865c13e3a0 0x7f865c140790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 -- 172.21.15.47:0/4282679684 >> 172.21.15.47:0/4282679684 conn(0x7f865c0aa8b0 msgr2=0x7f865c0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 -- 172.21.15.47:0/4282679684 shutdown_connections 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.359+0000 7f866089c700 1 -- 172.21.15.47:0/4282679684 wait complete. 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.360+0000 7f866089c700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.360+0000 7f866089c700 10 monclient: monmap: 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.360+0000 7f866089c700 1 librados: starting msgr at 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.360+0000 7f866089c700 1 librados: starting objecter 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.360+0000 7f866089c700 1 Processor -- start 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 -- start start 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 librados: setting wanted keys 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 librados: calling monclient init 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient: init 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient: _add_conns ranks=[2,5,7,6,3,1,8,4,0] 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f865c0a09e0 0x7f865c1cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): picked mon.noname-b con 0x7f865c0a09e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c137d90 0x7f865c1cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): picked mon.noname-h con 0x7f865c137d90 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f865c13e3a0 0x7f865c09be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): picked mon.noname-i con 0x7f865c13e3a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:43.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0a34f0 con 0x7f865c0a09e0 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0a23c0 con 0x7f865c137d90 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f865c09e9e0 con 0x7f865c13e3a0 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f866089c700 10 monclient(hunting): authenticate will time out at 2822.762541s 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f865c0a09e0 0x7f865c1cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f865c0a09e0 0x7f865c1cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60180/0 (socket says 172.21.15.47:60180) 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 1 -- 172.21.15.47:0/424570056 learned_addr learned my addr 172.21.15.47:0/424570056 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): get_auth_request con 0x7f865c0a09e0 auth_method 0 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c137d90 0x7f865c1cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): get_auth_request con 0x7f865c137d90 auth_method 0 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f865c13e3a0 0x7f865c09be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): get_auth_request con 0x7f865c13e3a0 auth_method 0 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.361+0000 7f865a59c700 10 monclient(hunting): handle_auth_done global_id 114176 payload 824 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f865c13e3a0 msgr2=0x7f865c09be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f865c13e3a0 0x7f865c09be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c137d90 msgr2=0x7f865c1cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c137d90 0x7f865c1cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 monclient: found mon.noname-b 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009a70 con 0x7f865c0a09e0 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f865a59c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f865c0a09e0 0x7f865c1cc790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c520 tx=0x7f864c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 client.?.objecter ms_handle_connect 0x7f865c0a09e0 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f864c005540 con 0x7f865c0a09e0 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: dump: 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.392 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 5 monclient: authenticate success, global_id 114176 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c003680 con 0x7f865c0a09e0 2024-04-20T02:35:43.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 10 monclient: _renew_subs 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c09dc20 con 0x7f865c0a09e0 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f864c003c60 con 0x7f865c0a09e0 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f8658d99700 4 set_mon_vals no callback set 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 10 client.114176.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f864affd700 10 monclient: dump: 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.394 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 10 monclient: _renew_subs 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c0a19c0 con 0x7f865c0a09e0 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.362+0000 7f866089c700 1 librados: init done 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f866089c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f866089c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 0x7f865c09c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f866089c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f865c068180 con 0x7f865c06aac0 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f864c0037f0 con 0x7f865c0a09e0 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 0x7f865c09c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient: get_auth_request con 0x7f865c06aac0 auth_method 0 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f863c03a2f0 0x7f863c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f864c049420 con 0x7f865c0a09e0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 10 client.114176.objecter ms_dispatch 0x7f865c001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 3 client.114176.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 3 client.114176.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f864affd700 20 client.114176.objecter dump_active .. 0 homeless 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f865ad9d700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f863c03a2f0 0x7f863c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f865ad9d700 10 monclient: get_auth_request con 0x7f863c03a2f0 auth_method 0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 10 monclient(hunting): handle_auth_done global_id 84731 payload 824 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.363+0000 7f8659d9b700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f865c0a0000 con 0x7f865c06aac0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 client.114176.objecter ms_handle_connect 0x7f865c06aac0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f8659d9b700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 0x7f865c09c340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8650005910 tx=0x7f865000fb00 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 client.114176.objecter resend_mon_ops 2024-04-20T02:35:43.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f86500070d0 con 0x7f865c06aac0 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 monclient: dump: 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.398 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 client.114176.objecter ms_handle_connect 0x7f863c03a2f0 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f865ad9d700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f863c03a2f0 0x7f863c03c7b0 secure :-1 s=READY pgs=3405 cs=0 l=1 rev1=1 crypto rx=0x7f865c09e400 tx=0x7f8654011000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8650005b50 con 0x7f865c06aac0 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 msgr2=0x7f865c09c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 0x7f865c09c340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8650005910 tx=0x7f865000fb00 comp rx=0 tx=0).stop 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 msgr2=0x7f865c09c340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:43.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.364+0000 7f864affd700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c06aac0 0x7f865c09c340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f866089c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f866089c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 0x7f865c09c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f866089c700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f865c06a400 con 0x7f865c0a41b0 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 0x7f865c09c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient: get_auth_request con 0x7f865c0a41b0 auth_method 0 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.372+0000 7f8659d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f8659d9b700 10 monclient(hunting): handle_auth_done global_id 84740 payload 824 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f8659d9b700 1 -- 172.21.15.47:0/424570056 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f865c0a3a10 con 0x7f865c0a41b0 2024-04-20T02:35:43.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f8659d9b700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 0x7f865c09c880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8650005910 tx=0x7f86500010a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 client.114176.objecter ms_handle_connect 0x7f865c0a41b0 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 client.114176.objecter resend_mon_ops 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8650023070 con 0x7f865c0a41b0 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 monclient: dump: 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.401 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 <== mon.5 v2:172.21.15.179:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9354 (secure 0 0 0) 0x7f8650027410 con 0x7f865c0a41b0 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 msgr2=0x7f865c09c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 0x7f865c09c880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8650005910 tx=0x7f86500010a0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 msgr2=0x7f865c09c880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:43.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.373+0000 7f864affd700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f865c0a41b0 0x7f865c09c880 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8650005910 tx=0x7f86500010a0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:43.403 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.404 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.405 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:43.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.407 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.408 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:43.409 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.410 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.411 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.412 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.413 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.414 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.415 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:43.416 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.417 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:43.418 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.419 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.422 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.423 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": [ 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.424 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.425 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.425 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:43.425 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 10 librados: watch_flush enter 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 10 librados: watch_flush exit 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 20 client.114176.objecter shutdown clearing up homeless session... 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 10 client.114176.objecter successfully canceled tick 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f863c03a2f0 msgr2=0x7f863c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f863c03a2f0 0x7f863c03c7b0 secure :-1 s=READY pgs=3405 cs=0 l=1 rev1=1 crypto rx=0x7f865c09e400 tx=0x7f8654011000 comp rx=0 tx=0).stop 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 10 monclient: shutdown 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f865c0a09e0 msgr2=0x7f865c1cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 --2- 172.21.15.47:0/424570056 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f865c0a09e0 0x7f865c1cc790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c520 tx=0x7f864c00c830 comp rx=0 tx=0).stop 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f865ad9d700 1 -- 172.21.15.47:0/424570056 reap_dead start 2024-04-20T02:35:43.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f865ad9d700 1 -- 172.21.15.47:0/424570056 reap_dead start 2024-04-20T02:35:43.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 shutdown_connections 2024-04-20T02:35:43.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 >> 172.21.15.47:0/424570056 conn(0x7f865c0aa8b0 msgr2=0x7f865c133fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 shutdown_connections 2024-04-20T02:35:43.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 -- 172.21.15.47:0/424570056 wait complete. 2024-04-20T02:35:43.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.374+0000 7f866089c700 1 librados: shutdown 2024-04-20T02:35:43.428 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:43.428 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-04-20T02:35:43.428 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.609+0000 7f77b10ac700 10 monclient: get_monmap_and_config 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.609+0000 7f77b10ac700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.610+0000 7f77b10ac700 10 monclient: monmap: 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:43.611136+0000 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:43.611136+0000 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.610 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:43.611 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.610+0000 7f77b10ac700 1 Processor -- start 2024-04-20T02:35:43.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.610+0000 7f77b10ac700 1 -- start start 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.610+0000 7f77b10ac700 10 monclient: init 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient: _add_conns ranks=[9,7,8,1,3,6,5,10,0,2,4] 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77ac137d90 0x7f77ac0a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): picked mon.noname-a con 0x7f77ac137d90 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac138170 0x7f77ac0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): picked mon.noname-i con 0x7f77ac138170 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f77ac1385b0 0x7f77ac0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): picked mon.noname-e con 0x7f77ac1385b0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ac09fa40 con 0x7f77ac137d90 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac09dfc0 con 0x7f77ac138170 2024-04-20T02:35:43.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0a7bc0 con 0x7f77ac1385b0 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77b10ac700 10 monclient(hunting): authenticate will time out at 2823.012670s 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77ac137d90 0x7f77ac0a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f77ac1385b0 0x7f77ac0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f77ac137d90 0x7f77ac0a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53474/0 (socket says 172.21.15.47:53474) 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 1 -- 172.21.15.47:0/62069934 learned_addr learned my addr 172.21.15.47:0/62069934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): get_auth_request con 0x7f77ac137d90 auth_method 0 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): get_auth_request con 0x7f77ac1385b0 auth_method 0 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77aa59c700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac138170 0x7f77ac0a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.611+0000 7f77ab59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aa59c700 10 monclient(hunting): get_auth_request con 0x7f77ac138170 auth_method 0 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 10 monclient(hunting): handle_auth_done global_id 104418 payload 274 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 -- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f77ac1385b0 msgr2=0x7f77ac0a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f77ac1385b0 0x7f77ac0a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 -- 172.21.15.47:0/62069934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac138170 msgr2=0x7f77ac0a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac138170 0x7f77ac0a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 monclient: found mon.noname-a 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 -- 172.21.15.47:0/62069934 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0751e0 con 0x7f77ac137d90 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aa59c700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac138170 0x7f77ac0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77ab59e700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f77ac1385b0 0x7f77ac0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77aad9d700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ac137d90 0x7f77ac0a09e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f779c0098f0 tx=0x7f779c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2ede8cd5713b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 1 -- 172.21.15.47:0/62069934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f779c011aa0 con 0x7f77ac137d90 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: dump: 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.625 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77b10ac700 5 monclient: authenticate success, global_id 104418 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77b10ac700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 1 -- 172.21.15.47:0/62069934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c011c00 con 0x7f77ac137d90 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77b10ac700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77b10ac700 4 set_mon_vals no callback set 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 1 -- 172.21.15.47:0/62069934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f779c011d70 con 0x7f77ac137d90 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.612+0000 7f77a9d9b700 10 monclient: dump: 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.627 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 10 monclient: shutdown 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 -- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ac137d90 msgr2=0x7f77ac0a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ac137d90 0x7f77ac0a09e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f779c0098f0 tx=0x7f779c009c00 comp rx=0 tx=0).stop 2024-04-20T02:35:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 -- 172.21.15.47:0/62069934 shutdown_connections 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f77ac1385b0 0x7f77ac0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77ac137d90 0x7f77ac0a09e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 --2- 172.21.15.47:0/62069934 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac138170 0x7f77ac0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 -- 172.21.15.47:0/62069934 >> 172.21.15.47:0/62069934 conn(0x7f77ac0aa8b0 msgr2=0x7f77ac0a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 -- 172.21.15.47:0/62069934 shutdown_connections 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.613+0000 7f77b10ac700 1 -- 172.21.15.47:0/62069934 wait complete. 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 10 monclient: monmap: 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 1 librados: starting msgr at 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 1 librados: starting objecter 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 1 Processor -- start 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 1 -- start start 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 1 librados: setting wanted keys 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 1 librados: calling monclient init 2024-04-20T02:35:43.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.614+0000 7f77b10ac700 10 monclient: init 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient: _add_conns ranks=[7,4,5,8,3,0,1,6,2] 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac137d90 0x7f77ac1cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): picked mon.noname-i con 0x7f77ac137d90 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77ac138170 0x7f77ac136d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): picked mon.noname-c con 0x7f77ac138170 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f77ac1385b0 0x7f77ac1372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): picked mon.noname-h con 0x7f77ac1385b0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac1241f0 con 0x7f77ac138170 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac1cd070 con 0x7f77ac1385b0 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac124440 con 0x7f77ac137d90 2024-04-20T02:35:43.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77b10ac700 10 monclient(hunting): authenticate will time out at 2823.016199s 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77ac138170 0x7f77ac136d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac137d90 0x7f77ac1cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77ac138170 0x7f77ac136d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54764/0 (socket says 172.21.15.47:54764) 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 1 -- 172.21.15.47:0/2009751147 learned_addr learned my addr 172.21.15.47:0/2009751147 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): get_auth_request con 0x7f77ac138170 auth_method 0 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): get_auth_request con 0x7f77ac137d90 auth_method 0 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f77ac1385b0 0x7f77ac1372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): get_auth_request con 0x7f77ac1385b0 auth_method 0 2024-04-20T02:35:43.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77ab59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient(hunting): handle_auth_done global_id 114172 payload 824 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f77ac1385b0 msgr2=0x7f77ac1372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f77ac1385b0 0x7f77ac1372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77ac138170 msgr2=0x7f77ac136d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77ac138170 0x7f77ac136d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 monclient: found mon.noname-i 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a4009a70 con 0x7f77ac137d90 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aa59c700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f77ac138170 0x7f77ac136d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77aad9d700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac137d90 0x7f77ac1cc380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f779c016000 tx=0x7f779c004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77937fe700 10 client.?.objecter ms_handle_connect 0x7f77ac137d90 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.615+0000 7f77937fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f779c004490 con 0x7f77ac137d90 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: dump: 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.634 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c003b10 con 0x7f77ac137d90 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 5 monclient: authenticate success, global_id 114172 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 monclient: _renew_subs 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c009550 con 0x7f77ac137d90 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f779c003c80 con 0x7f77ac137d90 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 10 monclient: dump: 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.636 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77a959a700 4 set_mon_vals no callback set 2024-04-20T02:35:43.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 client.114172.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 monclient: _renew_subs 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.179:3303/0 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac09dc20 con 0x7f77ac137d90 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 1 librados: init done 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 0x7f77ac1208d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac09ea40 con 0x7f77ac06aac0 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.616+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f779c021070 con 0x7f77ac137d90 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 0x7f77ac1208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient: get_auth_request con 0x7f77ac06aac0 auth_method 0 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f779803a2f0 0x7f779803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f779c04b8d0 con 0x7f77ac137d90 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 client.114172.objecter ms_dispatch 0x7f77ac001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 3 client.114172.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 3 client.114172.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 20 client.114172.objecter dump_active .. 0 homeless 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77ab59e700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f779803a2f0 0x7f779803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77ab59e700 10 monclient: get_auth_request con 0x7f779803a2f0 auth_method 0 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 10 monclient(hunting): handle_auth_done global_id 114181 payload 824 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77ac09f760 con 0x7f77ac06aac0 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77aa59c700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 0x7f77ac1208d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77a0010d10 tx=0x7f77a000e4b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 client.114172.objecter ms_handle_connect 0x7f77ac06aac0 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 client.114172.objecter resend_mon_ops 2024-04-20T02:35:43.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77a000ec50 con 0x7f77ac06aac0 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77ab59e700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f779803a2f0 0x7f779803c7b0 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 crypto rx=0x7f77ac1cc8c0 tx=0x7f77a400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 monclient: dump: 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.617+0000 7f77937fe700 10 client.114172.objecter ms_handle_connect 0x7f779803a2f0 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f77a000edb0 con 0x7f77ac06aac0 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 msgr2=0x7f77ac1208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 0x7f77ac1208d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77a0010d10 tx=0x7f77a000e4b0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 msgr2=0x7f77ac1208d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:43.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.618+0000 7f77937fe700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac06aac0 0x7f77ac1208d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77b10ac700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-04-20T02:35:43.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77b10ac700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 0x7f77ac121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77b10ac700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac068180 con 0x7f77ac120e10 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 0x7f77ac121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient: get_auth_request con 0x7f77ac120e10 auth_method 0 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 10 monclient(hunting): handle_auth_done global_id 114190 payload 824 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.626+0000 7f77aa59c700 1 -- 172.21.15.47:0/2009751147 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f77ac124050 con 0x7f77ac120e10 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77aa59c700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 0x7f77ac121220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f77a0010d10 tx=0x7f77a0004750 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 client.114172.objecter ms_handle_connect 0x7f77ac120e10 2024-04-20T02:35:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 client.114172.objecter resend_mon_ops 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77a0016c30 con 0x7f77ac120e10 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 monclient: dump: 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 <== mon.7 v2:172.21.15.179:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9184 (secure 0 0 0) 0x7f77a0016d90 con 0x7f77ac120e10 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 msgr2=0x7f77ac121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 0x7f77ac121220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f77a0010d10 tx=0x7f77a0004750 comp rx=0 tx=0).stop 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 msgr2=0x7f77ac121220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T02:35:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.627+0000 7f77937fe700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac120e10 0x7f77ac121220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "peon", 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: "election_epoch": 120, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum": [ 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 0, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 1, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 2, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 3, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 4, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 5, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 6, 2024-04-20T02:35:43.647 INFO:teuthology.orchestra.run.smithi047.stdout: 7, 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: 8 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_age": 2, 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "required_con": "2449958758054445060", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "required_mon": [ 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.648 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_con": "4540138322906710015", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "quorum_mon": [ 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "outside_quorum": [], 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "extra_probe_peers": [], 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "sync_provider": [], 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "monmap": { 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 1, 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "72abca9e-d588-4ab7-bb3a-a08105100daf", 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-04-20T02:15:04.177248Z", 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release": 18, 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min_mon_release_name": "reef", 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "election_strategy": 3, 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "disallowed_leaders: ": "", 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false, 2024-04-20T02:35:43.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tiebreaker_mon": "", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_ranks: ": "", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "features": { 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "persistent": [ 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "kraken", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "luminous", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "mimic", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap-prune", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "nautilus", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "elector-pinging", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy", 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "reef" 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.651 INFO:teuthology.orchestra.run.smithi047.stdout: "optional": [] 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "mons": [ 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 0, 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "a", 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3300", 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.652 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789", 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6789/0", 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6789/0", 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 1, 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "f", 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.653 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3300", 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789", 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6789/0", 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6789/0", 2024-04-20T02:35:43.654 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 2, 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "b", 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.655 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3301", 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790", 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6790/0", 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 3, 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "g", 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3301", 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790", 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.657 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6790/0", 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 4, 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "c", 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.658 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3302", 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791", 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6791/0", 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.659 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 5, 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "h", 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3302", 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791", 2024-04-20T02:35:43.660 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6791/0", 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 6, 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "d", 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3303", 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792", 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6792/0", 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.662 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 7, 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "i", 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:3303", 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792", 2024-04-20T02:35:43.663 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.179:6792/0", 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "rank": 8, 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "e", 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-04-20T02:35:43.664 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:3304", 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793", 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 0 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6793/0", 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "priority": 0, 2024-04-20T02:35:43.665 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 0, 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_location": "{}" 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "feature_map": { 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": [ 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 1 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "client": [ 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T02:35:43.666 INFO:teuthology.orchestra.run.smithi047.stdout: "features": "0x3f01cfbffffdffff", 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: "release": "luminous", 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 2 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": false 2024-04-20T02:35:43.667 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 10 librados: watch_flush enter 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 10 librados: watch_flush exit 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 20 client.114172.objecter shutdown clearing up homeless session... 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 10 client.114172.objecter successfully canceled tick 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f779803a2f0 msgr2=0x7f779803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f779803a2f0 0x7f779803c7b0 secure :-1 s=READY pgs=3406 cs=0 l=1 rev1=1 crypto rx=0x7f77ac1cc8c0 tx=0x7f77a400b040 comp rx=0 tx=0).stop 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 10 monclient: shutdown 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac137d90 msgr2=0x7f77ac1cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77ab59e700 1 -- 172.21.15.47:0/2009751147 reap_dead start 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac137d90 0x7f77ac1cc380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f779c016000 tx=0x7f779c004750 comp rx=0 tx=0).stop 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 shutdown_connections 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 --2- 172.21.15.47:0/2009751147 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f77ac137d90 0x7f77ac1cc380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 >> 172.21.15.47:0/2009751147 conn(0x7f77ac0aa8b0 msgr2=0x7f77ac0a5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 shutdown_connections 2024-04-20T02:35:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 -- 172.21.15.47:0/2009751147 wait complete. 2024-04-20T02:35:43.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.628+0000 7f77b10ac700 1 librados: shutdown 2024-04-20T02:35:43.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-04-20T02:35:43.670 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-20T02:35:43.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-20T02:35:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.807+0000 7f5bf202e700 10 monclient: get_monmap_and_config 2024-04-20T02:35:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.807+0000 7f5bf202e700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.807+0000 7f5bf202e700 10 monclient: monmap: 2024-04-20T02:35:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.807 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:43.808623+0000 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:43.808623+0000 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:43.808 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 1 Processor -- start 2024-04-20T02:35:43.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 1 -- start start 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient: init 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient: _add_conns ranks=[2,6,8,7,0,10,1,9,5,4,3] 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec0a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient(hunting): picked mon.noname-b con 0x7f5bec0a0ad0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec0a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient(hunting): picked mon.noname-d con 0x7f5bec0a2800 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5bec0a31c0 0x7f5bec13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient(hunting): picked mon.noname-e con 0x7f5bec0a31c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.808+0000 7f5bf202e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec09ee40 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bf202e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bf202e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0a3620 con 0x7f5bec0a2800 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bf202e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bf202e700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec09ddb0 con 0x7f5bec0a31c0 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bf202e700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bf202e700 10 monclient(hunting): authenticate will time out at 2823.210028s 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec0a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec0a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec0a0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60186/0 (socket says 172.21.15.47:60186) 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec0a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56276/0 (socket says 172.21.15.47:56276) 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 1 -- 172.21.15.47:0/518871543 learned_addr learned my addr 172.21.15.47:0/518871543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): get_auth_request con 0x7f5bec0a0ad0 auth_method 0 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): get_auth_request con 0x7f5bec0a2800 auth_method 0 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5bec0a31c0 0x7f5bec13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): get_auth_request con 0x7f5bec0a31c0 auth_method 0 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bebfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient(hunting): handle_auth_done global_id 55581 payload 274 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 -- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5bec0a31c0 msgr2=0x7f5bec13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5bec0a31c0 0x7f5bec13e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 -- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 msgr2=0x7f5bec0a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec0a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 monclient: found mon.noname-d 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 -- 172.21.15.47:0/518871543 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0751e0 con 0x7f5bec0a2800 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beb7fe700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec0a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5beaffd700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec0a2c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009870 tx=0x7f5be0009b80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=563810cc9ccedfa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bea7fc700 1 -- 172.21.15.47:0/518871543 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5be0011aa0 con 0x7f5bec0a2800 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.809+0000 7f5bea7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: dump: 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.823 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 1 -- 172.21.15.47:0/518871543 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be0011c00 con 0x7f5bec0a2800 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 5 monclient: authenticate success, global_id 55581 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 1 -- 172.21.15.47:0/518871543 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5be0011d70 con 0x7f5bec0a2800 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 4 set_mon_vals no callback set 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bea7fc700 10 monclient: dump: 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.825 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 10 monclient: shutdown 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:43.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 -- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 msgr2=0x7f5bec0a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec0a2c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009870 tx=0x7f5be0009b80 comp rx=0 tx=0).stop 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 -- 172.21.15.47:0/518871543 shutdown_connections 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5bec0a31c0 0x7f5bec13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec0a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 --2- 172.21.15.47:0/518871543 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec0a2c80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 -- 172.21.15.47:0/518871543 >> 172.21.15.47:0/518871543 conn(0x7f5bec0a7420 msgr2=0x7f5bec0a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 -- 172.21.15.47:0/518871543 shutdown_connections 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.810+0000 7f5bf202e700 1 -- 172.21.15.47:0/518871543 wait complete. 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 10 monclient: build_initial_monmap 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 10 monclient: monmap: 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:43.827 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 1 librados: starting msgr at 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 1 librados: starting objecter 2024-04-20T02:35:43.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 1 Processor -- start 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 1 -- start start 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 1 librados: setting wanted keys 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 1 librados: calling monclient init 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.811+0000 7f5bf202e700 10 monclient: init 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient: _add_conns ranks=[2,6,0,1,4,8,5,3,7] 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec1cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): picked mon.noname-b con 0x7f5bec0a0ad0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec1ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): picked mon.noname-d con 0x7f5bec0a2800 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0a31c0 0x7f5bec1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): picked mon.noname-a con 0x7f5bec0a31c0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec1a8a00 con 0x7f5bec0a31c0 2024-04-20T02:35:43.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0720c0 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec1a8000 con 0x7f5bec0a2800 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bf202e700 10 monclient(hunting): authenticate will time out at 2823.213420s 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec1cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec1ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec1cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60200/0 (socket says 172.21.15.47:60200) 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 -- 172.21.15.47:0/4112728389 learned_addr learned my addr 172.21.15.47:0/4112728389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec1ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56286/0 (socket says 172.21.15.47:56286) 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): get_auth_request con 0x7f5bec0a0ad0 auth_method 0 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): get_auth_request con 0x7f5bec0a2800 auth_method 0 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0a31c0 0x7f5bec1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): get_auth_request con 0x7f5bec0a31c0 auth_method 0 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5bebfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient(hunting): handle_auth_done global_id 114194 payload 824 2024-04-20T02:35:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 -- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 msgr2=0x7f5bec1ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec1ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 -- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0a31c0 msgr2=0x7f5bec1a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0a31c0 0x7f5bec1a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 monclient: found mon.noname-b 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beb7fe700 1 -- 172.21.15.47:0/4112728389 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc009550 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.812+0000 7f5beaffd700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec1ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5beb7fe700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec1cc2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00ec10 tx=0x7f5bdc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 client.?.objecter ms_handle_connect 0x7f5bec0a0ad0 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5bdc00ccc0 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: dump: 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.834 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.835 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 5 monclient: authenticate success, global_id 114194 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 10 monclient: _renew_subs 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc00ce60 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be00094a0 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5bdc0138e0 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5be9ffb700 4 set_mon_vals no callback set 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 10 client.114194.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bd3fff700 10 monclient: dump: 2024-04-20T02:35:43.836 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:43.837 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 10 monclient: _renew_subs 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec1a9b30 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.813+0000 7f5bf202e700 1 librados: init done 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5bdc013470 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bf202e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bf202e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bf202e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec067770 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5bd803a2f0 0x7f5bd803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f5bdc04a7c0 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 10 client.114194.objecter ms_dispatch 0x7f5bec001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 3 client.114194.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 3 client.114194.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5bd3fff700 20 client.114194.objecter dump_active .. 0 homeless 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5beaffd700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5bd803a2f0 0x7f5bd803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.814+0000 7f5beaffd700 10 monclient: get_auth_request con 0x7f5bd803a2f0 auth_method 0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.815+0000 7f5beaffd700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5bd803a2f0 0x7f5bd803c7b0 secure :-1 s=READY pgs=3407 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009870 tx=0x7f5be0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.815+0000 7f5bd3fff700 10 client.114194.objecter ms_handle_connect 0x7f5bd803a2f0 2024-04-20T02:35:43.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.819+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f5bdc01b3b0 con 0x7f5bec0a0ad0 2024-04-20T02:35:43.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.819+0000 7f5bd3fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:43.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:43.819+0000 7f5bd3fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.130+0000 7f5bf202e700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-04-20T02:35:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.130+0000 7f5bf202e700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-04-20T02:35:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.130+0000 7f5bf202e700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:44.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.130+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f5bec1aa050 con 0x7f5bec0a0ad0 2024-04-20T02:35:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.132+0000 7f5bd3fff700 1 -- 172.21.15.47:0/4112728389 <== mon.2 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5bdc017020 con 0x7f5bec0a0ad0 2024-04-20T02:35:44.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.132+0000 7f5bd3fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-04-20T02:35:44.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.132+0000 7f5bd3fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:44.136 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=1,health=12,kv=11,logm=54} crc {auth=2313025440,config=3678701417,health=2809446585,kv=1530195301,logm=1670045139}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=1,health=12,kv=10,logm=54} crc {auth=4115909623,config=3678701417,health=1534934899,kv=2226345239,logm=4157308877}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=22,config=2,health=12,kv=11,logm=53} crc {auth=2234974715,config=692619927,health=2809446585,kv=1530195301,logm=2376871631}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=22,config=2,health=12,kv=11,logm=53} crc {auth=986569927,config=692619927,health=2809446585,kv=2154850828,logm=1287041940}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.137 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=21,config=2,health=11,kv=11,logm=55} crc {auth=1170047953,config=692619927,health=737836741,kv=1530195301,logm=2944644434}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=21,config=2,health=11,kv=10,logm=56} crc {auth=1668951628,config=692619927,health=3146732824,kv=3022316769,logm=1504515864}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=22,config=2,health=11,kv=11,logm=54} crc {auth=3222952827,config=692619927,health=3850398915,kv=2154850828,logm=37202924}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=1,health=10,kv=11,logm=55} crc {auth=4257936656,config=1228700967,health=994837828,kv=1530195301,logm=2164092124}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.132+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=23,config=2,health=12,kv=11,logm=52} crc {auth=1572251463,config=692619927,health=2809446585,kv=1530195301,logm=1438039913}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=5965322}) 2024-04-20T02:35:44.138 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=656526287}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=960291742}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3498978926}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4071446391}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2906990882}) 2024-04-20T02:35:44.139 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3069525743}) 2024-04-20T02:35:44.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=554528814}) 2024-04-20T02:35:44.140 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.135+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1482477669}) 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.134+0000 7f5bf202e700 10 librados: watch_flush enter 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.134+0000 7f5bf202e700 10 librados: watch_flush exit 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.134+0000 7f5bf202e700 20 client.114194.objecter shutdown clearing up homeless session... 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.134+0000 7f5bf202e700 10 client.114194.objecter successfully canceled tick 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5bd803a2f0 msgr2=0x7f5bd803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5bd803a2f0 0x7f5bd803c7b0 secure :-1 s=READY pgs=3407 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009870 tx=0x7f5be0008030 comp rx=0 tx=0).stop 2024-04-20T02:35:44.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 10 monclient: shutdown 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 msgr2=0x7f5bec1cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec1cc2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00ec10 tx=0x7f5bdc00ef20 comp rx=0 tx=0).stop 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 shutdown_connections 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5bd803a2f0 0x7f5bd803c7b0 unknown :-1 s=CLOSED pgs=3407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bec0a31c0 0x7f5bec1a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5bec0a0ad0 0x7f5bec1cc2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 --2- 172.21.15.47:0/4112728389 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5bec0a2800 0x7f5bec1ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 >> 172.21.15.47:0/4112728389 conn(0x7f5bec0a7420 msgr2=0x7f5bec0a5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 shutdown_connections 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 -- 172.21.15.47:0/4112728389 wait complete. 2024-04-20T02:35:44.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:44.135+0000 7f5bf202e700 1 librados: shutdown 2024-04-20T02:35:44.155 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-04-20T02:35:44.457 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.457 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.457 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2648390106}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2982129355}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=406058197}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3910377445}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=519172973}) 2024-04-20T02:35:44.458 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351346599}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=587714673}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2073292450}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.138+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3750501473}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1109220307}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=47902615}) 2024-04-20T02:35:44.459 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2880380970}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1068919848}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3816300466}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3234433764}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=732996306}) 2024-04-20T02:35:44.460 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072454151}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.140+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3664460458}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2856363563}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3974475867}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2109356018}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.461 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2837725374}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3413634759}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=583815450}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=480338786}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1919798142}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.143+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4250438678}) 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.462 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4212564964}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2231427718}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1942220091}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2989336338}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3471568894}) 2024-04-20T02:35:44.463 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=346695447}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2554462201}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1810341456}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.145+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2161638079}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3111682029}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3069132744}) 2024-04-20T02:35:44.464 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3675533390}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1383907081}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4153197097}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3891955866}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1918151116}) 2024-04-20T02:35:44.465 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1445383526}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.147+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194552191}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.148+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.148+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1925451665}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1578711842}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3134071628}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.466 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1705987715}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=644178661}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3716813503}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2162614247}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1902905366}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.149+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2731773004}) 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.467 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2740192257}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3522834937}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=404975004}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=643113616}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2658458855}) 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.468 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=447304176}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2304669191}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2005406839}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.150+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1278677080}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2221581403}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2505505625}) 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.469 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4188096077}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2423420022}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=218576085}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2644989831}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1155050856}) 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.470 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3353044606}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.152+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=561452994}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=106735665}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4213883407}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1262157505}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.471 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2464472814}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2530003517}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2103571112}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1221608}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540638771}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.154+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=496485224}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.155+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.155+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.472 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.155+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1401098198}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1126375516}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=565827968}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1112677009}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4054836362}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.473 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1999515574}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=999642732}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3274761739}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.156+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2630277946}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3095120861}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=652409189}) 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.474 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3694298685}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=205344734}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3950318132}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=482515302}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1187014377}) 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.475 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750269024}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.157+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3523523794}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2193016327}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=153610584}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1635226284}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3568093161}) 2024-04-20T02:35:44.476 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2924266929}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2329509304}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2061037320}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2286421136}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.159+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3646987120}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2626989368}) 2024-04-20T02:35:44.477 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2360799579}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1931035221}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4062117957}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2220263678}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.478 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2757528643}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3251446460}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903482049}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.160+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1528948845}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3469930157}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2144260242}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.479 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2268448102}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=939245074}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4066247122}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.162+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1966088556}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.163+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.163+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.163+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2699128738}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.163+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.163+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643267686}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.163+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=527841303}) 2024-04-20T02:35:44.480 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691395340}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3674388021}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1202983986}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3451266556}) 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.481 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1480078547}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3592629454}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1832175476}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=390993135}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.164+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4139526278}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.165+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=364142351}) 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.482 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2935998913}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2794673254}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2229262886}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=351042935}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2961490663}) 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.483 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3708801152}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3378545450}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.166+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2087884584}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=129738528}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3436591934}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3617941178}) 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.484 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1798336195}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2543752545}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2202777461}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963982026}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1789831578}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.168+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3070669315}) 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.485 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3409524052}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2750825810}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2037769424}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3264356740}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922942008}) 2024-04-20T02:35:44.486 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.169+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929467132}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1415777819}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4053509256}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.170+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1239463249}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160133453}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2430552116}) 2024-04-20T02:35:44.487 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3923592809}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=381121756}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3391588170}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2145253324}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3667426426}) 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.488 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583612350}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.171+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1498015331}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338736871}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3092579299}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2620825600}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2450341905}) 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.489 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4033826523}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3886787087}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2056043971}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4232617729}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.173+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2029752518}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3104308065}) 2024-04-20T02:35:44.490 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2114807545}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=598987737}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3262239889}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3212436404}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3230451767}) 2024-04-20T02:35:44.491 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2065173624}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1172076447}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.174+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=421013860}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2389041130}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1242173954}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=521371018}) 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.492 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=835801193}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=878817018}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2607534170}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3460350562}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3001919365}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.176+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=459879297}) 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.493 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3656479859}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1068395658}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1481226679}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3736310576}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3590200158}) 2024-04-20T02:35:44.494 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3596494196}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=199056018}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1021261310}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.177+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2598214570}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=578256993}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2124583999}) 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.495 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3425144692}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1170336906}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=739708229}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=713874185}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1725592020}) 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.496 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2762302335}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.179+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=343786408}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952968976}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=731045810}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4188423069}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1718284865}) 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.180+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.497 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691644372}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2288943452}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2148668872}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3685768027}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.181+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=789537313}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2911838120}) 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.498 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=400834656}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4166523464}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1212465575}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1534375530}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.499 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1991148568}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4121224832}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2747732635}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.182+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3556244402}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3640434496}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3237426831}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.183+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2882176520}) 2024-04-20T02:35:44.500 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3080757463}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2968708080}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=249244338}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2737708489}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603148648}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.184+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=618482618}) 2024-04-20T02:35:44.501 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2349283972}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739824459}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1285498709}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=443135567}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1839573490}) 2024-04-20T02:35:44.502 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293327417}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2211926005}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029825736}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.185+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1036176235}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2220391127}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.503 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1375698944}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1681527634}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.186+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1743061098}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3234739521}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2553434665}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.504 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1501883848}) 2024-04-20T02:35:44.505 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.505 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3298046085}) 2024-04-20T02:35:44.505 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.187+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=357140794}) 2024-04-20T02:35:44.505 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.505 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.505 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1928556984}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1659615464}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1750637360}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3247191967}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.506 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961431156}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=305915510}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1479402941}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2919470178}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.188+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741187033}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=323306987}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.507 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2687367506}) 2024-04-20T02:35:44.508 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.508 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.508 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=176862179}) 2024-04-20T02:35:44.508 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.508 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.508 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1834287721}) 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=629003507}) 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2440424562}) 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.509 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1351256380}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.189+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.190+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665339239}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.190+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4097824476}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3763746375}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3885508136}) 2024-04-20T02:35:44.510 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=362634088}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=815923454}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2588381663}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=645100777}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=816586467}) 2024-04-20T02:35:44.511 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072443030}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.191+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1988353440}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.192+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.192+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.192+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3409580958}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.192+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.192+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2245411505}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3215841838}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3598680219}) 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.512 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2881280580}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1009863696}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=679876833}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3148398616}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.193+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=573084487}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2376170887}) 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.513 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3660057136}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=736197891}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3483496884}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2314328197}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3250696956}) 2024-04-20T02:35:44.514 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1550578890}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=712457384}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.194+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2032682836}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2463677201}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=793934162}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=975307472}) 2024-04-20T02:35:44.515 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2902313125}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2306185284}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2779245257}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.195+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.196+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=745881436}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.196+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.196+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=706254427}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.196+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3720786738}) 2024-04-20T02:35:44.516 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=341277850}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=822590695}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4038274084}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=239312690}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3158584725}) 2024-04-20T02:35:44.517 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3772450663}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1878639363}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3071857803}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.197+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3055064803}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1204362351}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2193060868}) 2024-04-20T02:35:44.518 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3679714689}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2281700308}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1364468797}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1329528637}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.519 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3968363381}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=561235959}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.198+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2183599755}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=974806174}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3149555275}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2391961354}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.520 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=638756360}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3139005764}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=147092726}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=411974776}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3287643222}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.200+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2805175858}) 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.521 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2602939174}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2725919079}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.201+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4087979165}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=425153543}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99645961}) 2024-04-20T02:35:44.522 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2552999687}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=894644507}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351448427}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.202+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4231106827}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586226995}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.523 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1938897775}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3486321973}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2324217668}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2973946134}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2126680105}) 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.524 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3067893182}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2523521922}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.203+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3514127645}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1147499103}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2814291273}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2750347718}) 2024-04-20T02:35:44.525 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3907881800}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=515515669}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3055897179}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2794545093}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1538207673}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.204+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2169467910}) 2024-04-20T02:35:44.526 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3144069546}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2101698069}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2732044441}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1545634828}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2885220172}) 2024-04-20T02:35:44.527 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4191852676}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3389509777}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079925954}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.206+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4128714288}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=274599415}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.528 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1027275332}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.207+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=673427158}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1041431960}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3348070599}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=687242552}) 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.529 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2172429024}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1257223286}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.208+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1016160988}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2168076474}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=446326644}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=154270677}) 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.530 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1494342614}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3316150430}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3725474678}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1288562890}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264152076}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.209+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3244403374}) 2024-04-20T02:35:44.531 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.210+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.210+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.210+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=56229229}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2511248988}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=396875371}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3829578691}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3193617390}) 2024-04-20T02:35:44.532 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3480351581}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=172057443}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043869560}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.211+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4057925362}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=608338205}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.533 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4110550730}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=121089293}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=19752892}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3109466389}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1435734145}) 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.534 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2807800013}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2400690658}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.212+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3966664840}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1672313496}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=378246555}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.213+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2968025527}) 2024-04-20T02:35:44.535 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2504373530}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2283830804}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2280807622}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2704234619}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1367804689}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.214+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2181865771}) 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.536 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3285179422}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4228706487}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3988871785}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1568207490}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=762337764}) 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.537 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3031564129}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3752725839}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1080160156}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.215+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=154232320}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=345009821}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=902729903}) 2024-04-20T02:35:44.538 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=367856720}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1359250591}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3649304060}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.216+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=589229055}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2241423191}) 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.539 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3079227025}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.217+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940575177}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3061826293}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1870468241}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1678616946}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1085731529}) 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.540 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4202151503}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3051683460}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=599108666}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2598840685}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.218+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246024551}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809025594}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.541 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=228385135}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1256193714}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3983342269}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1287354203}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3452875720}) 2024-04-20T02:35:44.542 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2254822182}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=76612891}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.220+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=564209660}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2129007209}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3142536927}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4091304332}) 2024-04-20T02:35:44.543 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4037168747}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2166503449}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=983072127}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1475172657}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2908161899}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.222+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1589422368}) 2024-04-20T02:35:44.544 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1699387873}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2964132736}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2351516200}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.223+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1988974765}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2564618461}) 2024-04-20T02:35:44.545 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=796142645}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=461320253}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838112130}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.224+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2155253143}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1904390139}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3701976980}) 2024-04-20T02:35:44.546 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=136606606}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2431116614}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1815101185}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2052561088}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.547 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2848131687}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3642960379}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.225+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4282941863}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4153795576}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2009104291}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2345586095}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.548 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=539598565}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2269227548}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=524563932}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2459321848}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1878476637}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.227+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1036374925}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3725955472}) 2024-04-20T02:35:44.549 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302622696}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2214254413}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2293096944}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=674307779}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2336729645}) 2024-04-20T02:35:44.550 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2255761721}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3094174455}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.228+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1693987307}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2309523702}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1590897544}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.551 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2118102911}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2701653006}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=788222692}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2608919637}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3622719291}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2931089868}) 2024-04-20T02:35:44.552 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.230+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3012688597}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1185557177}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1777425112}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=597854453}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=152314142}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2128620571}) 2024-04-20T02:35:44.553 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1224587268}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4003491614}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629815484}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.231+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2061064094}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.232+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=284485160}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2237959583}) 2024-04-20T02:35:44.554 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2793183955}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2914976901}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1361545081}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2573806064}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2763311617}) 2024-04-20T02:35:44.555 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933032494}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.233+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3832656217}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1157811841}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3343083938}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1310994313}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2333301935}) 2024-04-20T02:35:44.556 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2704926757}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=404571697}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2552042102}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178629879}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.234+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3903764298}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.235+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3386276434}) 2024-04-20T02:35:44.557 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3908934814}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2449821123}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3770267834}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=873322694}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.558 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3611543746}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3246929159}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1138070830}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.236+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2691002874}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2993707279}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1696217552}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1342527706}) 2024-04-20T02:35:44.559 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.237+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3761747467}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=214293483}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525831002}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=822253420}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1331515775}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.238+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2984048010}) 2024-04-20T02:35:44.560 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=623061705}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1307701016}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2088558104}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1790838826}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1480820474}) 2024-04-20T02:35:44.561 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2780879825}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=395569698}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4035106663}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.239+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=559960460}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3283997653}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=640069035}) 2024-04-20T02:35:44.562 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3890105005}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2733903223}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1930480028}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=520351106}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2741958366}) 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.563 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1483098378}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.241+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2372420925}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2622151332}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2031619625}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1400587045}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2033650880}) 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.564 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2688974818}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3582320784}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4208488755}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1502785221}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.242+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=292414114}) 2024-04-20T02:35:44.565 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=82087666}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=683836794}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=268753990}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2466752393}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2392906563}) 2024-04-20T02:35:44.566 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1166441029}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2604184216}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=468254363}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.244+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3579126661}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440043554}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.567 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3395836669}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3469797387}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1458123540}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1795485102}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=921018483}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=521220718}) 2024-04-20T02:35:44.568 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=409179247}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.245+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1390048837}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1027206594}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=723820328}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1431302843}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.569 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1342642828}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1923743326}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1427004235}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1512137213}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529056732}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.247+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=359738941}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.570 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186696526}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3497208902}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2773967838}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=511216383}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=719633844}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.571 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2584721059}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.248+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3557622741}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.249+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.249+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1391782507}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.249+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=780286915}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989471216}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1900849236}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.572 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3903326568}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2925335199}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855482392}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608908324}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1701676895}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2824941321}) 2024-04-20T02:35:44.573 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.250+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1337286575}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.251+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.251+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2288200336}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2720039426}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=361440748}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=481706388}) 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.574 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=682536382}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4122438722}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4134876338}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3038800608}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.252+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3271611337}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3728099741}) 2024-04-20T02:35:44.575 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1298431332}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=710794664}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=736294099}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2275023892}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=322556235}) 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.576 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2861122301}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3101542185}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.253+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3638169493}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2083893523}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1985810957}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=794972375}) 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.577 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3442608990}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1076857611}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=970872413}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2638048032}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3909876960}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.255+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2057269391}) 2024-04-20T02:35:44.578 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2275941626}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1133279947}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=437129642}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2230818573}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2657262910}) 2024-04-20T02:35:44.579 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3003180668}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3522737419}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4215621264}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.256+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256373144}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=934460000}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2400739471}) 2024-04-20T02:35:44.580 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=889443656}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=446845824}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=664073970}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2988157696}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1329103888}) 2024-04-20T02:35:44.581 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2436324693}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.258+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2625490353}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1461322194}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3770285349}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2781466623}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.582 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2559589809}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1411489657}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1588641970}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=783920733}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452232832}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.259+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4197106941}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.583 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=284141161}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3391310896}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=870793257}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2474840009}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2857783981}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.584 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3946473447}) 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3512670541}) 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585876128}) 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.261+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1320520025}) 2024-04-20T02:35:44.585 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546901909}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2211407481}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2316784793}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1330600572}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.586 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3672062206}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2772920377}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2196339752}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.262+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2347408029}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.263+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=298297634}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=755895195}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.587 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3129278723}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1768953689}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=470487426}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1801748966}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2166567087}) 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.588 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1866045784}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5207286}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.264+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1520223213}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3404690257}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1078733225}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3528172585}) 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.589 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2807613243}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1218869226}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.265+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2370301091}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=534141048}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4281264170}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.266+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2659930077}) 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.590 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=257581373}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1889773770}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=782436390}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=969292110}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3412649860}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.267+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.591 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=934343088}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=803395712}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3268274359}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.268+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935874790}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4275933331}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4179087306}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.592 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3102122561}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1599057029}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=145178005}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1326067001}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1921042876}) 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.593 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=83588825}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.269+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1365692321}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=269698072}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3215298066}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=858523646}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2094508274}) 2024-04-20T02:35:44.594 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1177386899}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=845938214}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1521611176}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2122723993}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.271+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4006435042}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936263165}) 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.595 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3436830310}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=346019440}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=473431714}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=462460379}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3266475976}) 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.596 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2947281191}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=887762421}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.273+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2382132674}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2676553964}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2431801604}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1625220225}) 2024-04-20T02:35:44.597 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2288488702}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1811724932}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=711665016}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.274+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.275+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.275+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2763366377}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.275+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.275+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2320808819}) 2024-04-20T02:35:44.598 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.275+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2019059595}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1967287967}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1787191053}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2845813407}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1640760658}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.599 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3573038167}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=775621468}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3077437524}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=508434291}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.276+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1400422476}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=675669863}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2384772182}) 2024-04-20T02:35:44.600 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=273696156}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=32810412}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2291288091}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.277+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4225673108}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.601 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=309886688}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551813918}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.278+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1282042419}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2254804815}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3586751913}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2400084975}) 2024-04-20T02:35:44.602 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3935038180}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2348443807}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3849610005}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=51796572}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1663311174}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.279+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1323981407}) 2024-04-20T02:35:44.603 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4096518782}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2585812146}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.280+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3520534656}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=256202784}) 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.604 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3168117298}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=641284453}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4171904809}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1978552172}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.281+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1192837018}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3778116443}) 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.605 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1002001046}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3424753790}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3178492402}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.282+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1754242287}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3338976746}) 2024-04-20T02:35:44.606 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4037768392}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4237899781}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.283+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2022749023}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=831432363}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1916459218}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.607 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1179809255}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1406410827}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2400981734}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3126365419}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=110304945}) 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.608 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.284+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1804334187}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.285+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848366939}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1317869202}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1929486151}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3827059849}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3352945518}) 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.609 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2950971489}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3223692205}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=416591373}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321031967}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.286+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2575612248}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493946266}) 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.610 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1730664634}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=216486527}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=583685806}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4285764501}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2545997306}) 2024-04-20T02:35:44.611 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1131370033}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213924267}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.288+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1599988755}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3770671355}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4048518873}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.612 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1269986569}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4054991101}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2309034538}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1288893597}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3460743740}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1414615613}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.290+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=230806818}) 2024-04-20T02:35:44.613 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.291+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.291+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4146252695}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1829597758}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2501803028}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3652706477}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.614 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=600598150}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3093280579}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2118419302}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=715716078}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.292+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2366792075}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2638552758}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.615 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2675559776}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1726932923}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.293+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=596953192}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3730462085}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=368111842}) 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.616 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2309621933}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162898016}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.294+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3639560476}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1282190581}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2706343588}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3197934687}) 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.617 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3387785364}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2965966765}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3583130827}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1971371873}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3634361357}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.295+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=723453918}) 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.618 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=440100706}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2822844755}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2328302096}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1848324459}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2618232761}) 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.619 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3332238096}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4056123556}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774595165}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.297+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3609636943}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2724885593}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3363498378}) 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.620 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1562436098}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=964340907}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3791964202}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3000987029}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=375183534}) 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.621 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3053107964}) 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.299+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2323257241}) 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.300+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=99791337}) 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.622 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=480917075}) 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3644932395}) 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1483830949}) 2024-04-20T02:35:44.623 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.624 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.624 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2927471983}) 2024-04-20T02:35:44.624 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.624 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.624 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=231131120}) 2024-04-20T02:35:44.624 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=114590657}) 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3798480830}) 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.301+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1966012417}) 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.625 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.626 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3931367942}) 2024-04-20T02:35:44.626 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.626 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.626 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2423793501}) 2024-04-20T02:35:44.626 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.626 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=865004514}) 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3995206775}) 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.627 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1565282693}) 2024-04-20T02:35:44.628 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.302+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.628 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.628 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3646284395}) 2024-04-20T02:35:44.628 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.628 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.628 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=811181434}) 2024-04-20T02:35:44.629 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.629 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2262520827}) 2024-04-20T02:35:44.629 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.303+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=763527625}) 2024-04-20T02:35:44.629 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.629 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.629 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=337367473}) 2024-04-20T02:35:44.630 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.630 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.630 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1736698526}) 2024-04-20T02:35:44.630 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.630 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.630 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3176580065}) 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3786140762}) 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2422170584}) 2024-04-20T02:35:44.631 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2194749045}) 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=303682474}) 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2125162974}) 2024-04-20T02:35:44.632 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.305+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3159042241}) 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.306+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986126946}) 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3226634513}) 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.633 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2713277576}) 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=346763115}) 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3112302305}) 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.634 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4111111368}) 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4188050519}) 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434879080}) 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.307+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=139835128}) 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.635 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4150155434}) 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2648224012}) 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4148695908}) 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.636 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2855355204}) 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=398449828}) 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4028646867}) 2024-04-20T02:35:44.637 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=599440403}) 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.308+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3563031217}) 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.309+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1620436845}) 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.638 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1709829358}) 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1483961929}) 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4020155730}) 2024-04-20T02:35:44.639 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3135456250}) 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3320241727}) 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3826958556}) 2024-04-20T02:35:44.640 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=367816829}) 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1643433524}) 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.310+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1096667777}) 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.641 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1508560563}) 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4245998588}) 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2461622198}) 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.642 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2314229033}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.311+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4051058010}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=956574540}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.643 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2360937833}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3619465052}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.312+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3413627025}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3606294773}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=769446646}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=474680952}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3655835861}) 2024-04-20T02:35:44.644 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=384182557}) 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3830693243}) 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1541136593}) 2024-04-20T02:35:44.645 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1737138428}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.313+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3320777978}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207260962}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2008253415}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.646 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3478953826}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1576141836}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3485276291}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2488231869}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2481569542}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=338060986}) 2024-04-20T02:35:44.647 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.315+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1976425915}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=210164453}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2660521420}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2151652827}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1473085627}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.648 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855861403}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2088897795}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=109537374}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2474191069}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.317+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3323549632}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.318+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3914238100}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1798832606}) 2024-04-20T02:35:44.649 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1712292034}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2420917748}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2855689277}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2819056441}) 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.650 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3300107642}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3070009281}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.319+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3144111136}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=530980503}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3996402993}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3891374532}) 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.651 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2978651075}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897732581}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.320+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=673037378}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2075389055}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3660932332}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.321+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3693958048}) 2024-04-20T02:35:44.652 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=354518619}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696520875}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1530668218}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=520244515}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1310651942}) 2024-04-20T02:35:44.653 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1870927871}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3286000092}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1017699507}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.322+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1139046597}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.323+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.323+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.323+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=691727172}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=825904681}) 2024-04-20T02:35:44.654 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1914923305}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2767865856}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=236441932}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=559664378}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.655 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=534254969}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=711071908}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.324+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2977616648}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4263084143}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=909287487}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=569241666}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.656 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=70255792}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1327195111}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1690789908}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1057480185}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081580916}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.326+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4143357694}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.657 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3326881378}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1888265568}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1200494727}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.327+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2461961446}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2386212076}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.658 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1624440910}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3358370979}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2388313279}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.328+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=912385827}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3388383330}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1200607607}) 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.659 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=469167414}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3876270199}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2625000274}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2843779308}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1882307618}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=350928831}) 2024-04-20T02:35:44.660 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.329+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3244575593}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.330+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.330+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.330+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2151086147}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.330+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.330+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2250495512}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3663832722}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2880918047}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.661 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2263413228}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=150902727}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1819392808}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2767042106}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.331+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=219734423}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2745878136}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.662 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4272308576}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1099758431}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4210361838}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3305166519}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3824479332}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.663 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3271075314}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3495610273}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.332+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3277486640}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3477714659}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2466607983}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=445342227}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.664 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2435841157}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1554080815}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2281170877}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4045960549}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=519862368}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.334+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1029128738}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.665 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2256275503}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2793320353}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3516323351}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4084738582}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=14355358}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.666 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.335+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1783826982}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.336+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.336+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.336+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2919885557}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.336+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.336+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=949753508}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.336+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2219256478}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2904593551}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2955362086}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.667 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=893243182}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1495805816}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=778857022}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3198311383}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2616406006}) 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.668 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811090390}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.337+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1050518535}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=392634002}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1982827772}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.338+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=921396624}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=848964967}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.669 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1342967763}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2598624740}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3516064915}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=161721689}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.339+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1284939349}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2557662450}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.670 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4168093823}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3681369015}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=355064168}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4038839543}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1963194155}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.671 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=629322735}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.340+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.341+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165809530}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.341+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=673618256}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2206930494}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1141033873}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1728698507}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.672 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3001074596}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3812440756}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3767133087}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4138422249}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=426003104}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.342+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3822938613}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.673 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2956040223}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2990726865}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1500154593}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2932187977}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773580650}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.674 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3497965449}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3357484686}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=334979153}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.344+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=433587011}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1403401321}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2516677775}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.345+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.675 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3116338224}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2973834686}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610782476}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2486332773}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1434149326}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239037577}) 2024-04-20T02:35:44.676 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.346+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1530901022}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=496140604}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3414206818}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.347+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4002792770}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1071794290}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.677 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3363489138}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1981505403}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1762335205}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=211650512}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.348+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3000333428}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1321474146}) 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.678 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2723972623}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2514720097}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.349+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2504644547}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200383299}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2500985338}) 2024-04-20T02:35:44.679 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3574215821}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1986166496}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.350+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357339593}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.351+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=332637989}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2629877834}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=842221155}) 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.680 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2061490181}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=140668278}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=519758699}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=649224682}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=440821877}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.352+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419770796}) 2024-04-20T02:35:44.681 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693548433}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1110526773}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3553344975}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1060174602}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=141695632}) 2024-04-20T02:35:44.682 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3887218742}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.353+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3461798018}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.354+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.354+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2371677535}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.354+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2346346107}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2242038177}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2802130876}) 2024-04-20T02:35:44.683 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3347058416}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=204208507}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=28330767}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3506404575}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1108776985}) 2024-04-20T02:35:44.684 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1766375625}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.355+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1925039893}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11,monmap=3,osd_metadata=3} crc {logm=1021461280,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=3333834473,mon_config_key=2868358642,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=62,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=9,monmap=3,osd_metadata=3} crc {logm=844988590,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=2219364305,mon_config_key=757213125,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=63,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=3} crc {logm=4238729726,mdsmap=1531151339,mgr=2537084382,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=4092888292,mon_config_key=2868358642,monmap=3385036551}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=65,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=11,monmap=1} crc {logm=337973001,mdsmap=3585761405,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=2868358642,monmap=1228700967}) 2024-04-20T02:35:44.685 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=63,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11,monmap=3,osd_metadata=2} crc {logm=2142486711,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1990318260,mon_config_key=2868358642,monmap=3385036551,osd_metadata=242894556}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=64,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=2,osd_metadata=1} crc {logm=3727931296,mdsmap=1531151339,mgr=1417024152,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=759064529,monmap=3140514009,osd_metadata=3230729490}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=62,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=2} crc {logm=3328132623,mdsmap=2147943554,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=1581589272,mon_config_key=4071561259,monmap=3385036551,osd_metadata=430068545}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10,monmap=3,osd_metadata=3} crc {logm=1080098509,mdsmap=1531151339,mgr=1787245756,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=682623885,mon_config_key=1997154721,monmap=3385036551,osd_metadata=3480465}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.357+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=64,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,osd_metadata=1} crc {logm=3157294016,mdsmap=1531151339,mgr=2537084382,mgr_command_descs=470045235,mgr_metadata=2530826944,mgrstat=708344924,mon_config_key=1169884507,monmap=3385036551,osd_metadata=3230729490}) 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.686 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=2,osd_pg_creating=1,osd_snap=39,osdmap=58} crc {osd_metadata=1723635538,osd_pg_creating=2876805438,osd_snap=1850961888,osdmap=766842178}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=3,osd_snap=41,osdmap=56} crc {osd_metadata=1419339597,osd_snap=1404655098,osdmap=468551845}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=42,osdmap=54} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=1430979940,osdmap=2365477899}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_metadata=2,osd_pg_creating=1,osd_snap=42,osdmap=55} crc {osd_metadata=1723635538,osd_pg_creating=2876805439,osd_snap=80284358,osdmap=1919869715}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_metadata=2,osd_pg_creating=1,osd_snap=41,osdmap=56} crc {osd_metadata=3739743497,osd_pg_creating=2876805438,osd_snap=2949137454,osdmap=306550204}) 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.687 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=40,osdmap=56} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=949217530,osdmap=2530887620}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=43,osdmap=53} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=2847173898,osdmap=3740818708}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.359+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.360+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=771303872,osdmap=1714519247}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.360+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=43,osdmap=53} crc {osd_metadata=1419339597,osd_pg_creating=2876805438,osd_snap=249402865,osdmap=3775309879}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3434642054}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=217922116}) 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.688 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2556917045}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3826842798}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3052351337}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1846662698}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3760674229}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1201372257}) 2024-04-20T02:35:44.689 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.361+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3444004291}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2046714304}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=50760706}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2471729763}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1225741923}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.690 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2487439362}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1028885290}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1197851676}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1880177466}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.363+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1980673176}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.364+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.364+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.364+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2170988880}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.691 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=669889107}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1483987272}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=938617220}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3316363015}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.692 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3003539980}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1460385772}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1963608690}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.365+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2225348377}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=625854687}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=978010829}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.693 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4147428142}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1572286042}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3619397755}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4259611003}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1261003693}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2447176378}) 2024-04-20T02:35:44.694 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.367+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4001609822}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2401767482}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3925603359}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=56510468}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=941460324}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.695 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=270670152}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=500946103}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=533983343}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1589859373}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.370+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=99309366}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=88} crc {osdmap=3932649732}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.696 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=88} crc {osdmap=1925607978}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=86} crc {osdmap=1388736854}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=86} crc {osdmap=1877678512}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=86} crc {osdmap=3054315658}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=86} crc {osdmap=1835172266}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.697 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=87} crc {osdmap=3137152809}) 2024-04-20T02:35:44.698 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-04-20T02:35:44.698 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=85} crc {osdmap=3097702356}) 2024-04-20T02:35:44.698 INFO:tasks.ceph.mon.f.smithi179.stderr:2024-04-20T02:35:44.381+0000 7feee185a700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=81} crc {osdmap=4063694609}) 2024-04-20T02:35:45.157 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-04-20T02:35:45.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.301+0000 7f5d61a14700 10 monclient: get_monmap_and_config 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.301+0000 7f5d61a14700 10 monclient: build_initial_monmap 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.301+0000 7f5d61a14700 10 monclient: monmap: 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:45.302961+0000 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:45.302961+0000 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:45.302 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 1 Processor -- start 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 1 -- start start 2024-04-20T02:35:45.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 10 monclient: init 2024-04-20T02:35:45.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:45.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 10 monclient: _add_conns ranks=[4,2,3,9,1,6,8,7,0,5,10] 2024-04-20T02:35:45.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a4c80 0x7f5d5c0a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 10 monclient(hunting): picked mon.noname-c con 0x7f5d5c0a4c80 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:45.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.302+0000 7f5d61a14700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 0x7f5d5c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): picked mon.noname-b con 0x7f5d5c0a0d20 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5d5c0a55a0 0x7f5d5c137a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): picked mon.noname-g con 0x7f5d5c0a55a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0a0000 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0a6a10 con 0x7f5d5c0a55a0 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c09e440 con 0x7f5d5c0a4c80 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d61a14700 10 monclient(hunting): authenticate will time out at 2824.704071s 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 0x7f5d5c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 0x7f5d5c0a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60210/0 (socket says 172.21.15.47:60210) 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 -- 172.21.15.47:0/1039272675 learned_addr learned my addr 172.21.15.47:0/1039272675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:45.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): get_auth_request con 0x7f5d5c0a0d20 auth_method 0 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a4c80 0x7f5d5c0a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): get_auth_request con 0x7f5d5c0a4c80 auth_method 0 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5d5c0a55a0 0x7f5d5c137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): get_auth_request con 0x7f5d5c0a55a0 auth_method 0 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_done global_id 114203 payload 274 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 -- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a4c80 msgr2=0x7f5d5c0a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a4c80 0x7f5d5c0a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 -- 172.21.15.47:0/1039272675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5d5c0a55a0 msgr2=0x7f5d5c137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5d5c0a55a0 0x7f5d5c137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient: _finish_hunting 0 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 monclient: found mon.noname-b 2024-04-20T02:35:45.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 -- 172.21.15.47:0/1039272675 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0751e0 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5affd700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a4c80 0x7f5d5c0a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.303+0000 7f5d5a7fc700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 0x7f5d5c0a5060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009920 tx=0x7f5d50009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636183d320a6c22d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 1 -- 172.21.15.47:0/1039272675 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d50011aa0 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: dump: 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.317 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: _finish_auth 0 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 1 -- 172.21.15.47:0/1039272675 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d50011c00 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 5 monclient: authenticate success, global_id 114203 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 1 -- 172.21.15.47:0/1039272675 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d50011d70 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 4 set_mon_vals no callback set 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d59ffb700 10 monclient: dump: 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.319 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 10 monclient: shutdown 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 -- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 msgr2=0x7f5d5c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 0x7f5d5c0a5060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009920 tx=0x7f5d50009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 -- 172.21.15.47:0/1039272675 shutdown_connections 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a4c80 0x7f5d5c0a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5d5c0a55a0 0x7f5d5c137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 --2- 172.21.15.47:0/1039272675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a0d20 0x7f5d5c0a5060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 -- 172.21.15.47:0/1039272675 >> 172.21.15.47:0/1039272675 conn(0x7f5d5c0aa280 msgr2=0x7f5d5c0aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 -- 172.21.15.47:0/1039272675 shutdown_connections 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.304+0000 7f5d61a14700 1 -- 172.21.15.47:0/1039272675 wait complete. 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.305+0000 7f5d61a14700 10 monclient: build_initial_monmap 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.305+0000 7f5d61a14700 10 monclient: monmap: 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:45.321 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.305+0000 7f5d61a14700 1 librados: starting msgr at 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.305+0000 7f5d61a14700 1 librados: starting objecter 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.305+0000 7f5d61a14700 1 Processor -- start 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 -- start start 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 librados: setting wanted keys 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 librados: calling monclient init 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient: init 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:45.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient: _add_conns ranks=[4,2,6,0,5,1,7,8,3] 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 0x7f5d5c1cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): picked mon.noname-c con 0x7f5d5c0a0d20 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 0x7f5d5c135b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): picked mon.noname-b con 0x7f5d5c0a4c80 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5d5c0a55a0 0x7f5d5c136040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): picked mon.noname-d con 0x7f5d5c0a55a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c123b10 con 0x7f5d5c0a4c80 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c09ea00 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c09dd20 con 0x7f5d5c0a55a0 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d61a14700 10 monclient(hunting): authenticate will time out at 2824.707649s 2024-04-20T02:35:45.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 0x7f5d5c135b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 0x7f5d5c1cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5d5c0a55a0 0x7f5d5c136040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 0x7f5d5c135b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60212/0 (socket says 172.21.15.47:60212) 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 0x7f5d5c1cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54788/0 (socket says 172.21.15.47:54788) 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 1 -- 172.21.15.47:0/2523407300 learned_addr learned my addr 172.21.15.47:0/2523407300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): get_auth_request con 0x7f5d5c0a4c80 auth_method 0 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): get_auth_request con 0x7f5d5c0a0d20 auth_method 0 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): get_auth_request con 0x7f5d5c0a55a0 auth_method 0 2024-04-20T02:35:45.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5a7fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.306+0000 7f5d5b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 10 monclient(hunting): handle_auth_done global_id 84811 payload 824 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 -- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5d5c0a55a0 msgr2=0x7f5d5c136040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5d5c0a55a0 0x7f5d5c136040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 -- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 msgr2=0x7f5d5c135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 0x7f5d5c135b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 monclient: found mon.noname-c 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 -- 172.21.15.47:0/2523407300 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0094a0 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5a7fc700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 0x7f5d5c135b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d5affd700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 0x7f5d5c1cc770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00e9b0 tx=0x7f5d4c00ecc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:45.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 client.?.objecter ms_handle_connect 0x7f5d5c0a0d20 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d4c00ee50 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: dump: 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.327 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:45.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c00ccc0 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 5 monclient: authenticate success, global_id 84811 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 10 monclient: _renew_subs 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d50009550 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d4c00ce30 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d43fff700 10 monclient: dump: 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.329 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d597fa700 4 set_mon_vals no callback set 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 10 client.84811.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 10 monclient: _renew_subs 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:45.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c124c40 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.307+0000 7f5d61a14700 1 librados: init done 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5d4c00c850 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d61a14700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d61a14700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d61a14700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c125140 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5d4803a2f0 0x7f5d4803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f5d4c04a240 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 10 client.84811.objecter ms_dispatch 0x7f5d5c001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 3 client.84811.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 3 client.84811.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d43fff700 20 client.84811.objecter dump_active .. 0 homeless 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d5a7fc700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5d4803a2f0 0x7f5d4803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.308+0000 7f5d5a7fc700 10 monclient: get_auth_request con 0x7f5d4803a2f0 auth_method 0 2024-04-20T02:35:45.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.309+0000 7f5d43fff700 10 client.84811.objecter ms_handle_connect 0x7f5d4803a2f0 2024-04-20T02:35:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.309+0000 7f5d5a7fc700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5d4803a2f0 0x7f5d4803c7b0 secure :-1 s=READY pgs=3408 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009c70 tx=0x7f5d50008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.314+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f5d5c125140 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.314+0000 7f5d43fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:45.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.314+0000 7f5d43fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:45.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d61a14700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-04-20T02:35:45.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d61a14700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-04-20T02:35:45.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d61a14700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5d5c125140 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d43fff700 1 -- 172.21.15.47:0/2523407300 <== mon.4 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3176 (secure 0 0 0) 0x7f5d4c017c80 con 0x7f5d5c0a0d20 2024-04-20T02:35:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d43fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-04-20T02:35:45.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.625+0000 7f5d43fff700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:45.626 INFO:teuthology.orchestra.run.smithi047.stdout:{"election_epoch":120,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","modified":"2024-04-20T02:15:04.177248Z","created":"2024-04-20T02:15:04.177248Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3301","nonce":0},{"type":"v1","addr":"172.21.15.47:6790","nonce":0}]},"addr":"172.21.15.47:6790/0","public_addr":"172.21.15.47:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3301","nonce":0},{"type":"v1","addr":"172.21.15.179:6790","nonce":0}]},"addr":"172.21.15.179:6790/0","public_addr":"172.21.15.179:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3302","nonce":0},{"type":"v1","addr":"172.21.15.47:6791","nonce":0}]},"addr":"172.21.15.47:6791/0","public_addr":"172.21.15.47:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3302","nonce":0},{"type":"v1","addr":"172.21.15.179:6791","nonce":0}]},"addr":"172.21.15.179:6791/0","public_addr":"172.21.15.179:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3303","nonce":0},{"type":"v1","addr":"172.21.15.47:6792","nonce":0}]},"addr":"172.21.15.47:6792/0","public_addr":"172.21.15.47:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3303","nonce":0},{"type":"v1","addr":"172.21.15.179:6792","nonce":0}]},"addr":"172.21.15.179:6792/0","public_addr":"172.21.15.179:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3304","nonce":0},{"type":"v1","addr":"172.21.15.47:6793","nonce":0}]},"addr":"172.21.15.47:6793/0","public_addr":"172.21.15.47:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-20T02:35:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.627+0000 7f5d61a14700 10 librados: watch_flush enter 2024-04-20T02:35:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 10 librados: watch_flush exit 2024-04-20T02:35:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 20 client.84811.objecter shutdown clearing up homeless session... 2024-04-20T02:35:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 10 client.84811.objecter successfully canceled tick 2024-04-20T02:35:45.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5d4803a2f0 msgr2=0x7f5d4803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5d4803a2f0 0x7f5d4803c7b0 secure :-1 s=READY pgs=3408 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009c70 tx=0x7f5d50008030 comp rx=0 tx=0).stop 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 10 monclient: shutdown 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 msgr2=0x7f5d5c1cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 0x7f5d5c1cc770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00e9b0 tx=0x7f5d4c00ecc0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 shutdown_connections 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5d4803a2f0 0x7f5d4803c7b0 unknown :-1 s=CLOSED pgs=3408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5d5c0a0d20 0x7f5d5c1cc770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5d5c0a55a0 0x7f5d5c136040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 --2- 172.21.15.47:0/2523407300 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5d5c0a4c80 0x7f5d5c135b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 >> 172.21.15.47:0/2523407300 conn(0x7f5d5c0aa280 msgr2=0x7f5d5c136c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 shutdown_connections 2024-04-20T02:35:45.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 -- 172.21.15.47:0/2523407300 wait complete. 2024-04-20T02:35:45.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.628+0000 7f5d61a14700 1 librados: shutdown 2024-04-20T02:35:45.642 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-04-20T02:35:45.642 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T02:35:45.653 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T02:35:45.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:35:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.790+0000 7f2d45d4d700 10 monclient: get_monmap_and_config 2024-04-20T02:35:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.790+0000 7f2d45d4d700 10 monclient: build_initial_monmap 2024-04-20T02:35:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.790+0000 7f2d45d4d700 10 monclient: monmap: 2024-04-20T02:35:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:45.791260+0000 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:45.791260+0000 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:45.791 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:45.792 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:45.793 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.790+0000 7f2d45d4d700 1 Processor -- start 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 1 -- start start 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient: init 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient: _add_conns ranks=[1,9,10,5,2,6,4,3,0,7,8] 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2d400a0ad0 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d400a0eb0 0x7f2d400a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): picked mon.noname-a con 0x7f2d400a0eb0 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:45.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2d4013d9f0 0x7f2d4013fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): picked mon.noname-f con 0x7f2d4013d9f0 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400751e0 con 0x7f2d400a0ad0 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4009ee40 con 0x7f2d400a0eb0 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400a1330 con 0x7f2d4013d9f0 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:45.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d45d4d700 10 monclient(hunting): authenticate will time out at 2825.192637s 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d400a0eb0 0x7f2d400a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d400a0eb0 0x7f2d400a2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53490/0 (socket says 172.21.15.47:53490) 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 1 -- 172.21.15.47:0/4039246883 learned_addr learned my addr 172.21.15.47:0/4039246883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): get_auth_request con 0x7f2d400a0eb0 auth_method 0 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.791+0000 7f2d3ffff700 1 --2- 172.21.15.47:0/4039246883 >> v2:172.21.15.179:3300/0 conn(0x7f2d4013d9f0 0x7f2d4013fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3ffff700 10 monclient(hunting): get_auth_request con 0x7f2d4013d9f0 auth_method 0 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 10 monclient(hunting): handle_auth_done global_id 104436 payload 274 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 1 -- 172.21.15.47:0/4039246883 >> v2:172.21.15.179:3300/0 conn(0x7f2d4013d9f0 msgr2=0x7f2d4013fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 1 --2- 172.21.15.47:0/4039246883 >> v2:172.21.15.179:3300/0 conn(0x7f2d4013d9f0 0x7f2d4013fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 1 -- 172.21.15.47:0/4039246883 >> v1:172.21.15.179:6789/0 conn(0x7f2d400a0ad0 legacy=0x7f2d400a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 1 monclient: found mon.noname-a 2024-04-20T02:35:45.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 1 -- 172.21.15.47:0/4039246883 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4009ddb0 con 0x7f2d400a0eb0 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3effd700 1 --2- 172.21.15.47:0/4039246883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d400a0eb0 0x7f2d400a2f10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d4006f020 tx=0x7f2d30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b1f31868d72f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 1 -- 172.21.15.47:0/4039246883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d3000bae0 con 0x7f2d400a0eb0 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: dump: 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.808 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 1 -- 172.21.15.47:0/4039246883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3000bc40 con 0x7f2d400a0eb0 2024-04-20T02:35:45.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d45d4d700 5 monclient: authenticate success, global_id 104436 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d45d4d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 1 -- 172.21.15.47:0/4039246883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d3000bdb0 con 0x7f2d400a0eb0 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d45d4d700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d45d4d700 4 set_mon_vals no callback set 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.792+0000 7f2d3e7fc700 10 monclient: dump: 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.811 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 10 monclient: shutdown 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 -- 172.21.15.47:0/4039246883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d400a0eb0 msgr2=0x7f2d400a2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/4039246883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d400a0eb0 0x7f2d400a2f10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d4006f020 tx=0x7f2d30009bd0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 -- 172.21.15.47:0/4039246883 shutdown_connections 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/4039246883 >> v2:172.21.15.179:3300/0 conn(0x7f2d4013d9f0 0x7f2d4013fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/4039246883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d400a0eb0 0x7f2d400a2f10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 -- 172.21.15.47:0/4039246883 >> 172.21.15.47:0/4039246883 conn(0x7f2d400a7420 msgr2=0x7f2d400a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 -- 172.21.15.47:0/4039246883 shutdown_connections 2024-04-20T02:35:45.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 1 -- 172.21.15.47:0/4039246883 wait complete. 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 10 monclient: build_initial_monmap 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.793+0000 7f2d45d4d700 10 monclient: monmap: 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:45.813 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 librados: starting msgr at 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 librados: starting objecter 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 Processor -- start 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 -- start start 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 librados: setting wanted keys 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 librados: calling monclient init 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient: init 2024-04-20T02:35:45.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient: _add_conns ranks=[7,6,1,5,2,3,4,8,0] 2024-04-20T02:35:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2d400a0ad0 0x7f2d401c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): picked mon.noname-i con 0x7f2d400a0ad0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 0x7f2d40138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): picked mon.noname-d con 0x7f2d400a0eb0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d4013d9f0 0x7f2d401cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): picked mon.noname-f con 0x7f2d4013d9f0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401234e0 con 0x7f2d4013d9f0 2024-04-20T02:35:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4009d300 con 0x7f2d400a0eb0 2024-04-20T02:35:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:45.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400720c0 con 0x7f2d400a0ad0 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.794+0000 7f2d45d4d700 10 monclient(hunting): authenticate will time out at 2825.195987s 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 0x7f2d40138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 0x7f2d40138100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56300/0 (socket says 172.21.15.47:56300) 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 -- 172.21.15.47:0/783431084 learned_addr learned my addr 172.21.15.47:0/783431084 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): get_auth_request con 0x7f2d400a0eb0 auth_method 0 2024-04-20T02:35:45.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d4013d9f0 0x7f2d401cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2d400a0ad0 0x7f2d401c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): get_auth_request con 0x7f2d4013d9f0 auth_method 0 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): get_auth_request con 0x7f2d400a0ad0 auth_method 0 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:45.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient(hunting): handle_auth_done global_id 55608 payload 824 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 -- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2d400a0ad0 msgr2=0x7f2d401c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2d400a0ad0 0x7f2d401c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 -- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d4013d9f0 msgr2=0x7f2d401cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d4013d9f0 0x7f2d401cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 monclient: found mon.noname-d 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:45.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 -- 172.21.15.47:0/783431084 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30009550 con 0x7f2d400a0eb0 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3effd700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 0x7f2d40138100 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009c10 tx=0x7f2d30004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 client.?.objecter ms_handle_connect 0x7f2d400a0eb0 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d30004300 con 0x7f2d400a0eb0 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: dump: 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.822 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d30004480 con 0x7f2d400a0eb0 2024-04-20T02:35:45.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d45d4d700 5 monclient: authenticate success, global_id 55608 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d30024470 con 0x7f2d400a0eb0 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d45d4d700 10 monclient: _renew_subs 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d45d4d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d3dffb700 4 set_mon_vals no callback set 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4009e510 con 0x7f2d400a0eb0 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.795+0000 7f2d37fff700 10 monclient: dump: 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:45.824 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 10 client.55608.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 10 monclient: _renew_subs 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d40121850 con 0x7f2d400a0eb0 2024-04-20T02:35:45.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 1 librados: init done 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2d30021070 con 0x7f2d400a0eb0 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d37fff700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2d2003a2f0 0x7f2d2003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.796+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d401219e0 con 0x7f2d400a0eb0 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f2d3004b8c0 con 0x7f2d400a0eb0 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d37fff700 10 client.55608.objecter ms_dispatch 0x7f2d40001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:45.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d37fff700 3 client.55608.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d37fff700 3 client.55608.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d3f7fe700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2d2003a2f0 0x7f2d2003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d3f7fe700 10 monclient: get_auth_request con 0x7f2d2003a2f0 auth_method 0 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d37fff700 20 client.55608.objecter dump_active .. 0 homeless 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d37fff700 10 client.55608.objecter ms_handle_connect 0x7f2d2003a2f0 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.797+0000 7f2d3f7fe700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2d2003a2f0 0x7f2d2003c7b0 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 crypto rx=0x7f2d28007fd0 tx=0x7f2d28009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.802+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2d30006d80 con 0x7f2d400a0eb0 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.802+0000 7f2d37fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:45.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:45.802+0000 7f2d37fff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:46.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.013+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d4009f6e0 con 0x7f2d2003a2f0 2024-04-20T02:35:46.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.018+0000 7f2d37fff700 1 -- 172.21.15.47:0/783431084 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+134038 (secure 0 0 0) 0x7f2d28001560 con 0x7f2d2003a2f0 2024-04-20T02:35:46.020 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 10 librados: watch_flush enter 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 10 librados: watch_flush exit 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 20 client.55608.objecter shutdown clearing up homeless session... 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 10 client.55608.objecter successfully canceled tick 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2d2003a2f0 msgr2=0x7f2d2003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2d2003a2f0 0x7f2d2003c7b0 secure :-1 s=READY pgs=3409 cs=0 l=1 rev1=1 crypto rx=0x7f2d28007fd0 tx=0x7f2d28009040 comp rx=0 tx=0).stop 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 10 monclient: shutdown 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 msgr2=0x7f2d40138100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 0x7f2d40138100 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009c10 tx=0x7f2d30004750 comp rx=0 tx=0).stop 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 shutdown_connections 2024-04-20T02:35:46.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2d2003a2f0 0x7f2d2003c7b0 unknown :-1 s=CLOSED pgs=3409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d4013d9f0 0x7f2d401cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f2d400a0ad0 0x7f2d401c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 --2- 172.21.15.47:0/783431084 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2d400a0eb0 0x7f2d40138100 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 >> 172.21.15.47:0/783431084 conn(0x7f2d400a7420 msgr2=0x7f2d400a16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 shutdown_connections 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 -- 172.21.15.47:0/783431084 wait complete. 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.021+0000 7f2d45d4d700 1 librados: shutdown 2024-04-20T02:35:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:35:46.034 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1292,"stamp":"2024-04-20T02:35:44.614445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":106,"num_write":166,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470332,"kb_used_data":38368,"kb_used_omap":9,"kb_used_meta":431862,"kb_avail":565760708,"statfs":{"total":579820584960,"available":579338964992,"internally_reserved":0,"allocated":39288832,"data_stored":20149014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":442227367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002891"},"pg_stats":[{"pgid":"5.1c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794551+0000","last_change":"2024-04-20T02:16:44.180381+0000","last_active":"2024-04-20T02:34:02.794551+0000","last_peered":"2024-04-20T02:34:02.794551+0000","last_clean":"2024-04-20T02:34:02.794551+0000","last_became_active":"2024-04-20T02:16:44.179944+0000","last_became_peered":"2024-04-20T02:16:44.179944+0000","last_unstale":"2024-04-20T02:34:02.794551+0000","last_undegraded":"2024-04-20T02:34:02.794551+0000","last_fullsized":"2024-04-20T02:34:02.794551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:23:16.877269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":685,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794538+0000","last_change":"2024-04-20T02:16:44.131425+0000","last_active":"2024-04-20T02:34:02.794538+0000","last_peered":"2024-04-20T02:34:02.794538+0000","last_clean":"2024-04-20T02:34:02.794538+0000","last_became_active":"2024-04-20T02:16:44.130319+0000","last_became_peered":"2024-04-20T02:16:44.130319+0000","last_unstale":"2024-04-20T02:34:02.794538+0000","last_undegraded":"2024-04-20T02:34:02.794538+0000","last_fullsized":"2024-04-20T02:34:02.794538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:11:12.232716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467131+0000","last_change":"2024-04-20T02:16:44.124440+0000","last_active":"2024-04-20T02:34:02.467131+0000","last_peered":"2024-04-20T02:34:02.467131+0000","last_clean":"2024-04-20T02:34:02.467131+0000","last_became_active":"2024-04-20T02:16:44.117655+0000","last_became_peered":"2024-04-20T02:16:44.117655+0000","last_unstale":"2024-04-20T02:34:02.467131+0000","last_undegraded":"2024-04-20T02:34:02.467131+0000","last_fullsized":"2024-04-20T02:34:02.467131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:25:01.668733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794269+0000","last_change":"2024-04-20T02:16:44.113440+0000","last_active":"2024-04-20T02:34:02.794269+0000","last_peered":"2024-04-20T02:34:02.794269+0000","last_clean":"2024-04-20T02:34:02.794269+0000","last_became_active":"2024-04-20T02:16:44.111746+0000","last_became_peered":"2024-04-20T02:16:44.111746+0000","last_unstale":"2024-04-20T02:34:02.794269+0000","last_undegraded":"2024-04-20T02:34:02.794269+0000","last_fullsized":"2024-04-20T02:34:02.794269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:07:08.939799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794542+0000","last_change":"2024-04-20T02:16:44.117853+0000","last_active":"2024-04-20T02:34:02.794542+0000","last_peered":"2024-04-20T02:34:02.794542+0000","last_clean":"2024-04-20T02:34:02.794542+0000","last_became_active":"2024-04-20T02:16:44.117331+0000","last_became_peered":"2024-04-20T02:16:44.117331+0000","last_unstale":"2024-04-20T02:34:02.794542+0000","last_undegraded":"2024-04-20T02:34:02.794542+0000","last_fullsized":"2024-04-20T02:34:02.794542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:19:07.833372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466150+0000","last_change":"2024-04-20T02:16:44.123832+0000","last_active":"2024-04-20T02:34:02.466150+0000","last_peered":"2024-04-20T02:34:02.466150+0000","last_clean":"2024-04-20T02:34:02.466150+0000","last_became_active":"2024-04-20T02:16:44.123156+0000","last_became_peered":"2024-04-20T02:16:44.123156+0000","last_unstale":"2024-04-20T02:34:02.466150+0000","last_undegraded":"2024-04-20T02:34:02.466150+0000","last_fullsized":"2024-04-20T02:34:02.466150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:33:01.391874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793886+0000","last_change":"2024-04-20T02:16:44.117753+0000","last_active":"2024-04-20T02:34:02.793886+0000","last_peered":"2024-04-20T02:34:02.793886+0000","last_clean":"2024-04-20T02:34:02.793886+0000","last_became_active":"2024-04-20T02:16:44.117507+0000","last_became_peered":"2024-04-20T02:16:44.117507+0000","last_unstale":"2024-04-20T02:34:02.793886+0000","last_undegraded":"2024-04-20T02:34:02.793886+0000","last_fullsized":"2024-04-20T02:34:02.793886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:23:47.554656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":670,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455485+0000","last_change":"2024-04-20T02:16:44.229184+0000","last_active":"2024-04-20T02:34:01.455485+0000","last_peered":"2024-04-20T02:34:01.455485+0000","last_clean":"2024-04-20T02:34:01.455485+0000","last_became_active":"2024-04-20T02:16:44.228394+0000","last_became_peered":"2024-04-20T02:16:44.228394+0000","last_unstale":"2024-04-20T02:34:01.455485+0000","last_undegraded":"2024-04-20T02:34:01.455485+0000","last_fullsized":"2024-04-20T02:34:01.455485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:49:02.299939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793947+0000","last_change":"2024-04-20T02:16:44.171109+0000","last_active":"2024-04-20T02:34:02.793947+0000","last_peered":"2024-04-20T02:34:02.793947+0000","last_clean":"2024-04-20T02:34:02.793947+0000","last_became_active":"2024-04-20T02:16:44.170848+0000","last_became_peered":"2024-04-20T02:16:44.170848+0000","last_unstale":"2024-04-20T02:34:02.793947+0000","last_undegraded":"2024-04-20T02:34:02.793947+0000","last_fullsized":"2024-04-20T02:34:02.793947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:07:14.425892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.458239+0000","last_change":"2024-04-20T02:16:44.144117+0000","last_active":"2024-04-20T02:34:01.458239+0000","last_peered":"2024-04-20T02:34:01.458239+0000","last_clean":"2024-04-20T02:34:01.458239+0000","last_became_active":"2024-04-20T02:16:44.139769+0000","last_became_peered":"2024-04-20T02:16:44.139769+0000","last_unstale":"2024-04-20T02:34:01.458239+0000","last_undegraded":"2024-04-20T02:34:01.458239+0000","last_fullsized":"2024-04-20T02:34:01.458239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:36:23.054719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":678,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.640027+0000","last_change":"2024-04-20T02:16:44.190078+0000","last_active":"2024-04-20T02:34:01.640027+0000","last_peered":"2024-04-20T02:34:01.640027+0000","last_clean":"2024-04-20T02:34:01.640027+0000","last_became_active":"2024-04-20T02:16:44.187150+0000","last_became_peered":"2024-04-20T02:16:44.187150+0000","last_unstale":"2024-04-20T02:34:01.640027+0000","last_undegraded":"2024-04-20T02:34:01.640027+0000","last_fullsized":"2024-04-20T02:34:01.640027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:55.299032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794398+0000","last_change":"2024-04-20T02:16:44.174024+0000","last_active":"2024-04-20T02:34:02.794398+0000","last_peered":"2024-04-20T02:34:02.794398+0000","last_clean":"2024-04-20T02:34:02.794398+0000","last_became_active":"2024-04-20T02:16:44.167097+0000","last_became_peered":"2024-04-20T02:16:44.167097+0000","last_unstale":"2024-04-20T02:34:02.794398+0000","last_undegraded":"2024-04-20T02:34:02.794398+0000","last_fullsized":"2024-04-20T02:34:02.794398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:06:23.676407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467259+0000","last_change":"2024-04-20T02:16:44.168983+0000","last_active":"2024-04-20T02:34:02.467259+0000","last_peered":"2024-04-20T02:34:02.467259+0000","last_clean":"2024-04-20T02:34:02.467259+0000","last_became_active":"2024-04-20T02:16:44.168588+0000","last_became_peered":"2024-04-20T02:16:44.168588+0000","last_unstale":"2024-04-20T02:34:02.467259+0000","last_undegraded":"2024-04-20T02:34:02.467259+0000","last_fullsized":"2024-04-20T02:34:02.467259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:54:25.176259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467132+0000","last_change":"2024-04-20T02:16:44.104041+0000","last_active":"2024-04-20T02:34:02.467132+0000","last_peered":"2024-04-20T02:34:02.467132+0000","last_clean":"2024-04-20T02:34:02.467132+0000","last_became_active":"2024-04-20T02:16:44.103711+0000","last_became_peered":"2024-04-20T02:16:44.103711+0000","last_unstale":"2024-04-20T02:34:02.467132+0000","last_undegraded":"2024-04-20T02:34:02.467132+0000","last_fullsized":"2024-04-20T02:34:02.467132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:57:43.734300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.208649+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.207027+0000","last_became_peered":"2024-04-20T02:16:44.207027+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:48:34.254227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793955+0000","last_change":"2024-04-20T02:16:44.209432+0000","last_active":"2024-04-20T02:34:02.793955+0000","last_peered":"2024-04-20T02:34:02.793955+0000","last_clean":"2024-04-20T02:34:02.793955+0000","last_became_active":"2024-04-20T02:16:44.209154+0000","last_became_peered":"2024-04-20T02:16:44.209154+0000","last_unstale":"2024-04-20T02:34:02.793955+0000","last_undegraded":"2024-04-20T02:34:02.793955+0000","last_fullsized":"2024-04-20T02:34:02.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:55:12.327168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467157+0000","last_change":"2024-04-20T02:16:44.129276+0000","last_active":"2024-04-20T02:34:02.467157+0000","last_peered":"2024-04-20T02:34:02.467157+0000","last_clean":"2024-04-20T02:34:02.467157+0000","last_became_active":"2024-04-20T02:16:44.116619+0000","last_became_peered":"2024-04-20T02:16:44.116619+0000","last_unstale":"2024-04-20T02:34:02.467157+0000","last_undegraded":"2024-04-20T02:34:02.467157+0000","last_fullsized":"2024-04-20T02:34:02.467157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:26.681520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794720+0000","last_change":"2024-04-20T02:16:44.169597+0000","last_active":"2024-04-20T02:34:02.794720+0000","last_peered":"2024-04-20T02:34:02.794720+0000","last_clean":"2024-04-20T02:34:02.794720+0000","last_became_active":"2024-04-20T02:16:44.168518+0000","last_became_peered":"2024-04-20T02:16:44.168518+0000","last_unstale":"2024-04-20T02:34:02.794720+0000","last_undegraded":"2024-04-20T02:34:02.794720+0000","last_fullsized":"2024-04-20T02:34:02.794720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:11.785807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794535+0000","last_change":"2024-04-20T02:16:44.157424+0000","last_active":"2024-04-20T02:34:02.794535+0000","last_peered":"2024-04-20T02:34:02.794535+0000","last_clean":"2024-04-20T02:34:02.794535+0000","last_became_active":"2024-04-20T02:16:44.155758+0000","last_became_peered":"2024-04-20T02:16:44.155758+0000","last_unstale":"2024-04-20T02:34:02.794535+0000","last_undegraded":"2024-04-20T02:34:02.794535+0000","last_fullsized":"2024-04-20T02:34:02.794535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:33:30.913418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467066+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:34:02.467066+0000","last_peered":"2024-04-20T02:34:02.467066+0000","last_clean":"2024-04-20T02:34:02.467066+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:34:02.467066+0000","last_undegraded":"2024-04-20T02:34:02.467066+0000","last_fullsized":"2024-04-20T02:34:02.467066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":697,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466066+0000","last_change":"2024-04-20T02:16:44.190776+0000","last_active":"2024-04-20T02:34:02.466066+0000","last_peered":"2024-04-20T02:34:02.466066+0000","last_clean":"2024-04-20T02:34:02.466066+0000","last_became_active":"2024-04-20T02:16:44.190566+0000","last_became_peered":"2024-04-20T02:16:44.190566+0000","last_unstale":"2024-04-20T02:34:02.466066+0000","last_undegraded":"2024-04-20T02:34:02.466066+0000","last_fullsized":"2024-04-20T02:34:02.466066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:35:06.531468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467146+0000","last_change":"2024-04-20T02:16:44.169957+0000","last_active":"2024-04-20T02:34:02.467146+0000","last_peered":"2024-04-20T02:34:02.467146+0000","last_clean":"2024-04-20T02:34:02.467146+0000","last_became_active":"2024-04-20T02:16:44.169063+0000","last_became_peered":"2024-04-20T02:16:44.169063+0000","last_unstale":"2024-04-20T02:34:02.467146+0000","last_undegraded":"2024-04-20T02:34:02.467146+0000","last_fullsized":"2024-04-20T02:34:02.467146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:05:26.309126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":675,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465183+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:34:02.465183+0000","last_peered":"2024-04-20T02:34:02.465183+0000","last_clean":"2024-04-20T02:34:02.465183+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:34:02.465183+0000","last_undegraded":"2024-04-20T02:34:02.465183+0000","last_fullsized":"2024-04-20T02:34:02.465183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466569+0000","last_change":"2024-04-20T02:16:44.151129+0000","last_active":"2024-04-20T02:34:02.466569+0000","last_peered":"2024-04-20T02:34:02.466569+0000","last_clean":"2024-04-20T02:34:02.466569+0000","last_became_active":"2024-04-20T02:16:44.150893+0000","last_became_peered":"2024-04-20T02:16:44.150893+0000","last_unstale":"2024-04-20T02:34:02.466569+0000","last_undegraded":"2024-04-20T02:34:02.466569+0000","last_fullsized":"2024-04-20T02:34:02.466569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:43.272447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793882+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:34:02.793882+0000","last_peered":"2024-04-20T02:34:02.793882+0000","last_clean":"2024-04-20T02:34:02.793882+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:34:02.793882+0000","last_undegraded":"2024-04-20T02:34:02.793882+0000","last_fullsized":"2024-04-20T02:34:02.793882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794845+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:34:02.794845+0000","last_peered":"2024-04-20T02:34:02.794845+0000","last_clean":"2024-04-20T02:34:02.794845+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:34:02.794845+0000","last_undegraded":"2024-04-20T02:34:02.794845+0000","last_fullsized":"2024-04-20T02:34:02.794845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":833,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.639788+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:34:01.639788+0000","last_peered":"2024-04-20T02:34:01.639788+0000","last_clean":"2024-04-20T02:34:01.639788+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:34:01.639788+0000","last_undegraded":"2024-04-20T02:34:01.639788+0000","last_fullsized":"2024-04-20T02:34:01.639788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465201+0000","last_change":"2024-04-20T02:16:44.176094+0000","last_active":"2024-04-20T02:34:02.465201+0000","last_peered":"2024-04-20T02:34:02.465201+0000","last_clean":"2024-04-20T02:34:02.465201+0000","last_became_active":"2024-04-20T02:16:44.175527+0000","last_became_peered":"2024-04-20T02:16:44.175527+0000","last_unstale":"2024-04-20T02:34:02.465201+0000","last_undegraded":"2024-04-20T02:34:02.465201+0000","last_fullsized":"2024-04-20T02:34:02.465201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:50:24.091685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":671,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465339+0000","last_change":"2024-04-20T02:16:44.210652+0000","last_active":"2024-04-20T02:34:02.465339+0000","last_peered":"2024-04-20T02:34:02.465339+0000","last_clean":"2024-04-20T02:34:02.465339+0000","last_became_active":"2024-04-20T02:16:44.209341+0000","last_became_peered":"2024-04-20T02:16:44.209341+0000","last_unstale":"2024-04-20T02:34:02.465339+0000","last_undegraded":"2024-04-20T02:34:02.465339+0000","last_fullsized":"2024-04-20T02:34:02.465339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:25:25.620038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":688,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.638831+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:34:01.638831+0000","last_peered":"2024-04-20T02:34:01.638831+0000","last_clean":"2024-04-20T02:34:01.638831+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:34:01.638831+0000","last_undegraded":"2024-04-20T02:34:01.638831+0000","last_fullsized":"2024-04-20T02:34:01.638831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794399+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:34:02.794399+0000","last_peered":"2024-04-20T02:34:02.794399+0000","last_clean":"2024-04-20T02:34:02.794399+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:34:02.794399+0000","last_undegraded":"2024-04-20T02:34:02.794399+0000","last_fullsized":"2024-04-20T02:34:02.794399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465052+0000","last_change":"2024-04-20T02:16:44.153062+0000","last_active":"2024-04-20T02:34:02.465052+0000","last_peered":"2024-04-20T02:34:02.465052+0000","last_clean":"2024-04-20T02:34:02.465052+0000","last_became_active":"2024-04-20T02:16:44.150765+0000","last_became_peered":"2024-04-20T02:16:44.150765+0000","last_unstale":"2024-04-20T02:34:02.465052+0000","last_undegraded":"2024-04-20T02:34:02.465052+0000","last_fullsized":"2024-04-20T02:34:02.465052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:21.750599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":700,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466644+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:34:02.466644+0000","last_peered":"2024-04-20T02:34:02.466644+0000","last_clean":"2024-04-20T02:34:02.466644+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:34:02.466644+0000","last_undegraded":"2024-04-20T02:34:02.466644+0000","last_fullsized":"2024-04-20T02:34:02.466644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"18'7","reported_seq":693,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794269+0000","last_change":"2024-04-20T02:16:44.105521+0000","last_active":"2024-04-20T02:34:02.794269+0000","last_peered":"2024-04-20T02:34:02.794269+0000","last_clean":"2024-04-20T02:34:02.794269+0000","last_became_active":"2024-04-20T02:16:44.105229+0000","last_became_peered":"2024-04-20T02:16:44.105229+0000","last_unstale":"2024-04-20T02:34:02.794269+0000","last_undegraded":"2024-04-20T02:34:02.794269+0000","last_fullsized":"2024-04-20T02:34:02.794269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:23:45.284255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794750+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:34:02.794750+0000","last_peered":"2024-04-20T02:34:02.794750+0000","last_clean":"2024-04-20T02:34:02.794750+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:34:02.794750+0000","last_undegraded":"2024-04-20T02:34:02.794750+0000","last_fullsized":"2024-04-20T02:34:02.794750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.108891+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.108254+0000","last_became_peered":"2024-04-20T02:16:44.108254+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:29:23.508086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466402+0000","last_change":"2024-04-20T02:16:44.156064+0000","last_active":"2024-04-20T02:34:02.466402+0000","last_peered":"2024-04-20T02:34:02.466402+0000","last_clean":"2024-04-20T02:34:02.466402+0000","last_became_active":"2024-04-20T02:16:44.142929+0000","last_became_peered":"2024-04-20T02:16:44.142929+0000","last_unstale":"2024-04-20T02:34:02.466402+0000","last_undegraded":"2024-04-20T02:34:02.466402+0000","last_fullsized":"2024-04-20T02:34:02.466402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:33:42.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":678,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794395+0000","last_change":"2024-04-20T02:16:44.162486+0000","last_active":"2024-04-20T02:34:02.794395+0000","last_peered":"2024-04-20T02:34:02.794395+0000","last_clean":"2024-04-20T02:34:02.794395+0000","last_became_active":"2024-04-20T02:16:44.159975+0000","last_became_peered":"2024-04-20T02:16:44.159975+0000","last_unstale":"2024-04-20T02:34:02.794395+0000","last_undegraded":"2024-04-20T02:34:02.794395+0000","last_fullsized":"2024-04-20T02:34:02.794395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:05.675823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794549+0000","last_change":"2024-04-20T02:16:44.131415+0000","last_active":"2024-04-20T02:34:02.794549+0000","last_peered":"2024-04-20T02:34:02.794549+0000","last_clean":"2024-04-20T02:34:02.794549+0000","last_became_active":"2024-04-20T02:16:44.131124+0000","last_became_peered":"2024-04-20T02:16:44.131124+0000","last_unstale":"2024-04-20T02:34:02.794549+0000","last_undegraded":"2024-04-20T02:34:02.794549+0000","last_fullsized":"2024-04-20T02:34:02.794549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:41:15.350582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":678,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.639660+0000","last_change":"2024-04-20T02:16:44.142261+0000","last_active":"2024-04-20T02:34:01.639660+0000","last_peered":"2024-04-20T02:34:01.639660+0000","last_clean":"2024-04-20T02:34:01.639660+0000","last_became_active":"2024-04-20T02:16:44.141808+0000","last_became_peered":"2024-04-20T02:16:44.141808+0000","last_unstale":"2024-04-20T02:34:01.639660+0000","last_undegraded":"2024-04-20T02:34:01.639660+0000","last_fullsized":"2024-04-20T02:34:01.639660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:02:38.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455383+0000","last_change":"2024-04-20T02:16:44.100117+0000","last_active":"2024-04-20T02:34:01.455383+0000","last_peered":"2024-04-20T02:34:01.455383+0000","last_clean":"2024-04-20T02:34:01.455383+0000","last_became_active":"2024-04-20T02:16:44.099559+0000","last_became_peered":"2024-04-20T02:16:44.099559+0000","last_unstale":"2024-04-20T02:34:01.455383+0000","last_undegraded":"2024-04-20T02:34:01.455383+0000","last_fullsized":"2024-04-20T02:34:01.455383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:21:50.484507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607797,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":6373376,"data_stored":3181863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.314,"15min":0.22900000000000001},"max":{"1min":0.68000000000000005,"5min":0.80000000000000004,"15min":2.9460000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":2.8570000000000002},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.30399999999999999},"max":{"1min":0.64500000000000002,"5min":0.76900000000000002,"15min":2.391},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":1.292},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.747,"15min":1.1899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68999999999999995,"5min":0.81599999999999995,"15min":1.2450000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":1.3029999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.25900000000000001},"max":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.65400000000000003},"last":0.499}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.40999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":1.534},"last":0.40100000000000002}]}]},{"osd":3,"up_from":12,"seq":51539607796,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77532,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294308,"statfs":{"total":96636764160,"available":96557371392,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.22600000000000001},"max":{"1min":0.748,"5min":1.0880000000000001,"15min":1.4550000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":1.103,"15min":1.397},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.439,"5min":0.307,"15min":0.26000000000000001},"max":{"1min":0.70199999999999996,"5min":1.03,"15min":1.5409999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":1.073,"15min":1.5880000000000001},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":1.0600000000000001,"15min":1.5589999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70899999999999996,"5min":1.1259999999999999,"15min":2.2599999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.24299999999999999},"max":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":1.0469999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.71099999999999997,"15min":1.335},"last":0.39800000000000002}]},{"osd":5,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.23100000000000001},"max":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.90500000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.29899999999999999,"15min":0.27900000000000003},"max":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.95899999999999996},"last":0.45100000000000001}]}]},{"osd":2,"up_from":12,"seq":51539607796,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69344,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94302496,"statfs":{"total":96636764160,"available":96565755904,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"max":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.88800000000000001},"last":0.313},{"interface":"front","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.48399999999999999,"5min":0.67600000000000005,"15min":0.96899999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.28899999999999998,"5min":0.28699999999999998,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.58099999999999996,"15min":0.97799999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.33200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.45700000000000002,"5min":0.61799999999999999,"15min":1.089},"last":0.38200000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.62,"5min":0.77000000000000002,"15min":3.823},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.34499999999999997},"max":{"1min":1.226,"5min":1.226,"15min":3.1440000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81200000000000006,"15min":1.1519999999999999},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.52000000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.66500000000000004,"5min":0.85499999999999998,"15min":1.2270000000000001},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.1919999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.51500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":0.92000000000000004,"15min":1.2070000000000001},"last":0.56999999999999995}]}]},{"osd":1,"up_from":12,"seq":51539607799,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6744,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6905856,"data_stored":3710791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.48699999999999999,"15min":1.8220000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.247},"max":{"1min":0.51100000000000001,"5min":0.73499999999999999,"15min":0.92700000000000005},"last":0.33700000000000002}]},{"osd":2,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.55400000000000005,"15min":2.649},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.26500000000000001},"max":{"1min":0.57999999999999996,"5min":0.65900000000000003,"15min":2.5499999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.98699999999999999,"15min":1.034},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.68500000000000005,"5min":0.90100000000000002,"15min":1.7450000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.35999999999999999},"max":{"1min":0.77000000000000002,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":1.282},"last":0.64500000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640499,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6740,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6901760,"data_stored":3710786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.28599999999999998,"15min":0.247},"max":{"1min":0.68000000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.37,"15min":0.29699999999999999},"max":{"1min":0.66400000000000003,"5min":0.73599999999999999,"15min":0.86299999999999999},"last":0.625}]},{"osd":1,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":0.82999999999999996,"15min":0.92300000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.83599999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.69299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.22800000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65700000000000003},"last":0.45300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.433},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.34799999999999998,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.84399999999999997},"last":0.46899999999999997}]}]},{"osd":0,"up_from":12,"seq":51539607797,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81624,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94290216,"statfs":{"total":96636764160,"available":96553181184,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77199815},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.23300000000000001},"max":{"1min":0.45900000000000002,"5min":0.56699999999999995,"15min":0.71699999999999997},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.8440000000000001},"last":0.32700000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":7.8849999999999998},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.308,"15min":0.27500000000000002},"max":{"1min":0.54400000000000004,"5min":0.71799999999999997,"15min":4.9870000000000001},"last":0.30299999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":1.2410000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.46300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85599999999999998,"15min":1.2969999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":0.98899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61599999999999999},"min":{"1min":0.54900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.90400000000000003,"15min":0.98999999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.85399999999999998,"15min":1.0069999999999999},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":1.0269999999999999},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:35:46.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:35:46.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.173+0000 7f66e255e700 10 monclient: get_monmap_and_config 2024-04-20T02:35:46.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.173+0000 7f66e255e700 10 monclient: build_initial_monmap 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.173+0000 7f66e255e700 10 monclient: monmap: 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:46.174805+0000 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:46.174805+0000 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:46.174 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:46.175 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:46.175 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:46.175 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:46.175 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 1 Processor -- start 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 1 -- start start 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 10 monclient: init 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 10 monclient: _add_conns ranks=[10,9,6,0,3,2,5,8,1,4,7] 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f66dc0a0ad0 0x7f66dc0a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 10 monclient(hunting): picked mon.noname-f con 0x7f66dc0a0ad0 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66dc0a2800 0x7f66dc0a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.174+0000 7f66e255e700 10 monclient(hunting): picked mon.noname-a con 0x7f66dc0a2800 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 0x7f66dc13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 10 monclient(hunting): picked mon.noname-d con 0x7f66dc0a31c0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66dc09ee40 con 0x7f66dc0a2800 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66dc09ddb0 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0673a0 con 0x7f66dc0a31c0 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e255e700 10 monclient(hunting): authenticate will time out at 2825.576049s 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 0x7f66dc13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66dc0a2800 0x7f66dc0a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66dc0a2800 0x7f66dc0a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53506/0 (socket says 172.21.15.47:53506) 2024-04-20T02:35:46.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 0x7f66dc13e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56312/0 (socket says 172.21.15.47:56312) 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 -- 172.21.15.47:0/1234177009 learned_addr learned my addr 172.21.15.47:0/1234177009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): get_auth_request con 0x7f66dc0a2800 auth_method 0 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): get_auth_request con 0x7f66dc0a31c0 auth_method 0 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 1 --2- 172.21.15.47:0/1234177009 >> v2:172.21.15.179:3300/0 conn(0x7f66dc0a0ad0 0x7f66dc0a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): get_auth_request con 0x7f66dc0a0ad0 auth_method 0 2024-04-20T02:35:46.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66dbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient(hunting): handle_auth_done global_id 104445 payload 274 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 -- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 msgr2=0x7f66dc13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 --2- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 0x7f66dc13e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 -- 172.21.15.47:0/1234177009 >> v2:172.21.15.179:3300/0 conn(0x7f66dc0a0ad0 msgr2=0x7f66dc0a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 --2- 172.21.15.47:0/1234177009 >> v2:172.21.15.179:3300/0 conn(0x7f66dc0a0ad0 0x7f66dc0a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 monclient: found mon.noname-a 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 -- 172.21.15.47:0/1234177009 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0751e0 con 0x7f66dc0a2800 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66e0afb700 1 --2- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 0x7f66dc13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66db7fe700 1 --2- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66dc0a2800 0x7f66dc0a2c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a4770 tx=0x7f66d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dff66d35d8b75a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66daffd700 1 -- 172.21.15.47:0/1234177009 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66d0011aa0 con 0x7f66dc0a2800 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66daffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66daffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.175+0000 7f66daffd700 10 monclient: dump: 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.189 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 10 monclient: _finish_auth 0 2024-04-20T02:35:46.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 1 -- 172.21.15.47:0/1234177009 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d0011c00 con 0x7f66dc0a2800 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 5 monclient: authenticate success, global_id 104445 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 1 -- 172.21.15.47:0/1234177009 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66d0011d70 con 0x7f66dc0a2800 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 4 set_mon_vals no callback set 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66daffd700 10 monclient: dump: 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.191 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 10 monclient: shutdown 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:46.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 -- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66dc0a2800 msgr2=0x7f66dc0a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 --2- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66dc0a2800 0x7f66dc0a2c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a4770 tx=0x7f66d0009bd0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 -- 172.21.15.47:0/1234177009 shutdown_connections 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 --2- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a31c0 0x7f66dc13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 --2- 172.21.15.47:0/1234177009 >> v2:172.21.15.179:3300/0 conn(0x7f66dc0a0ad0 0x7f66dc0a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 --2- 172.21.15.47:0/1234177009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66dc0a2800 0x7f66dc0a2c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 -- 172.21.15.47:0/1234177009 >> 172.21.15.47:0/1234177009 conn(0x7f66dc0a7420 msgr2=0x7f66dc0a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 -- 172.21.15.47:0/1234177009 shutdown_connections 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.176+0000 7f66e255e700 1 -- 172.21.15.47:0/1234177009 wait complete. 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.177+0000 7f66e255e700 10 monclient: build_initial_monmap 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.177+0000 7f66e255e700 10 monclient: monmap: 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:46.193 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.177+0000 7f66e255e700 1 librados: starting msgr at 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.177+0000 7f66e255e700 1 librados: starting objecter 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 1 Processor -- start 2024-04-20T02:35:46.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 1 -- start start 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 1 librados: setting wanted keys 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 1 librados: calling monclient init 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 10 monclient: init 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 10 monclient: _add_conns ranks=[6,2,1,7,5,0,3,4,8] 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 0x7f66dc1cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 10 monclient(hunting): picked mon.noname-d con 0x7f66dc0a0ad0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66dc0a2800 0x7f66dc1bca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.178+0000 7f66e255e700 10 monclient(hunting): picked mon.noname-b con 0x7f66dc0a2800 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc0a31c0 0x7f66dc1bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient(hunting): picked mon.noname-f con 0x7f66dc0a31c0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc1a8c50 con 0x7f66dc0a31c0 2024-04-20T02:35:46.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc09e510 con 0x7f66dc0a2800 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0720c0 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient(hunting): authenticate will time out at 2825.580046s 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 0x7f66dc1cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66dc0a2800 0x7f66dc1bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 0x7f66dc1cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56328/0 (socket says 172.21.15.47:56328) 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 -- 172.21.15.47:0/2804067849 learned_addr learned my addr 172.21.15.47:0/2804067849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): get_auth_request con 0x7f66dc0a0ad0 auth_method 0 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): get_auth_request con 0x7f66dc0a2800 auth_method 0 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc0a31c0 0x7f66dc1bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): get_auth_request con 0x7f66dc0a31c0 auth_method 0 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e0afb700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient(hunting): handle_auth_done global_id 55626 payload 824 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 -- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66dc0a2800 msgr2=0x7f66dc1bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66dc0a2800 0x7f66dc1bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 -- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc0a31c0 msgr2=0x7f66dc1bcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc0a31c0 0x7f66dc1bcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 monclient: found mon.noname-d 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 -- 172.21.15.47:0/2804067849 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d4009a70 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66db7fe700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66dc0a2800 0x7f66dc1bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66dbfff700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 0x7f66dc1cc560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e9e0 tx=0x7f66cc00ecf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 client.?.objecter ms_handle_connect 0x7f66dc0a0ad0 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66cc009710 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 monclient: dump: 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.199 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 monclient: _finish_auth 0 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc00ee70 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 5 monclient: authenticate success, global_id 55626 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient: _renew_subs 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d0009550 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.179+0000 7f66d8ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66cc00ccc0 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66d8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66d8ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66da7fc700 4 set_mon_vals no callback set 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 10 client.55626.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66d8ff9700 10 monclient: dump: 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.201 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 10 monclient: _renew_subs 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc1aa100 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 1 librados: init done 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.180+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc067770 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f66cc00c850 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f66c803a2f0 0x7f66c803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f66cc04ad40 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 10 client.55626.objecter ms_dispatch 0x7f66dc001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 3 client.55626.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 3 client.55626.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.181+0000 7f66d8ff9700 20 client.55626.objecter dump_active .. 0 homeless 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.182+0000 7f66db7fe700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f66c803a2f0 0x7f66c803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.182+0000 7f66db7fe700 10 monclient: get_auth_request con 0x7f66c803a2f0 auth_method 0 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.182+0000 7f66d8ff9700 10 client.55626.objecter ms_handle_connect 0x7f66c803a2f0 2024-04-20T02:35:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.182+0000 7f66db7fe700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f66c803a2f0 0x7f66c803c7b0 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a4770 tx=0x7f66d0008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.186+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f66cc00cb00 con 0x7f66dc0a0ad0 2024-04-20T02:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.186+0000 7f66d8ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.186+0000 7f66d8ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:46.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.401+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66dc09f6e0 con 0x7f66c803a2f0 2024-04-20T02:35:46.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.406+0000 7f66d8ff9700 1 -- 172.21.15.47:0/2804067849 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+134038 (secure 0 0 0) 0x7f66d0011890 con 0x7f66c803a2f0 2024-04-20T02:35:46.407 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.408+0000 7f66e255e700 10 librados: watch_flush enter 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 10 librados: watch_flush exit 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 20 client.55626.objecter shutdown clearing up homeless session... 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 10 client.55626.objecter successfully canceled tick 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f66c803a2f0 msgr2=0x7f66c803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f66c803a2f0 0x7f66c803c7b0 secure :-1 s=READY pgs=3410 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a4770 tx=0x7f66d0008030 comp rx=0 tx=0).stop 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 10 monclient: shutdown 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:46.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 msgr2=0x7f66dc1cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 0x7f66dc1cc560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e9e0 tx=0x7f66cc00ecf0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 shutdown_connections 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f66c803a2f0 0x7f66c803c7b0 unknown :-1 s=CLOSED pgs=3410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc0a31c0 0x7f66dc1bcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f66dc0a0ad0 0x7f66dc1cc560 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 --2- 172.21.15.47:0/2804067849 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f66dc0a2800 0x7f66dc1bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 >> 172.21.15.47:0/2804067849 conn(0x7f66dc0a7420 msgr2=0x7f66dc0a5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 shutdown_connections 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 -- 172.21.15.47:0/2804067849 wait complete. 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.409+0000 7f66e255e700 1 librados: shutdown 2024-04-20T02:35:46.410 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:35:46.422 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1292,"stamp":"2024-04-20T02:35:44.614445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":106,"num_write":166,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470332,"kb_used_data":38368,"kb_used_omap":9,"kb_used_meta":431862,"kb_avail":565760708,"statfs":{"total":579820584960,"available":579338964992,"internally_reserved":0,"allocated":39288832,"data_stored":20149014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":442227367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002891"},"pg_stats":[{"pgid":"5.1c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794551+0000","last_change":"2024-04-20T02:16:44.180381+0000","last_active":"2024-04-20T02:34:02.794551+0000","last_peered":"2024-04-20T02:34:02.794551+0000","last_clean":"2024-04-20T02:34:02.794551+0000","last_became_active":"2024-04-20T02:16:44.179944+0000","last_became_peered":"2024-04-20T02:16:44.179944+0000","last_unstale":"2024-04-20T02:34:02.794551+0000","last_undegraded":"2024-04-20T02:34:02.794551+0000","last_fullsized":"2024-04-20T02:34:02.794551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:23:16.877269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":685,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794538+0000","last_change":"2024-04-20T02:16:44.131425+0000","last_active":"2024-04-20T02:34:02.794538+0000","last_peered":"2024-04-20T02:34:02.794538+0000","last_clean":"2024-04-20T02:34:02.794538+0000","last_became_active":"2024-04-20T02:16:44.130319+0000","last_became_peered":"2024-04-20T02:16:44.130319+0000","last_unstale":"2024-04-20T02:34:02.794538+0000","last_undegraded":"2024-04-20T02:34:02.794538+0000","last_fullsized":"2024-04-20T02:34:02.794538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:11:12.232716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467131+0000","last_change":"2024-04-20T02:16:44.124440+0000","last_active":"2024-04-20T02:34:02.467131+0000","last_peered":"2024-04-20T02:34:02.467131+0000","last_clean":"2024-04-20T02:34:02.467131+0000","last_became_active":"2024-04-20T02:16:44.117655+0000","last_became_peered":"2024-04-20T02:16:44.117655+0000","last_unstale":"2024-04-20T02:34:02.467131+0000","last_undegraded":"2024-04-20T02:34:02.467131+0000","last_fullsized":"2024-04-20T02:34:02.467131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:25:01.668733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794269+0000","last_change":"2024-04-20T02:16:44.113440+0000","last_active":"2024-04-20T02:34:02.794269+0000","last_peered":"2024-04-20T02:34:02.794269+0000","last_clean":"2024-04-20T02:34:02.794269+0000","last_became_active":"2024-04-20T02:16:44.111746+0000","last_became_peered":"2024-04-20T02:16:44.111746+0000","last_unstale":"2024-04-20T02:34:02.794269+0000","last_undegraded":"2024-04-20T02:34:02.794269+0000","last_fullsized":"2024-04-20T02:34:02.794269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:07:08.939799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794542+0000","last_change":"2024-04-20T02:16:44.117853+0000","last_active":"2024-04-20T02:34:02.794542+0000","last_peered":"2024-04-20T02:34:02.794542+0000","last_clean":"2024-04-20T02:34:02.794542+0000","last_became_active":"2024-04-20T02:16:44.117331+0000","last_became_peered":"2024-04-20T02:16:44.117331+0000","last_unstale":"2024-04-20T02:34:02.794542+0000","last_undegraded":"2024-04-20T02:34:02.794542+0000","last_fullsized":"2024-04-20T02:34:02.794542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:19:07.833372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466150+0000","last_change":"2024-04-20T02:16:44.123832+0000","last_active":"2024-04-20T02:34:02.466150+0000","last_peered":"2024-04-20T02:34:02.466150+0000","last_clean":"2024-04-20T02:34:02.466150+0000","last_became_active":"2024-04-20T02:16:44.123156+0000","last_became_peered":"2024-04-20T02:16:44.123156+0000","last_unstale":"2024-04-20T02:34:02.466150+0000","last_undegraded":"2024-04-20T02:34:02.466150+0000","last_fullsized":"2024-04-20T02:34:02.466150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:33:01.391874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793886+0000","last_change":"2024-04-20T02:16:44.117753+0000","last_active":"2024-04-20T02:34:02.793886+0000","last_peered":"2024-04-20T02:34:02.793886+0000","last_clean":"2024-04-20T02:34:02.793886+0000","last_became_active":"2024-04-20T02:16:44.117507+0000","last_became_peered":"2024-04-20T02:16:44.117507+0000","last_unstale":"2024-04-20T02:34:02.793886+0000","last_undegraded":"2024-04-20T02:34:02.793886+0000","last_fullsized":"2024-04-20T02:34:02.793886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:23:47.554656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":670,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455485+0000","last_change":"2024-04-20T02:16:44.229184+0000","last_active":"2024-04-20T02:34:01.455485+0000","last_peered":"2024-04-20T02:34:01.455485+0000","last_clean":"2024-04-20T02:34:01.455485+0000","last_became_active":"2024-04-20T02:16:44.228394+0000","last_became_peered":"2024-04-20T02:16:44.228394+0000","last_unstale":"2024-04-20T02:34:01.455485+0000","last_undegraded":"2024-04-20T02:34:01.455485+0000","last_fullsized":"2024-04-20T02:34:01.455485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:49:02.299939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793947+0000","last_change":"2024-04-20T02:16:44.171109+0000","last_active":"2024-04-20T02:34:02.793947+0000","last_peered":"2024-04-20T02:34:02.793947+0000","last_clean":"2024-04-20T02:34:02.793947+0000","last_became_active":"2024-04-20T02:16:44.170848+0000","last_became_peered":"2024-04-20T02:16:44.170848+0000","last_unstale":"2024-04-20T02:34:02.793947+0000","last_undegraded":"2024-04-20T02:34:02.793947+0000","last_fullsized":"2024-04-20T02:34:02.793947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:07:14.425892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.458239+0000","last_change":"2024-04-20T02:16:44.144117+0000","last_active":"2024-04-20T02:34:01.458239+0000","last_peered":"2024-04-20T02:34:01.458239+0000","last_clean":"2024-04-20T02:34:01.458239+0000","last_became_active":"2024-04-20T02:16:44.139769+0000","last_became_peered":"2024-04-20T02:16:44.139769+0000","last_unstale":"2024-04-20T02:34:01.458239+0000","last_undegraded":"2024-04-20T02:34:01.458239+0000","last_fullsized":"2024-04-20T02:34:01.458239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:36:23.054719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":678,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.640027+0000","last_change":"2024-04-20T02:16:44.190078+0000","last_active":"2024-04-20T02:34:01.640027+0000","last_peered":"2024-04-20T02:34:01.640027+0000","last_clean":"2024-04-20T02:34:01.640027+0000","last_became_active":"2024-04-20T02:16:44.187150+0000","last_became_peered":"2024-04-20T02:16:44.187150+0000","last_unstale":"2024-04-20T02:34:01.640027+0000","last_undegraded":"2024-04-20T02:34:01.640027+0000","last_fullsized":"2024-04-20T02:34:01.640027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:55.299032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794398+0000","last_change":"2024-04-20T02:16:44.174024+0000","last_active":"2024-04-20T02:34:02.794398+0000","last_peered":"2024-04-20T02:34:02.794398+0000","last_clean":"2024-04-20T02:34:02.794398+0000","last_became_active":"2024-04-20T02:16:44.167097+0000","last_became_peered":"2024-04-20T02:16:44.167097+0000","last_unstale":"2024-04-20T02:34:02.794398+0000","last_undegraded":"2024-04-20T02:34:02.794398+0000","last_fullsized":"2024-04-20T02:34:02.794398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:06:23.676407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467259+0000","last_change":"2024-04-20T02:16:44.168983+0000","last_active":"2024-04-20T02:34:02.467259+0000","last_peered":"2024-04-20T02:34:02.467259+0000","last_clean":"2024-04-20T02:34:02.467259+0000","last_became_active":"2024-04-20T02:16:44.168588+0000","last_became_peered":"2024-04-20T02:16:44.168588+0000","last_unstale":"2024-04-20T02:34:02.467259+0000","last_undegraded":"2024-04-20T02:34:02.467259+0000","last_fullsized":"2024-04-20T02:34:02.467259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:54:25.176259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467132+0000","last_change":"2024-04-20T02:16:44.104041+0000","last_active":"2024-04-20T02:34:02.467132+0000","last_peered":"2024-04-20T02:34:02.467132+0000","last_clean":"2024-04-20T02:34:02.467132+0000","last_became_active":"2024-04-20T02:16:44.103711+0000","last_became_peered":"2024-04-20T02:16:44.103711+0000","last_unstale":"2024-04-20T02:34:02.467132+0000","last_undegraded":"2024-04-20T02:34:02.467132+0000","last_fullsized":"2024-04-20T02:34:02.467132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:57:43.734300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.208649+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.207027+0000","last_became_peered":"2024-04-20T02:16:44.207027+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:48:34.254227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793955+0000","last_change":"2024-04-20T02:16:44.209432+0000","last_active":"2024-04-20T02:34:02.793955+0000","last_peered":"2024-04-20T02:34:02.793955+0000","last_clean":"2024-04-20T02:34:02.793955+0000","last_became_active":"2024-04-20T02:16:44.209154+0000","last_became_peered":"2024-04-20T02:16:44.209154+0000","last_unstale":"2024-04-20T02:34:02.793955+0000","last_undegraded":"2024-04-20T02:34:02.793955+0000","last_fullsized":"2024-04-20T02:34:02.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:55:12.327168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467157+0000","last_change":"2024-04-20T02:16:44.129276+0000","last_active":"2024-04-20T02:34:02.467157+0000","last_peered":"2024-04-20T02:34:02.467157+0000","last_clean":"2024-04-20T02:34:02.467157+0000","last_became_active":"2024-04-20T02:16:44.116619+0000","last_became_peered":"2024-04-20T02:16:44.116619+0000","last_unstale":"2024-04-20T02:34:02.467157+0000","last_undegraded":"2024-04-20T02:34:02.467157+0000","last_fullsized":"2024-04-20T02:34:02.467157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:26.681520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794720+0000","last_change":"2024-04-20T02:16:44.169597+0000","last_active":"2024-04-20T02:34:02.794720+0000","last_peered":"2024-04-20T02:34:02.794720+0000","last_clean":"2024-04-20T02:34:02.794720+0000","last_became_active":"2024-04-20T02:16:44.168518+0000","last_became_peered":"2024-04-20T02:16:44.168518+0000","last_unstale":"2024-04-20T02:34:02.794720+0000","last_undegraded":"2024-04-20T02:34:02.794720+0000","last_fullsized":"2024-04-20T02:34:02.794720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:11.785807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794535+0000","last_change":"2024-04-20T02:16:44.157424+0000","last_active":"2024-04-20T02:34:02.794535+0000","last_peered":"2024-04-20T02:34:02.794535+0000","last_clean":"2024-04-20T02:34:02.794535+0000","last_became_active":"2024-04-20T02:16:44.155758+0000","last_became_peered":"2024-04-20T02:16:44.155758+0000","last_unstale":"2024-04-20T02:34:02.794535+0000","last_undegraded":"2024-04-20T02:34:02.794535+0000","last_fullsized":"2024-04-20T02:34:02.794535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:33:30.913418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467066+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:34:02.467066+0000","last_peered":"2024-04-20T02:34:02.467066+0000","last_clean":"2024-04-20T02:34:02.467066+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:34:02.467066+0000","last_undegraded":"2024-04-20T02:34:02.467066+0000","last_fullsized":"2024-04-20T02:34:02.467066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":697,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466066+0000","last_change":"2024-04-20T02:16:44.190776+0000","last_active":"2024-04-20T02:34:02.466066+0000","last_peered":"2024-04-20T02:34:02.466066+0000","last_clean":"2024-04-20T02:34:02.466066+0000","last_became_active":"2024-04-20T02:16:44.190566+0000","last_became_peered":"2024-04-20T02:16:44.190566+0000","last_unstale":"2024-04-20T02:34:02.466066+0000","last_undegraded":"2024-04-20T02:34:02.466066+0000","last_fullsized":"2024-04-20T02:34:02.466066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:35:06.531468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467146+0000","last_change":"2024-04-20T02:16:44.169957+0000","last_active":"2024-04-20T02:34:02.467146+0000","last_peered":"2024-04-20T02:34:02.467146+0000","last_clean":"2024-04-20T02:34:02.467146+0000","last_became_active":"2024-04-20T02:16:44.169063+0000","last_became_peered":"2024-04-20T02:16:44.169063+0000","last_unstale":"2024-04-20T02:34:02.467146+0000","last_undegraded":"2024-04-20T02:34:02.467146+0000","last_fullsized":"2024-04-20T02:34:02.467146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:05:26.309126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":675,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465183+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:34:02.465183+0000","last_peered":"2024-04-20T02:34:02.465183+0000","last_clean":"2024-04-20T02:34:02.465183+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:34:02.465183+0000","last_undegraded":"2024-04-20T02:34:02.465183+0000","last_fullsized":"2024-04-20T02:34:02.465183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466569+0000","last_change":"2024-04-20T02:16:44.151129+0000","last_active":"2024-04-20T02:34:02.466569+0000","last_peered":"2024-04-20T02:34:02.466569+0000","last_clean":"2024-04-20T02:34:02.466569+0000","last_became_active":"2024-04-20T02:16:44.150893+0000","last_became_peered":"2024-04-20T02:16:44.150893+0000","last_unstale":"2024-04-20T02:34:02.466569+0000","last_undegraded":"2024-04-20T02:34:02.466569+0000","last_fullsized":"2024-04-20T02:34:02.466569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:43.272447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793882+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:34:02.793882+0000","last_peered":"2024-04-20T02:34:02.793882+0000","last_clean":"2024-04-20T02:34:02.793882+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:34:02.793882+0000","last_undegraded":"2024-04-20T02:34:02.793882+0000","last_fullsized":"2024-04-20T02:34:02.793882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794845+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:34:02.794845+0000","last_peered":"2024-04-20T02:34:02.794845+0000","last_clean":"2024-04-20T02:34:02.794845+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:34:02.794845+0000","last_undegraded":"2024-04-20T02:34:02.794845+0000","last_fullsized":"2024-04-20T02:34:02.794845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":833,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.639788+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:34:01.639788+0000","last_peered":"2024-04-20T02:34:01.639788+0000","last_clean":"2024-04-20T02:34:01.639788+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:34:01.639788+0000","last_undegraded":"2024-04-20T02:34:01.639788+0000","last_fullsized":"2024-04-20T02:34:01.639788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465201+0000","last_change":"2024-04-20T02:16:44.176094+0000","last_active":"2024-04-20T02:34:02.465201+0000","last_peered":"2024-04-20T02:34:02.465201+0000","last_clean":"2024-04-20T02:34:02.465201+0000","last_became_active":"2024-04-20T02:16:44.175527+0000","last_became_peered":"2024-04-20T02:16:44.175527+0000","last_unstale":"2024-04-20T02:34:02.465201+0000","last_undegraded":"2024-04-20T02:34:02.465201+0000","last_fullsized":"2024-04-20T02:34:02.465201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:50:24.091685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":671,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465339+0000","last_change":"2024-04-20T02:16:44.210652+0000","last_active":"2024-04-20T02:34:02.465339+0000","last_peered":"2024-04-20T02:34:02.465339+0000","last_clean":"2024-04-20T02:34:02.465339+0000","last_became_active":"2024-04-20T02:16:44.209341+0000","last_became_peered":"2024-04-20T02:16:44.209341+0000","last_unstale":"2024-04-20T02:34:02.465339+0000","last_undegraded":"2024-04-20T02:34:02.465339+0000","last_fullsized":"2024-04-20T02:34:02.465339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:25:25.620038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":688,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.638831+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:34:01.638831+0000","last_peered":"2024-04-20T02:34:01.638831+0000","last_clean":"2024-04-20T02:34:01.638831+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:34:01.638831+0000","last_undegraded":"2024-04-20T02:34:01.638831+0000","last_fullsized":"2024-04-20T02:34:01.638831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794399+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:34:02.794399+0000","last_peered":"2024-04-20T02:34:02.794399+0000","last_clean":"2024-04-20T02:34:02.794399+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:34:02.794399+0000","last_undegraded":"2024-04-20T02:34:02.794399+0000","last_fullsized":"2024-04-20T02:34:02.794399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465052+0000","last_change":"2024-04-20T02:16:44.153062+0000","last_active":"2024-04-20T02:34:02.465052+0000","last_peered":"2024-04-20T02:34:02.465052+0000","last_clean":"2024-04-20T02:34:02.465052+0000","last_became_active":"2024-04-20T02:16:44.150765+0000","last_became_peered":"2024-04-20T02:16:44.150765+0000","last_unstale":"2024-04-20T02:34:02.465052+0000","last_undegraded":"2024-04-20T02:34:02.465052+0000","last_fullsized":"2024-04-20T02:34:02.465052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:21.750599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":700,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466644+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:34:02.466644+0000","last_peered":"2024-04-20T02:34:02.466644+0000","last_clean":"2024-04-20T02:34:02.466644+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:34:02.466644+0000","last_undegraded":"2024-04-20T02:34:02.466644+0000","last_fullsized":"2024-04-20T02:34:02.466644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"18'7","reported_seq":693,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794269+0000","last_change":"2024-04-20T02:16:44.105521+0000","last_active":"2024-04-20T02:34:02.794269+0000","last_peered":"2024-04-20T02:34:02.794269+0000","last_clean":"2024-04-20T02:34:02.794269+0000","last_became_active":"2024-04-20T02:16:44.105229+0000","last_became_peered":"2024-04-20T02:16:44.105229+0000","last_unstale":"2024-04-20T02:34:02.794269+0000","last_undegraded":"2024-04-20T02:34:02.794269+0000","last_fullsized":"2024-04-20T02:34:02.794269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:23:45.284255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794750+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:34:02.794750+0000","last_peered":"2024-04-20T02:34:02.794750+0000","last_clean":"2024-04-20T02:34:02.794750+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:34:02.794750+0000","last_undegraded":"2024-04-20T02:34:02.794750+0000","last_fullsized":"2024-04-20T02:34:02.794750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.108891+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.108254+0000","last_became_peered":"2024-04-20T02:16:44.108254+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:29:23.508086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466402+0000","last_change":"2024-04-20T02:16:44.156064+0000","last_active":"2024-04-20T02:34:02.466402+0000","last_peered":"2024-04-20T02:34:02.466402+0000","last_clean":"2024-04-20T02:34:02.466402+0000","last_became_active":"2024-04-20T02:16:44.142929+0000","last_became_peered":"2024-04-20T02:16:44.142929+0000","last_unstale":"2024-04-20T02:34:02.466402+0000","last_undegraded":"2024-04-20T02:34:02.466402+0000","last_fullsized":"2024-04-20T02:34:02.466402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:33:42.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":678,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794395+0000","last_change":"2024-04-20T02:16:44.162486+0000","last_active":"2024-04-20T02:34:02.794395+0000","last_peered":"2024-04-20T02:34:02.794395+0000","last_clean":"2024-04-20T02:34:02.794395+0000","last_became_active":"2024-04-20T02:16:44.159975+0000","last_became_peered":"2024-04-20T02:16:44.159975+0000","last_unstale":"2024-04-20T02:34:02.794395+0000","last_undegraded":"2024-04-20T02:34:02.794395+0000","last_fullsized":"2024-04-20T02:34:02.794395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:05.675823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794549+0000","last_change":"2024-04-20T02:16:44.131415+0000","last_active":"2024-04-20T02:34:02.794549+0000","last_peered":"2024-04-20T02:34:02.794549+0000","last_clean":"2024-04-20T02:34:02.794549+0000","last_became_active":"2024-04-20T02:16:44.131124+0000","last_became_peered":"2024-04-20T02:16:44.131124+0000","last_unstale":"2024-04-20T02:34:02.794549+0000","last_undegraded":"2024-04-20T02:34:02.794549+0000","last_fullsized":"2024-04-20T02:34:02.794549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:41:15.350582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":678,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.639660+0000","last_change":"2024-04-20T02:16:44.142261+0000","last_active":"2024-04-20T02:34:01.639660+0000","last_peered":"2024-04-20T02:34:01.639660+0000","last_clean":"2024-04-20T02:34:01.639660+0000","last_became_active":"2024-04-20T02:16:44.141808+0000","last_became_peered":"2024-04-20T02:16:44.141808+0000","last_unstale":"2024-04-20T02:34:01.639660+0000","last_undegraded":"2024-04-20T02:34:01.639660+0000","last_fullsized":"2024-04-20T02:34:01.639660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:02:38.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455383+0000","last_change":"2024-04-20T02:16:44.100117+0000","last_active":"2024-04-20T02:34:01.455383+0000","last_peered":"2024-04-20T02:34:01.455383+0000","last_clean":"2024-04-20T02:34:01.455383+0000","last_became_active":"2024-04-20T02:16:44.099559+0000","last_became_peered":"2024-04-20T02:16:44.099559+0000","last_unstale":"2024-04-20T02:34:01.455383+0000","last_undegraded":"2024-04-20T02:34:01.455383+0000","last_fullsized":"2024-04-20T02:34:01.455383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:21:50.484507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607797,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":6373376,"data_stored":3181863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.314,"15min":0.22900000000000001},"max":{"1min":0.68000000000000005,"5min":0.80000000000000004,"15min":2.9460000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":2.8570000000000002},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.30399999999999999},"max":{"1min":0.64500000000000002,"5min":0.76900000000000002,"15min":2.391},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":1.292},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.747,"15min":1.1899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68999999999999995,"5min":0.81599999999999995,"15min":1.2450000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":1.3029999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.25900000000000001},"max":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.65400000000000003},"last":0.499}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.40999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":1.534},"last":0.40100000000000002}]}]},{"osd":3,"up_from":12,"seq":51539607796,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77532,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294308,"statfs":{"total":96636764160,"available":96557371392,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.22600000000000001},"max":{"1min":0.748,"5min":1.0880000000000001,"15min":1.4550000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":1.103,"15min":1.397},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.439,"5min":0.307,"15min":0.26000000000000001},"max":{"1min":0.70199999999999996,"5min":1.03,"15min":1.5409999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":1.073,"15min":1.5880000000000001},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":1.0600000000000001,"15min":1.5589999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70899999999999996,"5min":1.1259999999999999,"15min":2.2599999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.24299999999999999},"max":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":1.0469999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.71099999999999997,"15min":1.335},"last":0.39800000000000002}]},{"osd":5,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.23100000000000001},"max":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.90500000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.29899999999999999,"15min":0.27900000000000003},"max":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.95899999999999996},"last":0.45100000000000001}]}]},{"osd":2,"up_from":12,"seq":51539607796,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69344,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94302496,"statfs":{"total":96636764160,"available":96565755904,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"max":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.88800000000000001},"last":0.313},{"interface":"front","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.48399999999999999,"5min":0.67600000000000005,"15min":0.96899999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.28899999999999998,"5min":0.28699999999999998,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.58099999999999996,"15min":0.97799999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.33200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.45700000000000002,"5min":0.61799999999999999,"15min":1.089},"last":0.38200000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.62,"5min":0.77000000000000002,"15min":3.823},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.34499999999999997},"max":{"1min":1.226,"5min":1.226,"15min":3.1440000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81200000000000006,"15min":1.1519999999999999},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.52000000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.66500000000000004,"5min":0.85499999999999998,"15min":1.2270000000000001},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.1919999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.51500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":0.92000000000000004,"15min":1.2070000000000001},"last":0.56999999999999995}]}]},{"osd":1,"up_from":12,"seq":51539607799,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6744,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6905856,"data_stored":3710791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.48699999999999999,"15min":1.8220000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.247},"max":{"1min":0.51100000000000001,"5min":0.73499999999999999,"15min":0.92700000000000005},"last":0.33700000000000002}]},{"osd":2,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.55400000000000005,"15min":2.649},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.26500000000000001},"max":{"1min":0.57999999999999996,"5min":0.65900000000000003,"15min":2.5499999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.98699999999999999,"15min":1.034},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.68500000000000005,"5min":0.90100000000000002,"15min":1.7450000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.35999999999999999},"max":{"1min":0.77000000000000002,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":1.282},"last":0.64500000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640499,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6740,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6901760,"data_stored":3710786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.28599999999999998,"15min":0.247},"max":{"1min":0.68000000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.37,"15min":0.29699999999999999},"max":{"1min":0.66400000000000003,"5min":0.73599999999999999,"15min":0.86299999999999999},"last":0.625}]},{"osd":1,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":0.82999999999999996,"15min":0.92300000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.83599999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.69299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.22800000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65700000000000003},"last":0.45300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.433},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.34799999999999998,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.84399999999999997},"last":0.46899999999999997}]}]},{"osd":0,"up_from":12,"seq":51539607797,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81624,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94290216,"statfs":{"total":96636764160,"available":96553181184,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77199815},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.23300000000000001},"max":{"1min":0.45900000000000002,"5min":0.56699999999999995,"15min":0.71699999999999997},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.8440000000000001},"last":0.32700000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":7.8849999999999998},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.308,"15min":0.27500000000000002},"max":{"1min":0.54400000000000004,"5min":0.71799999999999997,"15min":4.9870000000000001},"last":0.30299999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":1.2410000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.46300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85599999999999998,"15min":1.2969999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":0.98899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61599999999999999},"min":{"1min":0.54900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.90400000000000003,"15min":0.98999999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.85399999999999998,"15min":1.0069999999999999},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":1.0269999999999999},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:35:46.425 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T02:35:46.425 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.562+0000 7fd08a325700 10 monclient: get_monmap_and_config 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.562+0000 7fd08a325700 10 monclient: build_initial_monmap 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.562+0000 7fd08a325700 10 monclient: monmap: 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:46.563272+0000 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:46.563272+0000 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:46.562 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:46.563 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.562+0000 7fd08a325700 1 Processor -- start 2024-04-20T02:35:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 -- start start 2024-04-20T02:35:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient: init 2024-04-20T02:35:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:46.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient: _add_conns ranks=[10,5,7,4,3,8,1,6,9,2,0] 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd0840a2950 0x7fd0840a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): picked mon.noname-f con 0x7fd0840a2950 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): picked mon.noname-h con 0x7fd0840a3270 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd0841384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): picked mon.noname-i con 0x7fd084138040 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08409ef20 con 0x7fd0840a2950 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd08409e900 con 0x7fd0840a3270 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd084067390 con 0x7fd084138040 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd08a325700 10 monclient(hunting): authenticate will time out at 2825.964531s 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0888c2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd0841384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:41990/0 (socket says 172.21.15.47:41990) 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 1 -- 172.21.15.47:0/1252283062 learned_addr learned my addr 172.21.15.47:0/1252283062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 10 monclient(hunting): get_auth_request con 0x7fd0840a3270 auth_method 0 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0888c2700 10 monclient(hunting): get_auth_request con 0x7fd084138040 auth_method 0 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd083fff700 1 --2- 172.21.15.47:0/1252283062 >> v2:172.21.15.179:3300/0 conn(0x7fd0840a2950 0x7fd0840a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0888c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0888c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.563+0000 7fd0888c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): get_auth_request con 0x7fd0840a2950 auth_method 0 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd083fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 10 monclient(hunting): handle_auth_done global_id 114208 payload 274 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 -- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 msgr2=0x7fd084137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 --2- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 -- 172.21.15.47:0/1252283062 >> v2:172.21.15.179:3300/0 conn(0x7fd0840a2950 msgr2=0x7fd0840a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 --2- 172.21.15.47:0/1252283062 >> v2:172.21.15.179:3300/0 conn(0x7fd0840a2950 0x7fd0840a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 10 monclient: _finish_hunting 0 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 monclient: found mon.noname-i 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.179:3303/0 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 -- 172.21.15.47:0/1252283062 --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840751e0 con 0x7fd084138040 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0837fe700 1 --2- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd0888c2700 1 --2- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd0841384b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009dc0 tx=0x7fd07c00b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=816ac7570ce446a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 1 -- 172.21.15.47:0/1252283062 <== mon.7 v2:172.21.15.179:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd07c00e070 con 0x7fd084138040 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-04-20T02:35:46.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: dump: 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.578 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: _finish_auth 0 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 1 -- 172.21.15.47:0/1252283062 <== mon.7 v2:172.21.15.179:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0098f0 con 0x7fd084138040 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd08a325700 5 monclient: authenticate success, global_id 114208 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd08a325700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:46.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 1 -- 172.21.15.47:0/1252283062 <== mon.7 v2:172.21.15.179:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd07c0128a0 con 0x7fd084138040 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd08a325700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd08a325700 4 set_mon_vals no callback set 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.564+0000 7fd082ffd700 10 monclient: dump: 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.580 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 10 monclient: shutdown 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 -- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 msgr2=0x7fd0841384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 --2- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd0841384b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009dc0 tx=0x7fd07c00b330 comp rx=0 tx=0).stop 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 -- 172.21.15.47:0/1252283062 shutdown_connections 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 --2- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd0841384b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 --2- 172.21.15.47:0/1252283062 >> v2:172.21.15.179:3300/0 conn(0x7fd0840a2950 0x7fd0840a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 --2- 172.21.15.47:0/1252283062 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 -- 172.21.15.47:0/1252283062 >> 172.21.15.47:0/1252283062 conn(0x7fd0840a7570 msgr2=0x7fd0840a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 -- 172.21.15.47:0/1252283062 shutdown_connections 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.565+0000 7fd08a325700 1 -- 172.21.15.47:0/1252283062 wait complete. 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 10 monclient: build_initial_monmap 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 10 monclient: monmap: 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:46.582 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 1 librados: starting msgr at 2024-04-20T02:35:46.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 1 librados: starting objecter 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 1 Processor -- start 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 1 -- start start 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 1 librados: setting wanted keys 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 1 librados: calling monclient init 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.566+0000 7fd08a325700 10 monclient: init 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient: _add_conns ranks=[0,5,7,1,2,4,6,3,8] 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 0x7fd0841219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): picked mon.noname-a con 0x7fd0840a2950 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084121ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): picked mon.noname-h con 0x7fd0840a3270 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd084122420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): picked mon.noname-i con 0x7fd084138040 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:46.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084125be0 con 0x7fd0840a2950 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd084123360 con 0x7fd0840a3270 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd084134380 con 0x7fd084138040 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd08a325700 10 monclient(hunting): authenticate will time out at 2825.968175s 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 0x7fd0841219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 0x7fd0841219a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53508/0 (socket says 172.21.15.47:53508) 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 -- 172.21.15.47:0/2302359864 learned_addr learned my addr 172.21.15.47:0/2302359864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:46.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): get_auth_request con 0x7fd0840a2950 auth_method 0 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084121ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd084122420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): get_auth_request con 0x7fd0840a3270 auth_method 0 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): get_auth_request con 0x7fd084138040 auth_method 0 2024-04-20T02:35:46.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0837fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd0888c2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient(hunting): handle_auth_done global_id 104454 payload 824 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 -- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 msgr2=0x7fd084122420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd084122420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 -- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 msgr2=0x7fd084121ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084121ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:46.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 monclient: found mon.noname-a 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 -- 172.21.15.47:0/2302359864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009550 con 0x7fd0840a2950 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd083fff700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 0x7fd0841219a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e870 tx=0x7fd07400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd080ff9700 10 client.?.objecter ms_handle_connect 0x7fd0840a2950 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.567+0000 7fd080ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd074009590 con 0x7fd0840a2950 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: dump: 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.588 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.589 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: _finish_auth 0 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074013af0 con 0x7fd0840a2950 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 5 monclient: authenticate success, global_id 104454 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 monclient: _renew_subs 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c009a70 con 0x7fd0840a2950 2024-04-20T02:35:46.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd074013c60 con 0x7fd0840a2950 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd080ff9700 10 monclient: dump: 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.591 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd0827fc700 4 set_mon_vals no callback set 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 client.104454.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 monclient: _renew_subs 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd084122e00 con 0x7fd0840a2950 2024-04-20T02:35:46.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 1 librados: init done 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.568+0000 7fd08a325700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd084124b30 con 0x7fd0840a2950 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd074013680 con 0x7fd0840a2950 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd07003a320 0x7fd07003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fd07404a3c0 con 0x7fd0840a2950 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 10 client.104454.objecter ms_dispatch 0x7fd084001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 3 client.104454.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 3 client.104454.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd080ff9700 20 client.104454.objecter dump_active .. 0 homeless 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd0837fe700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd07003a320 0x7fd07003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.569+0000 7fd0837fe700 10 monclient: get_auth_request con 0x7fd07003a320 auth_method 0 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.570+0000 7fd080ff9700 10 client.104454.objecter ms_handle_connect 0x7fd07003a320 2024-04-20T02:35:46.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.570+0000 7fd0837fe700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd07003a320 0x7fd07003c7e0 secure :-1 s=READY pgs=3411 cs=0 l=1 rev1=1 crypto rx=0x7fd08409d210 tx=0x7fd0780092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.574+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd084124b30 con 0x7fd0840a2950 2024-04-20T02:35:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.574+0000 7fd080ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.574+0000 7fd080ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:46.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.786+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd084067cb0 con 0x7fd07003a320 2024-04-20T02:35:46.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.791+0000 7fd080ff9700 1 -- 172.21.15.47:0/2302359864 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+134038 (secure 0 0 0) 0x7fd078009e20 con 0x7fd07003a320 2024-04-20T02:35:46.792 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:35:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 10 librados: watch_flush enter 2024-04-20T02:35:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 10 librados: watch_flush exit 2024-04-20T02:35:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 20 client.104454.objecter shutdown clearing up homeless session... 2024-04-20T02:35:46.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 10 client.104454.objecter successfully canceled tick 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd07003a320 msgr2=0x7fd07003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd07003a320 0x7fd07003c7e0 secure :-1 s=READY pgs=3411 cs=0 l=1 rev1=1 crypto rx=0x7fd08409d210 tx=0x7fd0780092c0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 10 monclient: shutdown 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 msgr2=0x7fd0841219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 0x7fd0841219a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e870 tx=0x7fd07400eb80 comp rx=0 tx=0).stop 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 shutdown_connections 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd07003a320 0x7fd07003c7e0 unknown :-1 s=CLOSED pgs=3411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fd084138040 0x7fd084122420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0840a2950 0x7fd0841219a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 --2- 172.21.15.47:0/2302359864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd0840a3270 0x7fd084121ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 >> 172.21.15.47:0/2302359864 conn(0x7fd0840a7570 msgr2=0x7fd0840a5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:46.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 shutdown_connections 2024-04-20T02:35:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 -- 172.21.15.47:0/2302359864 wait complete. 2024-04-20T02:35:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.793+0000 7fd08a325700 1 librados: shutdown 2024-04-20T02:35:46.796 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:35:46.807 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1293,"stamp":"2024-04-20T02:35:46.614813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":106,"num_write":166,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470332,"kb_used_data":38368,"kb_used_omap":9,"kb_used_meta":431862,"kb_avail":565760708,"statfs":{"total":579820584960,"available":579338964992,"internally_reserved":0,"allocated":39288832,"data_stored":20149014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":442227367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002695"},"pg_stats":[{"pgid":"5.1c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794551+0000","last_change":"2024-04-20T02:16:44.180381+0000","last_active":"2024-04-20T02:34:02.794551+0000","last_peered":"2024-04-20T02:34:02.794551+0000","last_clean":"2024-04-20T02:34:02.794551+0000","last_became_active":"2024-04-20T02:16:44.179944+0000","last_became_peered":"2024-04-20T02:16:44.179944+0000","last_unstale":"2024-04-20T02:34:02.794551+0000","last_undegraded":"2024-04-20T02:34:02.794551+0000","last_fullsized":"2024-04-20T02:34:02.794551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:23:16.877269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":685,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794538+0000","last_change":"2024-04-20T02:16:44.131425+0000","last_active":"2024-04-20T02:34:02.794538+0000","last_peered":"2024-04-20T02:34:02.794538+0000","last_clean":"2024-04-20T02:34:02.794538+0000","last_became_active":"2024-04-20T02:16:44.130319+0000","last_became_peered":"2024-04-20T02:16:44.130319+0000","last_unstale":"2024-04-20T02:34:02.794538+0000","last_undegraded":"2024-04-20T02:34:02.794538+0000","last_fullsized":"2024-04-20T02:34:02.794538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:11:12.232716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467131+0000","last_change":"2024-04-20T02:16:44.124440+0000","last_active":"2024-04-20T02:34:02.467131+0000","last_peered":"2024-04-20T02:34:02.467131+0000","last_clean":"2024-04-20T02:34:02.467131+0000","last_became_active":"2024-04-20T02:16:44.117655+0000","last_became_peered":"2024-04-20T02:16:44.117655+0000","last_unstale":"2024-04-20T02:34:02.467131+0000","last_undegraded":"2024-04-20T02:34:02.467131+0000","last_fullsized":"2024-04-20T02:34:02.467131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:25:01.668733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794269+0000","last_change":"2024-04-20T02:16:44.113440+0000","last_active":"2024-04-20T02:34:02.794269+0000","last_peered":"2024-04-20T02:34:02.794269+0000","last_clean":"2024-04-20T02:34:02.794269+0000","last_became_active":"2024-04-20T02:16:44.111746+0000","last_became_peered":"2024-04-20T02:16:44.111746+0000","last_unstale":"2024-04-20T02:34:02.794269+0000","last_undegraded":"2024-04-20T02:34:02.794269+0000","last_fullsized":"2024-04-20T02:34:02.794269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:07:08.939799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794542+0000","last_change":"2024-04-20T02:16:44.117853+0000","last_active":"2024-04-20T02:34:02.794542+0000","last_peered":"2024-04-20T02:34:02.794542+0000","last_clean":"2024-04-20T02:34:02.794542+0000","last_became_active":"2024-04-20T02:16:44.117331+0000","last_became_peered":"2024-04-20T02:16:44.117331+0000","last_unstale":"2024-04-20T02:34:02.794542+0000","last_undegraded":"2024-04-20T02:34:02.794542+0000","last_fullsized":"2024-04-20T02:34:02.794542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:19:07.833372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466150+0000","last_change":"2024-04-20T02:16:44.123832+0000","last_active":"2024-04-20T02:34:02.466150+0000","last_peered":"2024-04-20T02:34:02.466150+0000","last_clean":"2024-04-20T02:34:02.466150+0000","last_became_active":"2024-04-20T02:16:44.123156+0000","last_became_peered":"2024-04-20T02:16:44.123156+0000","last_unstale":"2024-04-20T02:34:02.466150+0000","last_undegraded":"2024-04-20T02:34:02.466150+0000","last_fullsized":"2024-04-20T02:34:02.466150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:33:01.391874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793886+0000","last_change":"2024-04-20T02:16:44.117753+0000","last_active":"2024-04-20T02:34:02.793886+0000","last_peered":"2024-04-20T02:34:02.793886+0000","last_clean":"2024-04-20T02:34:02.793886+0000","last_became_active":"2024-04-20T02:16:44.117507+0000","last_became_peered":"2024-04-20T02:16:44.117507+0000","last_unstale":"2024-04-20T02:34:02.793886+0000","last_undegraded":"2024-04-20T02:34:02.793886+0000","last_fullsized":"2024-04-20T02:34:02.793886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:23:47.554656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":670,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455485+0000","last_change":"2024-04-20T02:16:44.229184+0000","last_active":"2024-04-20T02:34:01.455485+0000","last_peered":"2024-04-20T02:34:01.455485+0000","last_clean":"2024-04-20T02:34:01.455485+0000","last_became_active":"2024-04-20T02:16:44.228394+0000","last_became_peered":"2024-04-20T02:16:44.228394+0000","last_unstale":"2024-04-20T02:34:01.455485+0000","last_undegraded":"2024-04-20T02:34:01.455485+0000","last_fullsized":"2024-04-20T02:34:01.455485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:49:02.299939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793947+0000","last_change":"2024-04-20T02:16:44.171109+0000","last_active":"2024-04-20T02:34:02.793947+0000","last_peered":"2024-04-20T02:34:02.793947+0000","last_clean":"2024-04-20T02:34:02.793947+0000","last_became_active":"2024-04-20T02:16:44.170848+0000","last_became_peered":"2024-04-20T02:16:44.170848+0000","last_unstale":"2024-04-20T02:34:02.793947+0000","last_undegraded":"2024-04-20T02:34:02.793947+0000","last_fullsized":"2024-04-20T02:34:02.793947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:07:14.425892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.458239+0000","last_change":"2024-04-20T02:16:44.144117+0000","last_active":"2024-04-20T02:34:01.458239+0000","last_peered":"2024-04-20T02:34:01.458239+0000","last_clean":"2024-04-20T02:34:01.458239+0000","last_became_active":"2024-04-20T02:16:44.139769+0000","last_became_peered":"2024-04-20T02:16:44.139769+0000","last_unstale":"2024-04-20T02:34:01.458239+0000","last_undegraded":"2024-04-20T02:34:01.458239+0000","last_fullsized":"2024-04-20T02:34:01.458239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:36:23.054719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":678,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.640027+0000","last_change":"2024-04-20T02:16:44.190078+0000","last_active":"2024-04-20T02:34:01.640027+0000","last_peered":"2024-04-20T02:34:01.640027+0000","last_clean":"2024-04-20T02:34:01.640027+0000","last_became_active":"2024-04-20T02:16:44.187150+0000","last_became_peered":"2024-04-20T02:16:44.187150+0000","last_unstale":"2024-04-20T02:34:01.640027+0000","last_undegraded":"2024-04-20T02:34:01.640027+0000","last_fullsized":"2024-04-20T02:34:01.640027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:12:55.299032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794398+0000","last_change":"2024-04-20T02:16:44.174024+0000","last_active":"2024-04-20T02:34:02.794398+0000","last_peered":"2024-04-20T02:34:02.794398+0000","last_clean":"2024-04-20T02:34:02.794398+0000","last_became_active":"2024-04-20T02:16:44.167097+0000","last_became_peered":"2024-04-20T02:16:44.167097+0000","last_unstale":"2024-04-20T02:34:02.794398+0000","last_undegraded":"2024-04-20T02:34:02.794398+0000","last_fullsized":"2024-04-20T02:34:02.794398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:06:23.676407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467259+0000","last_change":"2024-04-20T02:16:44.168983+0000","last_active":"2024-04-20T02:34:02.467259+0000","last_peered":"2024-04-20T02:34:02.467259+0000","last_clean":"2024-04-20T02:34:02.467259+0000","last_became_active":"2024-04-20T02:16:44.168588+0000","last_became_peered":"2024-04-20T02:16:44.168588+0000","last_unstale":"2024-04-20T02:34:02.467259+0000","last_undegraded":"2024-04-20T02:34:02.467259+0000","last_fullsized":"2024-04-20T02:34:02.467259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:54:25.176259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467132+0000","last_change":"2024-04-20T02:16:44.104041+0000","last_active":"2024-04-20T02:34:02.467132+0000","last_peered":"2024-04-20T02:34:02.467132+0000","last_clean":"2024-04-20T02:34:02.467132+0000","last_became_active":"2024-04-20T02:16:44.103711+0000","last_became_peered":"2024-04-20T02:16:44.103711+0000","last_unstale":"2024-04-20T02:34:02.467132+0000","last_undegraded":"2024-04-20T02:34:02.467132+0000","last_fullsized":"2024-04-20T02:34:02.467132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:57:43.734300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.208649+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.207027+0000","last_became_peered":"2024-04-20T02:16:44.207027+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:48:34.254227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793955+0000","last_change":"2024-04-20T02:16:44.209432+0000","last_active":"2024-04-20T02:34:02.793955+0000","last_peered":"2024-04-20T02:34:02.793955+0000","last_clean":"2024-04-20T02:34:02.793955+0000","last_became_active":"2024-04-20T02:16:44.209154+0000","last_became_peered":"2024-04-20T02:16:44.209154+0000","last_unstale":"2024-04-20T02:34:02.793955+0000","last_undegraded":"2024-04-20T02:34:02.793955+0000","last_fullsized":"2024-04-20T02:34:02.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:55:12.327168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467157+0000","last_change":"2024-04-20T02:16:44.129276+0000","last_active":"2024-04-20T02:34:02.467157+0000","last_peered":"2024-04-20T02:34:02.467157+0000","last_clean":"2024-04-20T02:34:02.467157+0000","last_became_active":"2024-04-20T02:16:44.116619+0000","last_became_peered":"2024-04-20T02:16:44.116619+0000","last_unstale":"2024-04-20T02:34:02.467157+0000","last_undegraded":"2024-04-20T02:34:02.467157+0000","last_fullsized":"2024-04-20T02:34:02.467157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:26.681520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794720+0000","last_change":"2024-04-20T02:16:44.169597+0000","last_active":"2024-04-20T02:34:02.794720+0000","last_peered":"2024-04-20T02:34:02.794720+0000","last_clean":"2024-04-20T02:34:02.794720+0000","last_became_active":"2024-04-20T02:16:44.168518+0000","last_became_peered":"2024-04-20T02:16:44.168518+0000","last_unstale":"2024-04-20T02:34:02.794720+0000","last_undegraded":"2024-04-20T02:34:02.794720+0000","last_fullsized":"2024-04-20T02:34:02.794720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:11.785807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794535+0000","last_change":"2024-04-20T02:16:44.157424+0000","last_active":"2024-04-20T02:34:02.794535+0000","last_peered":"2024-04-20T02:34:02.794535+0000","last_clean":"2024-04-20T02:34:02.794535+0000","last_became_active":"2024-04-20T02:16:44.155758+0000","last_became_peered":"2024-04-20T02:16:44.155758+0000","last_unstale":"2024-04-20T02:34:02.794535+0000","last_undegraded":"2024-04-20T02:34:02.794535+0000","last_fullsized":"2024-04-20T02:34:02.794535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:33:30.913418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467066+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:34:02.467066+0000","last_peered":"2024-04-20T02:34:02.467066+0000","last_clean":"2024-04-20T02:34:02.467066+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:34:02.467066+0000","last_undegraded":"2024-04-20T02:34:02.467066+0000","last_fullsized":"2024-04-20T02:34:02.467066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":697,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466066+0000","last_change":"2024-04-20T02:16:44.190776+0000","last_active":"2024-04-20T02:34:02.466066+0000","last_peered":"2024-04-20T02:34:02.466066+0000","last_clean":"2024-04-20T02:34:02.466066+0000","last_became_active":"2024-04-20T02:16:44.190566+0000","last_became_peered":"2024-04-20T02:16:44.190566+0000","last_unstale":"2024-04-20T02:34:02.466066+0000","last_undegraded":"2024-04-20T02:34:02.466066+0000","last_fullsized":"2024-04-20T02:34:02.466066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:35:06.531468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467146+0000","last_change":"2024-04-20T02:16:44.169957+0000","last_active":"2024-04-20T02:34:02.467146+0000","last_peered":"2024-04-20T02:34:02.467146+0000","last_clean":"2024-04-20T02:34:02.467146+0000","last_became_active":"2024-04-20T02:16:44.169063+0000","last_became_peered":"2024-04-20T02:16:44.169063+0000","last_unstale":"2024-04-20T02:34:02.467146+0000","last_undegraded":"2024-04-20T02:34:02.467146+0000","last_fullsized":"2024-04-20T02:34:02.467146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:05:26.309126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":675,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465183+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:34:02.465183+0000","last_peered":"2024-04-20T02:34:02.465183+0000","last_clean":"2024-04-20T02:34:02.465183+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:34:02.465183+0000","last_undegraded":"2024-04-20T02:34:02.465183+0000","last_fullsized":"2024-04-20T02:34:02.465183+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:08:38.502189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466569+0000","last_change":"2024-04-20T02:16:44.151129+0000","last_active":"2024-04-20T02:34:02.466569+0000","last_peered":"2024-04-20T02:34:02.466569+0000","last_clean":"2024-04-20T02:34:02.466569+0000","last_became_active":"2024-04-20T02:16:44.150893+0000","last_became_peered":"2024-04-20T02:16:44.150893+0000","last_unstale":"2024-04-20T02:34:02.466569+0000","last_undegraded":"2024-04-20T02:34:02.466569+0000","last_fullsized":"2024-04-20T02:34:02.466569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:43.272447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793882+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:34:02.793882+0000","last_peered":"2024-04-20T02:34:02.793882+0000","last_clean":"2024-04-20T02:34:02.793882+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:34:02.793882+0000","last_undegraded":"2024-04-20T02:34:02.793882+0000","last_fullsized":"2024-04-20T02:34:02.793882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794845+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:34:02.794845+0000","last_peered":"2024-04-20T02:34:02.794845+0000","last_clean":"2024-04-20T02:34:02.794845+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:34:02.794845+0000","last_undegraded":"2024-04-20T02:34:02.794845+0000","last_fullsized":"2024-04-20T02:34:02.794845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:34:04.913194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":833,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.639788+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:34:01.639788+0000","last_peered":"2024-04-20T02:34:01.639788+0000","last_clean":"2024-04-20T02:34:01.639788+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:34:01.639788+0000","last_undegraded":"2024-04-20T02:34:01.639788+0000","last_fullsized":"2024-04-20T02:34:01.639788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:37:55.029970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465201+0000","last_change":"2024-04-20T02:16:44.176094+0000","last_active":"2024-04-20T02:34:02.465201+0000","last_peered":"2024-04-20T02:34:02.465201+0000","last_clean":"2024-04-20T02:34:02.465201+0000","last_became_active":"2024-04-20T02:16:44.175527+0000","last_became_peered":"2024-04-20T02:16:44.175527+0000","last_unstale":"2024-04-20T02:34:02.465201+0000","last_undegraded":"2024-04-20T02:34:02.465201+0000","last_fullsized":"2024-04-20T02:34:02.465201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:50:24.091685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":671,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465339+0000","last_change":"2024-04-20T02:16:44.210652+0000","last_active":"2024-04-20T02:34:02.465339+0000","last_peered":"2024-04-20T02:34:02.465339+0000","last_clean":"2024-04-20T02:34:02.465339+0000","last_became_active":"2024-04-20T02:16:44.209341+0000","last_became_peered":"2024-04-20T02:16:44.209341+0000","last_unstale":"2024-04-20T02:34:02.465339+0000","last_undegraded":"2024-04-20T02:34:02.465339+0000","last_fullsized":"2024-04-20T02:34:02.465339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:25:25.620038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":688,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.638831+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:34:01.638831+0000","last_peered":"2024-04-20T02:34:01.638831+0000","last_clean":"2024-04-20T02:34:01.638831+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:34:01.638831+0000","last_undegraded":"2024-04-20T02:34:01.638831+0000","last_fullsized":"2024-04-20T02:34:01.638831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794399+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:34:02.794399+0000","last_peered":"2024-04-20T02:34:02.794399+0000","last_clean":"2024-04-20T02:34:02.794399+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:34:02.794399+0000","last_undegraded":"2024-04-20T02:34:02.794399+0000","last_fullsized":"2024-04-20T02:34:02.794399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:24:29.201539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465052+0000","last_change":"2024-04-20T02:16:44.153062+0000","last_active":"2024-04-20T02:34:02.465052+0000","last_peered":"2024-04-20T02:34:02.465052+0000","last_clean":"2024-04-20T02:34:02.465052+0000","last_became_active":"2024-04-20T02:16:44.150765+0000","last_became_peered":"2024-04-20T02:16:44.150765+0000","last_unstale":"2024-04-20T02:34:02.465052+0000","last_undegraded":"2024-04-20T02:34:02.465052+0000","last_fullsized":"2024-04-20T02:34:02.465052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:21.750599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":700,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466644+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:34:02.466644+0000","last_peered":"2024-04-20T02:34:02.466644+0000","last_clean":"2024-04-20T02:34:02.466644+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:34:02.466644+0000","last_undegraded":"2024-04-20T02:34:02.466644+0000","last_fullsized":"2024-04-20T02:34:02.466644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"18'7","reported_seq":693,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794269+0000","last_change":"2024-04-20T02:16:44.105521+0000","last_active":"2024-04-20T02:34:02.794269+0000","last_peered":"2024-04-20T02:34:02.794269+0000","last_clean":"2024-04-20T02:34:02.794269+0000","last_became_active":"2024-04-20T02:16:44.105229+0000","last_became_peered":"2024-04-20T02:16:44.105229+0000","last_unstale":"2024-04-20T02:34:02.794269+0000","last_undegraded":"2024-04-20T02:34:02.794269+0000","last_fullsized":"2024-04-20T02:34:02.794269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:23:45.284255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794750+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:34:02.794750+0000","last_peered":"2024-04-20T02:34:02.794750+0000","last_clean":"2024-04-20T02:34:02.794750+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:34:02.794750+0000","last_undegraded":"2024-04-20T02:34:02.794750+0000","last_fullsized":"2024-04-20T02:34:02.794750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:33:55.319807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.108891+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.108254+0000","last_became_peered":"2024-04-20T02:16:44.108254+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:29:23.508086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466402+0000","last_change":"2024-04-20T02:16:44.156064+0000","last_active":"2024-04-20T02:34:02.466402+0000","last_peered":"2024-04-20T02:34:02.466402+0000","last_clean":"2024-04-20T02:34:02.466402+0000","last_became_active":"2024-04-20T02:16:44.142929+0000","last_became_peered":"2024-04-20T02:16:44.142929+0000","last_unstale":"2024-04-20T02:34:02.466402+0000","last_undegraded":"2024-04-20T02:34:02.466402+0000","last_fullsized":"2024-04-20T02:34:02.466402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:33:42.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":678,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794395+0000","last_change":"2024-04-20T02:16:44.162486+0000","last_active":"2024-04-20T02:34:02.794395+0000","last_peered":"2024-04-20T02:34:02.794395+0000","last_clean":"2024-04-20T02:34:02.794395+0000","last_became_active":"2024-04-20T02:16:44.159975+0000","last_became_peered":"2024-04-20T02:16:44.159975+0000","last_unstale":"2024-04-20T02:34:02.794395+0000","last_undegraded":"2024-04-20T02:34:02.794395+0000","last_fullsized":"2024-04-20T02:34:02.794395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:05.675823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794549+0000","last_change":"2024-04-20T02:16:44.131415+0000","last_active":"2024-04-20T02:34:02.794549+0000","last_peered":"2024-04-20T02:34:02.794549+0000","last_clean":"2024-04-20T02:34:02.794549+0000","last_became_active":"2024-04-20T02:16:44.131124+0000","last_became_peered":"2024-04-20T02:16:44.131124+0000","last_unstale":"2024-04-20T02:34:02.794549+0000","last_undegraded":"2024-04-20T02:34:02.794549+0000","last_fullsized":"2024-04-20T02:34:02.794549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:41:15.350582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":678,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.639660+0000","last_change":"2024-04-20T02:16:44.142261+0000","last_active":"2024-04-20T02:34:01.639660+0000","last_peered":"2024-04-20T02:34:01.639660+0000","last_clean":"2024-04-20T02:34:01.639660+0000","last_became_active":"2024-04-20T02:16:44.141808+0000","last_became_peered":"2024-04-20T02:16:44.141808+0000","last_unstale":"2024-04-20T02:34:01.639660+0000","last_undegraded":"2024-04-20T02:34:01.639660+0000","last_fullsized":"2024-04-20T02:34:01.639660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:02:38.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455383+0000","last_change":"2024-04-20T02:16:44.100117+0000","last_active":"2024-04-20T02:34:01.455383+0000","last_peered":"2024-04-20T02:34:01.455383+0000","last_clean":"2024-04-20T02:34:01.455383+0000","last_became_active":"2024-04-20T02:16:44.099559+0000","last_became_peered":"2024-04-20T02:16:44.099559+0000","last_unstale":"2024-04-20T02:34:01.455383+0000","last_undegraded":"2024-04-20T02:34:01.455383+0000","last_fullsized":"2024-04-20T02:34:01.455383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:21:50.484507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607797,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":6373376,"data_stored":3181863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.314,"15min":0.22900000000000001},"max":{"1min":0.68000000000000005,"5min":0.80000000000000004,"15min":2.9460000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":2.8570000000000002},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.30399999999999999},"max":{"1min":0.64500000000000002,"5min":0.76900000000000002,"15min":2.391},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":1.292},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.747,"15min":1.1899999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68999999999999995,"5min":0.81599999999999995,"15min":1.2450000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":1.3029999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.25900000000000001},"max":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.65400000000000003},"last":0.499}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.40999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":1.534},"last":0.40100000000000002}]}]},{"osd":3,"up_from":12,"seq":51539607796,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77532,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294308,"statfs":{"total":96636764160,"available":96557371392,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.22600000000000001},"max":{"1min":0.748,"5min":1.0880000000000001,"15min":1.4550000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":1.103,"15min":1.397},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.439,"5min":0.307,"15min":0.26000000000000001},"max":{"1min":0.70199999999999996,"5min":1.03,"15min":1.5409999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":1.073,"15min":1.5880000000000001},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":1.0600000000000001,"15min":1.5589999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70899999999999996,"5min":1.1259999999999999,"15min":2.2599999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.24299999999999999},"max":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":1.0469999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.71099999999999997,"15min":1.335},"last":0.39800000000000002}]},{"osd":5,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.23100000000000001},"max":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.90500000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.29899999999999999,"15min":0.27900000000000003},"max":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.95899999999999996},"last":0.45100000000000001}]}]},{"osd":2,"up_from":12,"seq":51539607796,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69344,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94302496,"statfs":{"total":96636764160,"available":96565755904,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"max":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.88800000000000001},"last":0.313},{"interface":"front","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.48399999999999999,"5min":0.67600000000000005,"15min":0.96899999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.28899999999999998,"5min":0.28699999999999998,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.58099999999999996,"15min":0.97799999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.33200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.45700000000000002,"5min":0.61799999999999999,"15min":1.089},"last":0.38200000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.62,"5min":0.77000000000000002,"15min":3.823},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.34499999999999997},"max":{"1min":1.226,"5min":1.226,"15min":3.1440000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81200000000000006,"15min":1.1519999999999999},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.52000000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.66500000000000004,"5min":0.85499999999999998,"15min":1.2270000000000001},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.1919999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.51500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":0.92000000000000004,"15min":1.2070000000000001},"last":0.56999999999999995}]}]},{"osd":1,"up_from":12,"seq":51539607799,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6744,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6905856,"data_stored":3710791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.48699999999999999,"15min":1.8220000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.247},"max":{"1min":0.51100000000000001,"5min":0.73499999999999999,"15min":0.92700000000000005},"last":0.33700000000000002}]},{"osd":2,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.55400000000000005,"15min":2.649},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.26500000000000001},"max":{"1min":0.57999999999999996,"5min":0.65900000000000003,"15min":2.5499999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.98699999999999999,"15min":1.034},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.68500000000000005,"5min":0.90100000000000002,"15min":1.7450000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.35999999999999999},"max":{"1min":0.77000000000000002,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":1.282},"last":0.64500000000000002}]}]},{"osd":5,"up_from":11,"seq":47244640499,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6740,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6901760,"data_stored":3710786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.28599999999999998,"15min":0.247},"max":{"1min":0.68000000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.37,"15min":0.29699999999999999},"max":{"1min":0.66400000000000003,"5min":0.73599999999999999,"15min":0.86299999999999999},"last":0.625}]},{"osd":1,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":0.82999999999999996,"15min":0.92300000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.83599999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.69299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.22800000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65700000000000003},"last":0.45300000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.433},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.34799999999999998,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.84399999999999997},"last":0.46899999999999997}]}]},{"osd":0,"up_from":12,"seq":51539607797,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81624,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94290216,"statfs":{"total":96636764160,"available":96553181184,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77199815},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.23300000000000001},"max":{"1min":0.45900000000000002,"5min":0.56699999999999995,"15min":0.71699999999999997},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.8440000000000001},"last":0.32700000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":7.8849999999999998},"last":0.312},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.308,"15min":0.27500000000000002},"max":{"1min":0.54400000000000004,"5min":0.71799999999999997,"15min":4.9870000000000001},"last":0.30299999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":1.2410000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.46300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85599999999999998,"15min":1.2969999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":0.98899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61599999999999999},"min":{"1min":0.54900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.90400000000000003,"15min":0.98999999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.85399999999999998,"15min":1.0069999999999999},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":1.0269999999999999},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:35:46.809 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T02:35:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.944+0000 7f14d4a2f700 10 monclient: get_monmap_and_config 2024-04-20T02:35:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.944+0000 7f14d4a2f700 10 monclient: build_initial_monmap 2024-04-20T02:35:46.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.944+0000 7f14d4a2f700 10 monclient: monmap: 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:46.945659+0000 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:46.945659+0000 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:46.945 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:46.946 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:46.946 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 Processor -- start 2024-04-20T02:35:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 -- start start 2024-04-20T02:35:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient: init 2024-04-20T02:35:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient: _add_conns ranks=[6,5,10,7,0,4,9,8,2,3,1] 2024-04-20T02:35:46.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 0x7f14d00a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): picked mon.noname-d con 0x7f14d00a2950 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f14d00a3270 0x7f14d0137b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): picked mon.noname-h con 0x7f14d00a3270 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f14d0138040 0x7f14d01384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): picked mon.noname-f con 0x7f14d0138040 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d009ef20 con 0x7f14d0138040 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d009e900 con 0x7f14d00a3270 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0067390 con 0x7f14d00a2950 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.945+0000 7f14d4a2f700 10 monclient(hunting): authenticate will time out at 2826.346914s 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 0x7f14d00a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 0x7f14d00a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56332/0 (socket says 172.21.15.47:56332) 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 -- 172.21.15.47:0/2119224269 learned_addr learned my addr 172.21.15.47:0/2119224269 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): get_auth_request con 0x7f14d00a2950 auth_method 0 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 1 --2- 172.21.15.47:0/2119224269 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f14d00a3270 0x7f14d0137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 1 --2- 172.21.15.47:0/2119224269 >> v2:172.21.15.179:3300/0 conn(0x7f14d0138040 0x7f14d01384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): get_auth_request con 0x7f14d00a3270 auth_method 0 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): get_auth_request con 0x7f14d0138040 auth_method 0 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ced9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient(hunting): handle_auth_done global_id 55635 payload 274 2024-04-20T02:35:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 -- 172.21.15.47:0/2119224269 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f14d00a3270 msgr2=0x7f14d0137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 --2- 172.21.15.47:0/2119224269 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f14d00a3270 0x7f14d0137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 -- 172.21.15.47:0/2119224269 >> v2:172.21.15.179:3300/0 conn(0x7f14d0138040 msgr2=0x7f14d01384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 --2- 172.21.15.47:0/2119224269 >> v2:172.21.15.179:3300/0 conn(0x7f14d0138040 0x7f14d01384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient: _finish_hunting 0 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 monclient: found mon.noname-d 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 -- 172.21.15.47:0/2119224269 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d00751e0 con 0x7f14d00a2950 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14ce59c700 1 --2- 172.21.15.47:0/2119224269 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 0x7f14d00a2d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009920 tx=0x7f14b8009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=12802774ed3c9b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 1 -- 172.21.15.47:0/2119224269 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14b8011aa0 con 0x7f14d00a2950 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: dump: 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: _finish_auth 0 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: _check_auth_tickets 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 1 -- 172.21.15.47:0/2119224269 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b8011c00 con 0x7f14d00a2950 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14d4a2f700 5 monclient: authenticate success, global_id 55635 2024-04-20T02:35:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14d4a2f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14d4a2f700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 1 -- 172.21.15.47:0/2119224269 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14b8011d70 con 0x7f14d00a2950 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14d4a2f700 4 set_mon_vals no callback set 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.946+0000 7f14cd59a700 10 monclient: dump: 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 10 monclient: shutdown 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2119224269 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 msgr2=0x7f14d00a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2119224269 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 0x7f14d00a2d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009920 tx=0x7f14b8009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2119224269 shutdown_connections 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2119224269 >> v2:172.21.15.179:3300/0 conn(0x7f14d0138040 0x7f14d01384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2119224269 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f14d00a2950 0x7f14d00a2d30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2119224269 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f14d00a3270 0x7f14d0137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2119224269 >> 172.21.15.47:0/2119224269 conn(0x7f14d00a7570 msgr2=0x7f14d00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2119224269 shutdown_connections 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.947+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2119224269 wait complete. 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 10 monclient: build_initial_monmap 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 10 monclient: monmap: 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:46.969 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:46.970 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 1 librados: starting msgr at 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 1 librados: starting objecter 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 1 Processor -- start 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 1 -- start start 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 1 librados: setting wanted keys 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 1 librados: calling monclient init 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.948+0000 7f14d4a2f700 10 monclient: init 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient: _add_conns ranks=[1,4,0,2,8,7,5,6,3] 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d00a2950 0x7f14d01219a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): picked mon.noname-f con 0x7f14d00a2950 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 0x7f14d0121ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): picked mon.noname-c con 0x7f14d00a3270 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 0x7f14d0122420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): picked mon.noname-a con 0x7f14d0138040 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01232e0 con 0x7f14d0138040 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01250c0 con 0x7f14d00a2950 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0123930 con 0x7f14d00a3270 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:46.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14d4a2f700 10 monclient(hunting): authenticate will time out at 2826.350493s 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 0x7f14d0122420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 0x7f14d0121ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 0x7f14d0121ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54794/0 (socket says 172.21.15.47:54794) 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 0x7f14d0122420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53514/0 (socket says 172.21.15.47:53514) 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 1 -- 172.21.15.47:0/2678339325 learned_addr learned my addr 172.21.15.47:0/2678339325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): get_auth_request con 0x7f14d00a3270 auth_method 0 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): get_auth_request con 0x7f14d0138040 auth_method 0 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d00a2950 0x7f14d01219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): get_auth_request con 0x7f14d00a2950 auth_method 0 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14cdd9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ce59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.949+0000 7f14ced9d700 10 monclient(hunting): handle_auth_done global_id 104463 payload 824 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 -- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 msgr2=0x7f14d0121ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 0x7f14d0121ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 -- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d00a2950 msgr2=0x7f14d01219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d00a2950 0x7f14d01219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 monclient: found mon.noname-a 2024-04-20T02:35:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 -- 172.21.15.47:0/2678339325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8009550 con 0x7f14d0138040 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14cdd9b700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 0x7f14d0121ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ced9d700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 0x7f14d0122420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14c800efc0 tx=0x7f14c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 client.?.objecter ms_handle_connect 0x7f14d0138040 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14c8009e60 con 0x7f14d0138040 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: dump: 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: _finish_auth 0 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800f040 con 0x7f14d0138040 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 5 monclient: authenticate success, global_id 104463 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14c80148f0 con 0x7f14d0138040 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14ccd99700 4 set_mon_vals no callback set 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14c6ffd700 10 monclient: dump: 2024-04-20T02:35:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:46.979 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 10 monclient: _renew_subs 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00720c0 con 0x7f14d0138040 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 10 client.104463.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 10 monclient: _renew_subs 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d0124de0 con 0x7f14d0138040 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.950+0000 7f14d4a2f700 1 librados: init done 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14d4a2f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f14c8014480 con 0x7f14d0138040 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14d4a2f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14d4a2f700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0124c40 con 0x7f14d0138040 2024-04-20T02:35:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14c6ffd700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f14b003a350 0x7f14b003c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f14c804ae90 con 0x7f14d0138040 2024-04-20T02:35:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14c6ffd700 10 client.104463.objecter ms_dispatch 0x7f14d0001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:46.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14c6ffd700 3 client.104463.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.951+0000 7f14c6ffd700 3 client.104463.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.952+0000 7f14c6ffd700 20 client.104463.objecter dump_active .. 0 homeless 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.952+0000 7f14ce59c700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f14b003a350 0x7f14b003c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.952+0000 7f14ce59c700 10 monclient: get_auth_request con 0x7f14b003a350 auth_method 0 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.952+0000 7f14c6ffd700 10 client.104463.objecter ms_handle_connect 0x7f14b003a350 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.952+0000 7f14ce59c700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f14b003a350 0x7f14b003c810 secure :-1 s=READY pgs=3412 cs=0 l=1 rev1=1 crypto rx=0x7f14b8014040 tx=0x7f14b8008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.956+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f14d0124c40 con 0x7f14d0138040 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.956+0000 7f14c6ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:46.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:46.956+0000 7f14c6ffd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.173+0000 7f14d4a2f700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:35:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.173+0000 7f14d4a2f700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:35:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.173+0000 7f14d4a2f700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:47.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.173+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14d0124960 con 0x7f14d0138040 2024-04-20T02:35:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.174+0000 7f14c6ffd700 1 -- 172.21.15.47:0/2678339325 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v697) v1 ==== 74+0+14866 (secure 0 0 0) 0x7f14d0124960 con 0x7f14d0138040 2024-04-20T02:35:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.174+0000 7f14c6ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-04-20T02:35:47.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.174+0000 7f14c6ffd700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:35:47.175 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:35:47.175 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":697,"fsid":"72abca9e-d588-4ab7-bb3a-a08105100daf","created":"2024-04-20T02:15:34.248098+0000","modified":"2024-04-20T02:34:02.451775+0000","last_up_change":"2024-04-20T02:15:43.202903+0000","last_in_change":"2024-04-20T02:15:36.229103+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":99,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":260,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T02:15:44.345718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T02:15:44.739933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-04-20T02:16:42.185012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a1c1b335-9be6-43d2-94b3-8cd5fb042a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":692,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":42309},{"type":"v1","addr":"172.21.15.47:6801","nonce":42309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":42309},{"type":"v1","addr":"172.21.15.47:6803","nonce":42309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6809","nonce":42309},{"type":"v1","addr":"172.21.15.47:6811","nonce":42309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":42309},{"type":"v1","addr":"172.21.15.47:6806","nonce":42309}]},"public_addr":"172.21.15.47:6801/42309","cluster_addr":"172.21.15.47:6803/42309","heartbeat_back_addr":"172.21.15.47:6811/42309","heartbeat_front_addr":"172.21.15.47:6806/42309","state":["exists","up"]},{"osd":1,"uuid":"ced35bb3-1a5e-416a-9d1d-c68163082c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":692,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6805","nonce":42310},{"type":"v1","addr":"172.21.15.47:6807","nonce":42310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":42310},{"type":"v1","addr":"172.21.15.47:6813","nonce":42310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6819","nonce":42310},{"type":"v1","addr":"172.21.15.47:6821","nonce":42310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6815","nonce":42310},{"type":"v1","addr":"172.21.15.47:6817","nonce":42310}]},"public_addr":"172.21.15.47:6807/42310","cluster_addr":"172.21.15.47:6813/42310","heartbeat_back_addr":"172.21.15.47:6821/42310","heartbeat_front_addr":"172.21.15.47:6817/42310","state":["exists","up"]},{"osd":2,"uuid":"c932cb96-511e-4863-8792-aa8c0bbb0e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":692,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":42311},{"type":"v1","addr":"172.21.15.47:6812","nonce":42311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":42311},{"type":"v1","addr":"172.21.15.47:6816","nonce":42311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":42311},{"type":"v1","addr":"172.21.15.47:6823","nonce":42311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":42311},{"type":"v1","addr":"172.21.15.47:6820","nonce":42311}]},"public_addr":"172.21.15.47:6812/42311","cluster_addr":"172.21.15.47:6816/42311","heartbeat_back_addr":"172.21.15.47:6823/42311","heartbeat_front_addr":"172.21.15.47:6820/42311","state":["exists","up"]},{"osd":3,"uuid":"60f24ee2-e337-4b56-a45d-98a0cb6188bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":692,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":42294},{"type":"v1","addr":"172.21.15.179:6801","nonce":42294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":42294},{"type":"v1","addr":"172.21.15.179:6803","nonce":42294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":42294},{"type":"v1","addr":"172.21.15.179:6807","nonce":42294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":42294},{"type":"v1","addr":"172.21.15.179:6805","nonce":42294}]},"public_addr":"172.21.15.179:6801/42294","cluster_addr":"172.21.15.179:6803/42294","heartbeat_back_addr":"172.21.15.179:6807/42294","heartbeat_front_addr":"172.21.15.179:6805/42294","state":["exists","up"]},{"osd":4,"uuid":"78fd6b14-e8e8-420e-bb89-17b57d0b613e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":692,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":42295},{"type":"v1","addr":"172.21.15.179:6812","nonce":42295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":42295},{"type":"v1","addr":"172.21.15.179:6816","nonce":42295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":42295},{"type":"v1","addr":"172.21.15.179:6823","nonce":42295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":42295},{"type":"v1","addr":"172.21.15.179:6820","nonce":42295}]},"public_addr":"172.21.15.179:6812/42295","cluster_addr":"172.21.15.179:6816/42295","heartbeat_back_addr":"172.21.15.179:6823/42295","heartbeat_front_addr":"172.21.15.179:6820/42295","state":["exists","up"]},{"osd":5,"uuid":"fad619aa-0c5c-4c2d-82b3-510ce37c7c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":692,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":42296},{"type":"v1","addr":"172.21.15.179:6809","nonce":42296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":42296},{"type":"v1","addr":"172.21.15.179:6813","nonce":42296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":42296},{"type":"v1","addr":"172.21.15.179:6821","nonce":42296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":42296},{"type":"v1","addr":"172.21.15.179:6817","nonce":42296}]},"public_addr":"172.21.15.179:6809/42296","cluster_addr":"172.21.15.179:6813/42296","heartbeat_back_addr":"172.21.15.179:6821/42296","heartbeat_front_addr":"172.21.15.179:6817/42296","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.384773+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.348973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.447042+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.375411+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T02:15:41.151528+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":10,"snaps":[{"begin":1,"length":1},{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":13,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":2}]},{"pool":52,"snaps":[{"begin":6,"length":1},{"begin":11,"length":2}]},{"pool":76,"snaps":[{"begin":1,"length":1},{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":83,"snaps":[{"begin":2,"length":1}]},{"pool":91,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2}]},{"pool":92,"snaps":[{"begin":4,"length":4}]},{"pool":101,"snaps":[{"begin":2,"length":1}]},{"pool":106,"snaps":[{"begin":1,"length":1},{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":120,"snaps":[{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":123,"snaps":[{"begin":12,"length":1}]},{"pool":146,"snaps":[{"begin":14,"length":1}]},{"pool":157,"snaps":[{"begin":17,"length":1}]},{"pool":163,"snaps":[{"begin":19,"length":2}]},{"pool":175,"snaps":[{"begin":24,"length":1}]},{"pool":241,"snaps":[{"begin":2,"length":1}]},{"pool":242,"snaps":[{"begin":4,"length":1}]},{"pool":245,"snaps":[{"begin":6,"length":1}]},{"pool":249,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T02:35:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 10 librados: watch_flush enter 2024-04-20T02:35:47.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 10 librados: watch_flush exit 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 20 client.104463.objecter shutdown clearing up homeless session... 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 10 client.104463.objecter successfully canceled tick 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f14b003a350 msgr2=0x7f14b003c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f14b003a350 0x7f14b003c810 secure :-1 s=READY pgs=3412 cs=0 l=1 rev1=1 crypto rx=0x7f14b8014040 tx=0x7f14b8008030 comp rx=0 tx=0).stop 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 10 monclient: shutdown 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 msgr2=0x7f14d0122420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.176+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 0x7f14d0122420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14c800efc0 tx=0x7f14c800c3a0 comp rx=0 tx=0).stop 2024-04-20T02:35:47.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 shutdown_connections 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d0138040 0x7f14d0122420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f14b003a350 0x7f14b003c810 unknown :-1 s=CLOSED pgs=3412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14d00a2950 0x7f14d01219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 --2- 172.21.15.47:0/2678339325 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f14d00a3270 0x7f14d0121ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 >> 172.21.15.47:0/2678339325 conn(0x7f14d00a7570 msgr2=0x7f14d00a5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 shutdown_connections 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 -- 172.21.15.47:0/2678339325 wait complete. 2024-04-20T02:35:47.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:47.177+0000 7f14d4a2f700 1 librados: shutdown 2024-04-20T02:35:48.191 INFO:tasks.ceph:Scrubbing osd.0 2024-04-20T02:35:48.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-20T02:35:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.340+0000 7fd9228b0700 10 monclient: get_monmap_and_config 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.340+0000 7fd9228b0700 10 monclient: build_initial_monmap 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.340+0000 7fd9228b0700 10 monclient: monmap: 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:48.341895+0000 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:48.341895+0000 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:48.342 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.341+0000 7fd9228b0700 1 Processor -- start 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.341+0000 7fd9228b0700 1 -- start start 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.341+0000 7fd9228b0700 10 monclient: init 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient: _add_conns ranks=[4,1,8,2,7,5,3,6,9,0,10] 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c0a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): picked mon.noname-c con 0x7fd91c0a1aa0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd91c070840 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd91c070cc0 0x7fd91c135f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): picked mon.noname-e con 0x7fd91c070cc0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0751e0 con 0x7fd91c070840 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c072330 con 0x7fd91c0a1aa0 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c072610 con 0x7fd91c070cc0 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd9228b0700 10 monclient(hunting): authenticate will time out at 2827.743248s 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c0a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd91c070cc0 0x7fd91c135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c0a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54806/0 (socket says 172.21.15.47:54806) 2024-04-20T02:35:48.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 -- 172.21.15.47:0/2739661698 learned_addr learned my addr 172.21.15.47:0/2739661698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): get_auth_request con 0x7fd91c0a1aa0 auth_method 0 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): get_auth_request con 0x7fd91c070cc0 auth_method 0 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient(hunting): handle_auth_done global_id 84829 payload 274 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 -- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd91c070cc0 msgr2=0x7fd91c135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 --2- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd91c070cc0 0x7fd91c135f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 -- 172.21.15.47:0/2739661698 >> v1:172.21.15.179:6789/0 conn(0x7fd91c070840 legacy=0x7fd91c1357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 monclient: found mon.noname-c 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 -- 172.21.15.47:0/2739661698 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0728f0 con 0x7fd91c0a1aa0 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd920e4d700 1 --2- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd91c070cc0 0x7fd91c135f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.342+0000 7fd91bfff700 1 --2- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c0a1e80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009920 tx=0x7fd90c009c30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=835a294e1b352cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 1 -- 172.21.15.47:0/2739661698 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd90c011aa0 con 0x7fd91c0a1aa0 2024-04-20T02:35:48.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: dump: 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.357 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: _finish_auth 0 2024-04-20T02:35:48.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 5 monclient: authenticate success, global_id 84829 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 1 -- 172.21.15.47:0/2739661698 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c011c00 con 0x7fd91c0a1aa0 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 1 -- 172.21.15.47:0/2739661698 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd90c011d70 con 0x7fd91c0a1aa0 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 4 set_mon_vals no callback set 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd91affd700 10 monclient: dump: 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.359 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 10 monclient: shutdown 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 -- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 msgr2=0x7fd91c0a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 --2- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c0a1e80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009920 tx=0x7fd90c009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 -- 172.21.15.47:0/2739661698 shutdown_connections 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 --2- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fd91c070cc0 0x7fd91c135f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 --2- 172.21.15.47:0/2739661698 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c0a1e80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 -- 172.21.15.47:0/2739661698 >> 172.21.15.47:0/2739661698 conn(0x7fd91c066610 msgr2=0x7fd91c066a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 -- 172.21.15.47:0/2739661698 shutdown_connections 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.343+0000 7fd9228b0700 1 -- 172.21.15.47:0/2739661698 wait complete. 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.344+0000 7fd9228b0700 10 monclient: build_initial_monmap 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.344+0000 7fd9228b0700 10 monclient: monmap: 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:48.361 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.344+0000 7fd9228b0700 1 librados: starting msgr at 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.344+0000 7fd9228b0700 1 librados: starting objecter 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.344+0000 7fd9228b0700 1 Processor -- start 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 -- start start 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 librados: setting wanted keys 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 librados: calling monclient init 2024-04-20T02:35:48.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient: init 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient: _add_conns ranks=[4,3,5,0,7,8,6,2,1] 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c070840 0x7fd91c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): picked mon.noname-c con 0x7fd91c070840 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd91c070cc0 0x7fd91c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): picked mon.noname-g con 0x7fd91c070cc0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): picked mon.noname-h con 0x7fd91c0a1aa0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c123fd0 con 0x7fd91c070cc0 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1345d0 con 0x7fd91c070840 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c124c90 con 0x7fd91c0a1aa0 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:48.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd9228b0700 10 monclient(hunting): authenticate will time out at 2827.746789s 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c070840 0x7fd91c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c070840 0x7fd91c06c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54810/0 (socket says 172.21.15.47:54810) 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 1 -- 172.21.15.47:0/3722543864 learned_addr learned my addr 172.21.15.47:0/3722543864 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): get_auth_request con 0x7fd91c070840 auth_method 0 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.345+0000 7fd91b7fe700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd91c070cc0 0x7fd91c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd920e4d700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91b7fe700 10 monclient(hunting): get_auth_request con 0x7fd91c070cc0 auth_method 0 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91b7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd920e4d700 10 monclient(hunting): get_auth_request con 0x7fd91c0a1aa0 auth_method 0 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd920e4d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd920e4d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd920e4d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 10 monclient(hunting): handle_auth_done global_id 84838 payload 824 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 -- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd91c0a1aa0 msgr2=0x7fd91c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c06d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 -- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd91c070cc0 msgr2=0x7fd91c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd91c070cc0 0x7fd91c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 monclient: found mon.noname-c 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:48.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 -- 172.21.15.47:0/3722543864 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009a70 con 0x7fd91c070840 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91b7fe700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd91c070cc0 0x7fd91c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd920e4d700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fd91c0a1aa0 0x7fd91c06d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd91bfff700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c070840 0x7fd91c06c7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009840 tx=0x7fd90c004750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 client.?.objecter ms_handle_connect 0x7fd91c070840 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd90c0045f0 con 0x7fd91c070840 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: dump: 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.366 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: _finish_auth 0 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 5 monclient: authenticate success, global_id 84838 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c003b10 con 0x7fd91c070840 2024-04-20T02:35:48.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd90c003c80 con 0x7fd91c070840 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 10 monclient: _renew_subs 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c009550 con 0x7fd91c070840 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd919ffb700 4 set_mon_vals no callback set 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd918ff9700 10 monclient: dump: 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.368 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 10 client.84838.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 10 monclient: _renew_subs 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c09eea0 con 0x7fd91c070840 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.346+0000 7fd9228b0700 1 librados: init done 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd9228b0700 10 client.84838.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd9228b0700 15 client.84838.objecter _session_command_op_assign -1 1 2024-04-20T02:35:48.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd9228b0700 20 client.84838.objecter _get_session osd=-1 returning homeless 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd9228b0700 10 client.84838.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd9228b0700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd91c072bd0 con 0x7fd91c070840 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd90c021070 con 0x7fd91c070840 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd918ff9700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd90803a2f0 0x7fd90803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fd90c04b950 con 0x7fd91c070840 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd918ff9700 10 client.84838.objecter ms_dispatch 0x7fd91c001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd918ff9700 3 client.84838.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.347+0000 7fd918ff9700 3 client.84838.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 10 client.84838.objecter checking command 1 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7fd908041770 0x7fd908043c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 20 client.84838.objecter _get_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 15 client.84838.objecter _session_command_op_remove -1 1 2024-04-20T02:35:48.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 20 client.84838.objecter _get_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 20 client.84838.objecter get_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 15 client.84838.objecter _session_command_op_assign 0 1 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 4 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 10 client.84838.objecter _send_command 1 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 --> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9080442e0 con 0x7fd908041770 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 20 client.84838.objecter dump_active .. 0 homeless 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_get_version_reply(handle=1 version=697) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c0502c0 con 0x7fd91c070840 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 10 monclient: handle_get_version_reply finishing 1 version 697 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd920e4d700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7fd908041770 0x7fd908043c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd91b7fe700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd90803a2f0 0x7fd90803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd920e4d700 10 monclient: get_auth_request con 0x7fd908041770 auth_method 0 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd91b7fe700 10 monclient: get_auth_request con 0x7fd90803a2f0 auth_method 0 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 10 client.84838.objecter ms_handle_connect 0x7fd908041770 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd920e4d700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7fd908041770 0x7fd908043c10 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd918ff9700 10 client.84838.objecter ms_handle_connect 0x7fd90803a2f0 2024-04-20T02:35:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.348+0000 7fd91b7fe700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd90803a2f0 0x7fd90803c7b0 secure :-1 s=READY pgs=3413 cs=0 l=1 rev1=1 crypto rx=0x7fd91c06dd80 tx=0x7fd910009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.349+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== osd.0 v2:172.21.15.47:6800/42309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd9080442e0 con 0x7fd908041770 2024-04-20T02:35:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.349+0000 7fd918ff9700 10 client.84838.objecter ms_dispatch 0x7fd91c001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:35:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.349+0000 7fd918ff9700 10 client.84838.objecter _finish_command 1 = system:0 2024-04-20T02:35:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.349+0000 7fd918ff9700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 4 2024-04-20T02:35:48.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.349+0000 7fd918ff9700 15 client.84838.objecter _session_command_op_remove 0 1 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 10 client.84838.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 15 client.84838.objecter _session_command_op_assign -1 2 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 20 client.84838.objecter _get_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 20 client.84838.objecter _get_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 15 client.84838.objecter _session_command_op_remove -1 2 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 20 client.84838.objecter get_session s=0x7fd9080410a0 osd=0 3 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 15 client.84838.objecter _session_command_op_assign 0 2 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 4 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 10 client.84838.objecter _send_command 2 2024-04-20T02:35:48.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 --> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd91c0a3c00 con 0x7fd908041770 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd918ff9700 1 -- 172.21.15.47:0/3722543864 <== osd.0 v2:172.21.15.47:6800/42309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd91400f040 con 0x7fd908041770 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd918ff9700 10 client.84838.objecter ms_dispatch 0x7fd91c001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd918ff9700 10 client.84838.objecter _finish_command 2 = system:0 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd918ff9700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 4 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.364+0000 7fd918ff9700 15 client.84838.objecter _session_command_op_remove 0 2 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T02:35:48.374 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 10 librados: watch_flush enter 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 10 librados: watch_flush exit 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 10 client.84838.objecter close_session for osd.0 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7fd908041770 msgr2=0x7fd908043c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:6800/42309,v1:172.21.15.47:6801/42309] conn(0x7fd908041770 0x7fd908043c10 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 20 client.84838.objecter put_session s=0x7fd9080410a0 osd=0 1 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 20 client.84838.objecter shutdown clearing up homeless session... 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 10 client.84838.objecter successfully canceled tick 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd90803a2f0 msgr2=0x7fd90803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fd90803a2f0 0x7fd90803c7b0 secure :-1 s=READY pgs=3413 cs=0 l=1 rev1=1 crypto rx=0x7fd91c06dd80 tx=0x7fd910009040 comp rx=0 tx=0).stop 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 10 monclient: shutdown 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c070840 msgr2=0x7fd91c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 --2- 172.21.15.47:0/3722543864 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd91c070840 0x7fd91c06c7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009840 tx=0x7fd90c004750 comp rx=0 tx=0).stop 2024-04-20T02:35:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd920e4d700 1 -- 172.21.15.47:0/3722543864 reap_dead start 2024-04-20T02:35:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 shutdown_connections 2024-04-20T02:35:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 >> 172.21.15.47:0/3722543864 conn(0x7fd91c066610 msgr2=0x7fd91c09f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 shutdown_connections 2024-04-20T02:35:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 -- 172.21.15.47:0/3722543864 wait complete. 2024-04-20T02:35:48.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.365+0000 7fd9228b0700 1 librados: shutdown 2024-04-20T02:35:48.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-20T02:35:48.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.542+0000 7f17b2a45700 10 monclient: get_monmap_and_config 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.542+0000 7f17b2a45700 10 monclient: build_initial_monmap 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.542+0000 7f17b2a45700 10 monclient: monmap: 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:48.543969+0000 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:48.543969+0000 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:48.543 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:48.544 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:48.544 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.543+0000 7f17b2a45700 1 Processor -- start 2024-04-20T02:35:48.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.543+0000 7f17b2a45700 1 -- start start 2024-04-20T02:35:48.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.543+0000 7f17b2a45700 10 monclient: init 2024-04-20T02:35:48.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:48.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient: _add_conns ranks=[0,10,2,8,5,4,3,9,6,7,1] 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f17ac071630 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f17ac137930 0x7f17ac139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): picked mon.noname-f con 0x7f17ac137930 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f17ac13a260 0x7f17ac13c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): picked mon.noname-b con 0x7f17ac13a260 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac0751e0 con 0x7f17ac071630 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac072720 con 0x7f17ac137930 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0a0820 con 0x7f17ac13a260 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b2a45700 10 monclient(hunting): authenticate will time out at 2827.945403s 2024-04-20T02:35:48.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17abfff700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f17ac071630 0x7f17ac071a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57902/0 (socket says 172.21.15.47:57902) 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17abfff700 1 -- 172.21.15.47:0/3738066449 learned_addr learned my addr 172.21.15.47:0/3738066449 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 1 --2- 172.21.15.47:0/3738066449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f17ac13a260 0x7f17ac13c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): get_auth_request con 0x7f17ac13a260 auth_method 0 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 1 --2- 172.21.15.47:0/3738066449 >> v2:172.21.15.179:3300/0 conn(0x7f17ac137930 0x7f17ac139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): get_auth_request con 0x7f17ac137930 auth_method 0 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17ab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_done global_id 114230 payload 274 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 1 -- 172.21.15.47:0/3738066449 >> v2:172.21.15.179:3300/0 conn(0x7f17ac137930 msgr2=0x7f17ac139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 1 --2- 172.21.15.47:0/3738066449 >> v2:172.21.15.179:3300/0 conn(0x7f17ac137930 0x7f17ac139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 1 -- 172.21.15.47:0/3738066449 >> v1:172.21.15.47:6789/0 conn(0x7f17ac071630 legacy=0x7f17ac071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient: _finish_hunting 0 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 1 monclient: found mon.noname-b 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.544+0000 7f17b0fe2700 1 -- 172.21.15.47:0/3738066449 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac09e890 con 0x7f17ac13a260 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17b0fe2700 1 --2- 172.21.15.47:0/3738066449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f17ac13a260 0x7f17ac13c650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009dc0 tx=0x7f17a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4cce9aafed0c2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 1 -- 172.21.15.47:0/3738066449 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17a400e070 con 0x7f17ac13a260 2024-04-20T02:35:48.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: dump: 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: _finish_auth 0 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17b2a45700 5 monclient: authenticate success, global_id 114230 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17b2a45700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 1 -- 172.21.15.47:0/3738066449 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4003e90 con 0x7f17ac13a260 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17b2a45700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17b2a45700 4 set_mon_vals no callback set 2024-04-20T02:35:48.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 1 -- 172.21.15.47:0/3738066449 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17a400be90 con 0x7f17ac13a260 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.545+0000 7f17aaffd700 10 monclient: dump: 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.559 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 10 monclient: shutdown 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 -- 172.21.15.47:0/3738066449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f17ac13a260 msgr2=0x7f17ac13c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3738066449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f17ac13a260 0x7f17ac13c650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009dc0 tx=0x7f17a400b330 comp rx=0 tx=0).stop 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 -- 172.21.15.47:0/3738066449 shutdown_connections 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3738066449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f17ac13a260 0x7f17ac13c650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3738066449 >> v2:172.21.15.179:3300/0 conn(0x7f17ac137930 0x7f17ac139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 -- 172.21.15.47:0/3738066449 >> 172.21.15.47:0/3738066449 conn(0x7f17ac0a7570 msgr2=0x7f17ac0a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 -- 172.21.15.47:0/3738066449 shutdown_connections 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 1 -- 172.21.15.47:0/3738066449 wait complete. 2024-04-20T02:35:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 10 monclient: build_initial_monmap 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.546+0000 7f17b2a45700 10 monclient: monmap: 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:48.561 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 librados: starting msgr at 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 librados: starting objecter 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 Processor -- start 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 -- start start 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 librados: setting wanted keys 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 librados: calling monclient init 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 10 monclient: init 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 10 monclient: _add_conns ranks=[3,7,5,6,8,2,1,0,4] 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.547+0000 7f17b2a45700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17ac071630 0x7f17ac1c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): picked mon.noname-g con 0x7f17ac071630 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:48.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17ac137930 0x7f17ac136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): picked mon.noname-i con 0x7f17ac137930 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f17ac13a260 0x7f17ac13a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): picked mon.noname-h con 0x7f17ac13a260 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac1318e0 con 0x7f17ac071630 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac072380 con 0x7f17ac13a260 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac1323f0 con 0x7f17ac137930 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b2a45700 10 monclient(hunting): authenticate will time out at 2827.949083s 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17ac137930 0x7f17ac136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17ac071630 0x7f17ac1c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17ac137930 0x7f17ac136ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3303/0 says I am v2:172.21.15.47:41024/0 (socket says 172.21.15.47:41024) 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 1 -- 172.21.15.47:0/3351507835 learned_addr learned my addr 172.21.15.47:0/3351507835 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): get_auth_request con 0x7f17ac071630 auth_method 0 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f17ac13a260 0x7f17ac13a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): get_auth_request con 0x7f17ac137930 auth_method 0 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): get_auth_request con 0x7f17ac13a260 auth_method 0 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17ab7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient(hunting): handle_auth_done global_id 104403 payload 824 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 -- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17ac137930 msgr2=0x7f17ac136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17ac137930 0x7f17ac136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 -- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f17ac13a260 msgr2=0x7f17ac13a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f17ac13a260 0x7f17ac13a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 monclient: found mon.noname-g 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 -- 172.21.15.47:0/3351507835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4009a70 con 0x7f17ac071630 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17b0fe2700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f17ac13a260 0x7f17ac13a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17abfff700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17ac071630 0x7f17ac1c9f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c7b0 tx=0x7f179c00cac0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17a8ff9700 10 client.?.objecter ms_handle_connect 0x7f17ac071630 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.548+0000 7f17a8ff9700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f179c0070d0 con 0x7f17ac071630 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: dump: 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.566 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.567 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: _finish_auth 0 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: _check_auth_tickets 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c007250 con 0x7f17ac071630 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f179c0133c0 con 0x7f17ac071630 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 5 monclient: authenticate success, global_id 104403 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 10 monclient: _renew_subs 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac1ca440 con 0x7f17ac071630 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17aa7fc700 4 set_mon_vals no callback set 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17a8ff9700 10 monclient: dump: 2024-04-20T02:35:48.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 10 client.104403.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 10 monclient: _renew_subs 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac132080 con 0x7f17ac071630 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.549+0000 7f17b2a45700 1 librados: init done 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mon.3 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f179c0073c0 con 0x7f17ac071630 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17b2a45700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f179803a320 0x7f179803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17b2a45700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17b2a45700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.179:3301/0 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac1326d0 con 0x7f17ac071630 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mon.3 v2:172.21.15.179:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f179c04a080 con 0x7f17ac071630 2024-04-20T02:35:48.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 10 client.104403.objecter ms_dispatch 0x7f17ac001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 3 client.104403.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 3 client.104403.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 20 client.104403.objecter dump_active .. 0 homeless 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17ab7fe700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f179803a320 0x7f179803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17ab7fe700 10 monclient: get_auth_request con 0x7f179803a320 auth_method 0 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17a8ff9700 10 client.104403.objecter ms_handle_connect 0x7f179803a320 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.550+0000 7f17ab7fe700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f179803a320 0x7f179803c7e0 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 crypto rx=0x7f17ac13bce0 tx=0x7f17a0009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.555+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mon.3 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f179c013520 con 0x7f17ac071630 2024-04-20T02:35:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.555+0000 7f17a8ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:48.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.555+0000 7f17a8ff9700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:48.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.773+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f17ac072120 con 0x7f179803a320 2024-04-20T02:35:48.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.774+0000 7f17a8ff9700 1 -- 172.21.15.47:0/3351507835 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f17a0009db0 con 0x7f179803a320 2024-04-20T02:35:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 10 librados: watch_flush enter 2024-04-20T02:35:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 10 librados: watch_flush exit 2024-04-20T02:35:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 20 client.104403.objecter shutdown clearing up homeless session... 2024-04-20T02:35:48.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 10 client.104403.objecter successfully canceled tick 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f179803a320 msgr2=0x7f179803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f179803a320 0x7f179803c7e0 secure :-1 s=READY pgs=3414 cs=0 l=1 rev1=1 crypto rx=0x7f17ac13bce0 tx=0x7f17a0009210 comp rx=0 tx=0).stop 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.776+0000 7f17b2a45700 10 monclient: shutdown 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17ac071630 msgr2=0x7f17ac1c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17ac071630 0x7f17ac1c9f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c7b0 tx=0x7f179c00cac0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 shutdown_connections 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f179803a320 0x7f179803c7e0 unknown :-1 s=CLOSED pgs=3414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f17ac13a260 0x7f17ac13a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f17ac071630 0x7f17ac1c9f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 --2- 172.21.15.47:0/3351507835 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f17ac137930 0x7f17ac136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 >> 172.21.15.47:0/3351507835 conn(0x7f17ac0a7570 msgr2=0x7f17ac0701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 shutdown_connections 2024-04-20T02:35:48.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 -- 172.21.15.47:0/3351507835 wait complete. 2024-04-20T02:35:48.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.777+0000 7f17b2a45700 1 librados: shutdown 2024-04-20T02:35:48.779 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2024-04-20T02:35:48.790 INFO:tasks.ceph:Scrubbing osd.1 2024-04-20T02:35:48.791 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-20T02:35:48.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.930+0000 7f2e678a9700 10 monclient: get_monmap_and_config 2024-04-20T02:35:48.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.930+0000 7f2e678a9700 10 monclient: build_initial_monmap 2024-04-20T02:35:48.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.930+0000 7f2e678a9700 10 monclient: monmap: 2024-04-20T02:35:48.930 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:48.931670+0000 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:48.931670+0000 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:48.931 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:48.932 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:48.932 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 1 Processor -- start 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 1 -- start start 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 10 monclient: init 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 10 monclient: _add_conns ranks=[10,5,1,4,8,0,9,7,6,2,3] 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e60071b50 0x7f2e60071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 10 monclient(hunting): picked mon.noname-f con 0x7f2e60071b50 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2e60072a70 0x7f2e60072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.931+0000 7f2e678a9700 10 monclient(hunting): picked mon.noname-h con 0x7f2e60072a70 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2e60137fe0 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e600751e0 con 0x7f2e60137fe0 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e600a4090 con 0x7f2e60071b50 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6006e610 con 0x7f2e60072a70 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e678a9700 10 monclient(hunting): authenticate will time out at 2828.333064s 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e60071b50 0x7f2e60071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2e60072a70 0x7f2e60072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2e60071b50 0x7f2e60071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:50702/0 (socket says 172.21.15.47:50702) 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 -- 172.21.15.47:0/380191518 learned_addr learned my addr 172.21.15.47:0/380191518 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): get_auth_request con 0x7f2e60071b50 auth_method 0 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65e46700 1 --1- 172.21.15.47:0/380191518 >> v1:172.21.15.179:6789/0 conn(0x7f2e60137fe0 0x7f2e6006c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:36836/0 (socket says 172.21.15.47:36836) 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): get_auth_request con 0x7f2e60072a70 auth_method 0 2024-04-20T02:35:48.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e64e44700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient(hunting): handle_auth_done global_id 56116 payload 274 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 -- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2e60072a70 msgr2=0x7f2e60072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 --2- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2e60072a70 0x7f2e60072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 -- 172.21.15.47:0/380191518 >> v1:172.21.15.179:6789/0 conn(0x7f2e60137fe0 legacy=0x7f2e6006c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient: _finish_hunting 0 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 monclient: found mon.noname-f 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:48.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 -- 172.21.15.47:0/380191518 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6006e980 con 0x7f2e60071b50 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.932+0000 7f2e65645700 1 --2- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60071b50 0x7f2e60071f30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2e500098f0 tx=0x7f2e50009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69476f1dfc83eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 1 -- 172.21.15.47:0/380191518 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e50011aa0 con 0x7f2e60071b50 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: dump: 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.946 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.947 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 1 -- 172.21.15.47:0/380191518 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50011c00 con 0x7f2e60071b50 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 1 -- 172.21.15.47:0/380191518 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e50011d70 con 0x7f2e60071b50 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 5 monclient: authenticate success, global_id 56116 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:48.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 4 set_mon_vals no callback set 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e57fff700 10 monclient: dump: 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.949 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 10 monclient: shutdown 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 -- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60071b50 msgr2=0x7f2e60071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 --2- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60071b50 0x7f2e60071f30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2e500098f0 tx=0x7f2e50009c00 comp rx=0 tx=0).stop 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 -- 172.21.15.47:0/380191518 shutdown_connections 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 --2- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60071b50 0x7f2e60071f30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 --2- 172.21.15.47:0/380191518 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f2e60072a70 0x7f2e60072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 -- 172.21.15.47:0/380191518 >> 172.21.15.47:0/380191518 conn(0x7f2e60066610 msgr2=0x7f2e60066a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 -- 172.21.15.47:0/380191518 shutdown_connections 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.933+0000 7f2e678a9700 1 -- 172.21.15.47:0/380191518 wait complete. 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 10 monclient: build_initial_monmap 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 10 monclient: monmap: 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:48.951 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 1 librados: starting msgr at 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 1 librados: starting objecter 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 1 Processor -- start 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 1 -- start start 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 1 librados: setting wanted keys 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 1 librados: calling monclient init 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.934+0000 7f2e678a9700 10 monclient: init 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient: _add_conns ranks=[2,6,1,0,5,8,4,3,7] 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e60071b50 0x7f2e601cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): picked mon.noname-b con 0x7f2e60071b50 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e60072a70 0x7f2e60134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): picked mon.noname-d con 0x7f2e60072a70 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60137fe0 0x7f2e60135240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): picked mon.noname-f con 0x7f2e60137fe0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e60120930 con 0x7f2e60137fe0 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e601ccf20 con 0x7f2e60071b50 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6006d8b0 con 0x7f2e60072a70 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e678a9700 10 monclient(hunting): authenticate will time out at 2828.336416s 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e60071b50 0x7f2e601cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e60072a70 0x7f2e60134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e60071b50 0x7f2e601cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60230/0 (socket says 172.21.15.47:60230) 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 -- 172.21.15.47:0/4124407371 learned_addr learned my addr 172.21.15.47:0/4124407371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): get_auth_request con 0x7f2e60071b50 auth_method 0 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): get_auth_request con 0x7f2e60072a70 auth_method 0 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60137fe0 0x7f2e60135240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): get_auth_request con 0x7f2e60137fe0 auth_method 0 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:48.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e64e44700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65e46700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient(hunting): handle_auth_done global_id 114239 payload 824 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 -- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e60072a70 msgr2=0x7f2e60134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2e60072a70 0x7f2e60134d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 -- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60137fe0 msgr2=0x7f2e60135240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e60137fe0 0x7f2e60135240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient: _finish_hunting 0 2024-04-20T02:35:48.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 monclient: found mon.noname-b 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.935+0000 7f2e65645700 1 -- 172.21.15.47:0/4124407371 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e50009550 con 0x7f2e60071b50 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e65645700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e60071b50 0x7f2e601cc2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e600703f0 tx=0x7f2e50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 client.?.objecter ms_handle_connect 0x7f2e60071b50 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e500043d0 con 0x7f2e60071b50 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:48.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: dump: 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.957 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: _finish_auth 0 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 5 monclient: authenticate success, global_id 114239 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50004550 con 0x7f2e60071b50 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 10 monclient: _renew_subs 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:48.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6006de00 con 0x7f2e60071b50 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e50024470 con 0x7f2e60071b50 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e55ffb700 10 monclient: dump: 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:48.959 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 10 client.114239.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e577fe700 4 set_mon_vals no callback set 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 10 monclient: _renew_subs 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e60121c10 con 0x7f2e60071b50 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.936+0000 7f2e678a9700 1 librados: init done 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e678a9700 10 client.114239.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e678a9700 15 client.114239.objecter _session_command_op_assign -1 1 2024-04-20T02:35:48.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e678a9700 20 client.114239.objecter _get_session osd=-1 returning homeless 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e678a9700 10 client.114239.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e678a9700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e600a4460 con 0x7f2e60071b50 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2e50021070 con 0x7f2e60071b50 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2e4403a2f0 0x7f2e4403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f2e5004b8b0 con 0x7f2e60071b50 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 10 client.114239.objecter ms_dispatch 0x7f2e60001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 3 client.114239.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 3 client.114239.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 10 client.114239.objecter checking command 1 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7f2e44041770 0x7f2e44043c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 20 client.114239.objecter _get_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 15 client.114239.objecter _session_command_op_remove -1 1 2024-04-20T02:35:48.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 20 client.114239.objecter _get_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 20 client.114239.objecter get_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 15 client.114239.objecter _session_command_op_assign 1 1 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 4 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 10 client.114239.objecter _send_command 1 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e440442e0 con 0x7f2e44041770 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 20 client.114239.objecter dump_active .. 0 homeless 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=697) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e500502c0 con 0x7f2e60071b50 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e55ffb700 10 monclient: handle_get_version_reply finishing 1 version 697 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e64e44700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2e4403a2f0 0x7f2e4403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.937+0000 7f2e64e44700 10 monclient: get_auth_request con 0x7f2e4403a2f0 auth_method 0 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.938+0000 7f2e65e46700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7f2e44041770 0x7f2e44043c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.938+0000 7f2e65e46700 10 monclient: get_auth_request con 0x7f2e44041770 auth_method 0 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.938+0000 7f2e55ffb700 10 client.114239.objecter ms_handle_connect 0x7f2e4403a2f0 2024-04-20T02:35:48.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.938+0000 7f2e64e44700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2e4403a2f0 0x7f2e4403c7b0 secure :-1 s=READY pgs=3415 cs=0 l=1 rev1=1 crypto rx=0x7f2e600a24c0 tx=0x7f2e5c009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.938+0000 7f2e55ffb700 10 client.114239.objecter ms_handle_connect 0x7f2e44041770 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.938+0000 7f2e65e46700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7f2e44041770 0x7f2e44043c10 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.939+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== osd.1 v2:172.21.15.47:6805/42310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2e440442e0 con 0x7f2e44041770 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.939+0000 7f2e55ffb700 10 client.114239.objecter ms_dispatch 0x7f2e60001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.939+0000 7f2e55ffb700 10 client.114239.objecter _finish_command 1 = system:0 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.939+0000 7f2e55ffb700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 4 2024-04-20T02:35:48.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.939+0000 7f2e55ffb700 15 client.114239.objecter _session_command_op_remove 1 1 2024-04-20T02:35:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 10 client.114239.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-04-20T02:35:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 15 client.114239.objecter _session_command_op_assign -1 2 2024-04-20T02:35:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 20 client.114239.objecter _get_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 20 client.114239.objecter _get_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 15 client.114239.objecter _session_command_op_remove -1 2 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 20 client.114239.objecter get_session s=0x7f2e440410a0 osd=1 3 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 15 client.114239.objecter _session_command_op_assign 1 2 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 4 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 10 client.114239.objecter _send_command 2 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 --> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2e6006ceb0 con 0x7f2e44041770 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e55ffb700 1 -- 172.21.15.47:0/4124407371 <== osd.1 v2:172.21.15.47:6805/42310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2e580071c0 con 0x7f2e44041770 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e55ffb700 10 client.114239.objecter ms_dispatch 0x7f2e60001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e55ffb700 10 client.114239.objecter _finish_command 2 = system:0 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e55ffb700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 4 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.954+0000 7f2e55ffb700 15 client.114239.objecter _session_command_op_remove 1 2 2024-04-20T02:35:48.965 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:48.966 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T02:35:48.966 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 10 librados: watch_flush enter 2024-04-20T02:35:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 10 librados: watch_flush exit 2024-04-20T02:35:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 10 client.114239.objecter close_session for osd.1 2024-04-20T02:35:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7f2e44041770 msgr2=0x7f2e44043c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:6805/42310,v1:172.21.15.47:6807/42310] conn(0x7f2e44041770 0x7f2e44043c10 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 20 client.114239.objecter put_session s=0x7f2e440410a0 osd=1 1 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 20 client.114239.objecter shutdown clearing up homeless session... 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 10 client.114239.objecter successfully canceled tick 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2e4403a2f0 msgr2=0x7f2e4403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2e4403a2f0 0x7f2e4403c7b0 secure :-1 s=READY pgs=3415 cs=0 l=1 rev1=1 crypto rx=0x7f2e600a24c0 tx=0x7f2e5c009210 comp rx=0 tx=0).stop 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 10 monclient: shutdown 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e60071b50 msgr2=0x7f2e601cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 --2- 172.21.15.47:0/4124407371 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2e60071b50 0x7f2e601cc2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e600703f0 tx=0x7f2e50004750 comp rx=0 tx=0).stop 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e65e46700 1 -- 172.21.15.47:0/4124407371 reap_dead start 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 shutdown_connections 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 >> 172.21.15.47:0/4124407371 conn(0x7f2e60066610 msgr2=0x7f2e60070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 shutdown_connections 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 -- 172.21.15.47:0/4124407371 wait complete. 2024-04-20T02:35:48.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:48.955+0000 7f2e678a9700 1 librados: shutdown 2024-04-20T02:35:48.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-20T02:35:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.107+0000 7f032bafb700 10 monclient: get_monmap_and_config 2024-04-20T02:35:49.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.107+0000 7f032bafb700 10 monclient: build_initial_monmap 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.107+0000 7f032bafb700 10 monclient: monmap: 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:49.108734+0000 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:49.108734+0000 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:49.108 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:49.109 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 Processor -- start 2024-04-20T02:35:49.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 -- start start 2024-04-20T02:35:49.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient: init 2024-04-20T02:35:49.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient: _add_conns ranks=[4,6,3,5,8,9,0,7,2,10,1] 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03240a1510 0x7f03240a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient(hunting): picked mon.noname-c con 0x7f03240a1510 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 0x7f03240a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient(hunting): picked mon.noname-d con 0x7f032409f9a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f032409fe00 0x7f03240a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient(hunting): picked mon.noname-g con 0x7f032409fe00 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240720a0 con 0x7f032409fe00 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0324070640 con 0x7f03240a1510 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.108+0000 7f032bafb700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0324071d30 con 0x7f032409f9a0 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032bafb700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032bafb700 10 monclient(hunting): authenticate will time out at 2828.510019s 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 0x7f03240a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03240a1510 0x7f03240a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 0x7f03240a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56340/0 (socket says 172.21.15.47:56340) 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 -- 172.21.15.47:0/3023675989 learned_addr learned my addr 172.21.15.47:0/3023675989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): get_auth_request con 0x7f032409f9a0 auth_method 0 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): get_auth_request con 0x7f03240a1510 auth_method 0 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032a098700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f032409fe00 0x7f03240a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032a098700 10 monclient(hunting): get_auth_request con 0x7f032409fe00 auth_method 0 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032a098700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032a098700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032a098700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient(hunting): handle_auth_done global_id 55653 payload 274 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 -- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03240a1510 msgr2=0x7f03240a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03240a1510 0x7f03240a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 -- 172.21.15.47:0/3023675989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f032409fe00 msgr2=0x7f03240a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f032409fe00 0x7f03240a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient: _finish_hunting 0 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 monclient: found mon.noname-d 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:49.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 -- 172.21.15.47:0/3023675989 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240751e0 con 0x7f032409f9a0 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329897700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03240a1510 0x7f03240a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032a098700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f032409fe00 0x7f03240a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0329096700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 0x7f03240a38f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0320009920 tx=0x7f0320009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f8a1996f903e39e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 1 -- 172.21.15.47:0/3023675989 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0320011aa0 con 0x7f032409f9a0 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: dump: 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.126 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.127 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: _finish_auth 0 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: _check_auth_tickets 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 1 -- 172.21.15.47:0/3023675989 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320011c00 con 0x7f032409f9a0 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032bafb700 5 monclient: authenticate success, global_id 55653 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f032bafb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 1 -- 172.21.15.47:0/3023675989 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0320011d70 con 0x7f032409f9a0 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.109+0000 7f0328895700 10 monclient: dump: 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 4 set_mon_vals no callback set 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 10 monclient: shutdown 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 -- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 msgr2=0x7f03240a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 0x7f03240a38f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0320009920 tx=0x7f0320009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 -- 172.21.15.47:0/3023675989 shutdown_connections 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f03240a1510 0x7f03240a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f032409fe00 0x7f03240a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 --2- 172.21.15.47:0/3023675989 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f032409f9a0 0x7f03240a38f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 -- 172.21.15.47:0/3023675989 >> 172.21.15.47:0/3023675989 conn(0x7f03240a8510 msgr2=0x7f03240a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 -- 172.21.15.47:0/3023675989 shutdown_connections 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.110+0000 7f032bafb700 1 -- 172.21.15.47:0/3023675989 wait complete. 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 10 monclient: build_initial_monmap 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 10 monclient: monmap: 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 1 librados: starting msgr at 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 1 librados: starting objecter 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 1 Processor -- start 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 1 -- start start 2024-04-20T02:35:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 1 librados: setting wanted keys 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 1 librados: calling monclient init 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.111+0000 7f032bafb700 10 monclient: init 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient: _add_conns ranks=[0,5,7,8,1,4,6,3,2] 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 0x7f0324124150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): picked mon.noname-a con 0x7f032409f9a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f032409fe00 0x7f0324124690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): picked mon.noname-h con 0x7f032409fe00 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f03240a1510 0x7f0324124bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): picked mon.noname-i con 0x7f03240a1510 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0324126470 con 0x7f032409f9a0 2024-04-20T02:35:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03241281f0 con 0x7f032409fe00 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f03240a2130 con 0x7f03240a1510 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032bafb700 10 monclient(hunting): authenticate will time out at 2828.513411s 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 0x7f0324124150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 0x7f0324124150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53520/0 (socket says 172.21.15.47:53520) 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 -- 172.21.15.47:0/2846870806 learned_addr learned my addr 172.21.15.47:0/2846870806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): get_auth_request con 0x7f032409f9a0 auth_method 0 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329096700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f032409fe00 0x7f0324124690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032a098700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f03240a1510 0x7f0324124bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329096700 10 monclient(hunting): get_auth_request con 0x7f032409fe00 auth_method 0 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329096700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329096700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329096700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient(hunting): handle_auth_done global_id 104481 payload 824 2024-04-20T02:35:49.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 -- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f03240a1510 msgr2=0x7f0324124bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f03240a1510 0x7f0324124bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 -- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f032409fe00 msgr2=0x7f0324124690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f032409fe00 0x7f0324124690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient: _finish_hunting 0 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 monclient: found mon.noname-a 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f0329897700 1 -- 172.21.15.47:0/2846870806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140094a0 con 0x7f032409f9a0 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032a098700 10 monclient: get_auth_request con 0x7f03240a1510 auth_method 0 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.112+0000 7f032a098700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f03240a1510 0x7f0324124bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T02:35:49.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f0329897700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 0x7f0324124150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f031400ebe0 tx=0x7f0314009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 client.?.objecter ms_handle_connect 0x7f032409f9a0 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f031400cd20 con 0x7f032409f9a0 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: dump: 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.137 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:49.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031400cea0 con 0x7f032409f9a0 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 5 monclient: authenticate success, global_id 104481 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 10 monclient: _renew_subs 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0320009550 con 0x7f032409f9a0 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03140138e0 con 0x7f032409f9a0 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031bfff700 4 set_mon_vals no callback set 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f031a7fc700 10 monclient: dump: 2024-04-20T02:35:49.139 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.140 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 10 client.104481.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 10 monclient: _renew_subs 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0324127dd0 con 0x7f032409f9a0 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.113+0000 7f032bafb700 1 librados: init done 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0314013470 con 0x7f032409f9a0 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f030803a2f0 0x7f030803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f031404a400 con 0x7f032409f9a0 2024-04-20T02:35:49.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 10 client.104481.objecter ms_dispatch 0x7f0324001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 3 client.104481.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 3 client.104481.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f032bafb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f032bafb700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f032bafb700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f031a7fc700 20 client.104481.objecter dump_active .. 0 homeless 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0324125a70 con 0x7f032409f9a0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f0329096700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f030803a2f0 0x7f030803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f0329096700 10 monclient: get_auth_request con 0x7f030803a2f0 auth_method 0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.115+0000 7f031a7fc700 10 client.104481.objecter ms_handle_connect 0x7f030803a2f0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.114+0000 7f0329096700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f030803a2f0 0x7f030803c7b0 secure :-1 s=READY pgs=3416 cs=0 l=1 rev1=1 crypto rx=0x7f0320009920 tx=0x7f0320008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.119+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f0314013720 con 0x7f032409f9a0 2024-04-20T02:35:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.119+0000 7f031a7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.119+0000 7f031a7fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:49.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.335+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f03240a1e30 con 0x7f030803a2f0 2024-04-20T02:35:49.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.336+0000 7f031a7fc700 1 -- 172.21.15.47:0/2846870806 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03200118f0 con 0x7f030803a2f0 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 10 librados: watch_flush enter 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 10 librados: watch_flush exit 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 20 client.104481.objecter shutdown clearing up homeless session... 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 10 client.104481.objecter successfully canceled tick 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f030803a2f0 msgr2=0x7f030803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f030803a2f0 0x7f030803c7b0 secure :-1 s=READY pgs=3416 cs=0 l=1 rev1=1 crypto rx=0x7f0320009920 tx=0x7f0320008030 comp rx=0 tx=0).stop 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 10 monclient: shutdown 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 msgr2=0x7f0324124150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 0x7f0324124150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f031400ebe0 tx=0x7f0314009e20 comp rx=0 tx=0).stop 2024-04-20T02:35:49.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 shutdown_connections 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f030803a2f0 0x7f030803c7b0 unknown :-1 s=CLOSED pgs=3416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f03240a1510 0x7f0324124bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f032409f9a0 0x7f0324124150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 --2- 172.21.15.47:0/2846870806 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f032409fe00 0x7f0324124690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 >> 172.21.15.47:0/2846870806 conn(0x7f03240a8510 msgr2=0x7f03240a4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 shutdown_connections 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 -- 172.21.15.47:0/2846870806 wait complete. 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.338+0000 7f032bafb700 1 librados: shutdown 2024-04-20T02:35:49.340 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2024-04-20T02:35:49.352 INFO:tasks.ceph:Scrubbing osd.2 2024-04-20T02:35:49.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-20T02:35:49.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.496+0000 7f0155632700 10 monclient: get_monmap_and_config 2024-04-20T02:35:49.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.496+0000 7f0155632700 10 monclient: build_initial_monmap 2024-04-20T02:35:49.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.496+0000 7f0155632700 10 monclient: monmap: 2024-04-20T02:35:49.496 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:49.496 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:49.497495+0000 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:49.497495+0000 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:49.497 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:49.498 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.496+0000 7f0155632700 1 Processor -- start 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 1 -- start start 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient: init 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient: _add_conns ranks=[3,1,5,7,10,2,4,8,6,0,9] 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f01500a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): picked mon.noname-g con 0x7f0150137e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0150138270 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f01500a6200 0x7f015013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): picked mon.noname-h con 0x7f01500a6200 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01500751e0 con 0x7f0150138270 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500a4220 con 0x7f0150137e90 2024-04-20T02:35:49.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f015009e0e0 con 0x7f01500a6200 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f0155632700 10 monclient(hunting): authenticate will time out at 2828.898700s 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f01500a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014f7fe700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f01500a6200 0x7f015013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f01500a3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.47:55482/0 (socket says 172.21.15.47:55482) 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014e7fc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0150138270 0x7f01500a5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.47:36848/0 (socket says 172.21.15.47:36848) 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 1 -- 172.21.15.47:0/1185322116 learned_addr learned my addr 172.21.15.47:0/1185322116 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:49.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014f7fe700 1 --2- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f01500a6200 0x7f015013fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3302/0 says I am v2:172.21.15.47:42064/0 (socket says 172.21.15.47:42064) 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 10 monclient(hunting): get_auth_request con 0x7f0150137e90 auth_method 0 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014f7fe700 10 monclient(hunting): get_auth_request con 0x7f01500a6200 auth_method 0 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.497+0000 7f014f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 10 monclient(hunting): handle_auth_done global_id 104421 payload 274 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 1 -- 172.21.15.47:0/1185322116 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725860343 0 0) 0x7f01500751e0 con 0x7f0150138270 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 1 -- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f01500a6200 msgr2=0x7f015013fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 1 --2- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f01500a6200 0x7f015013fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 1 -- 172.21.15.47:0/1185322116 >> v1:172.21.15.179:6789/0 conn(0x7f0150138270 legacy=0x7f01500a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 1 monclient: found mon.noname-g 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.179:3301/0 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 1 -- 172.21.15.47:0/1185322116 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500a2d30 con 0x7f0150137e90 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:35:49.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014effd700 1 --2- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f01500a3ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0140009920 tx=0x7f0140009c30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=2078f9d04ace4abb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 1 -- 172.21.15.47:0/1185322116 <== mon.3 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0140011aa0 con 0x7f0150137e90 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: dump: 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.515 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: _finish_auth 0 2024-04-20T02:35:49.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: _check_auth_tickets 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 1 -- 172.21.15.47:0/1185322116 <== mon.3 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140011c00 con 0x7f0150137e90 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f0155632700 5 monclient: authenticate success, global_id 104421 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f0155632700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 1 -- 172.21.15.47:0/1185322116 <== mon.3 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0140011d70 con 0x7f0150137e90 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f014dffb700 10 monclient: dump: 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.517 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f0155632700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.498+0000 7f0155632700 4 set_mon_vals no callback set 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 10 monclient: shutdown 2024-04-20T02:35:49.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 -- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 msgr2=0x7f01500a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 --2- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f01500a3ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0140009920 tx=0x7f0140009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 -- 172.21.15.47:0/1185322116 shutdown_connections 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 --2- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f01500a6200 0x7f015013fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 --2- 172.21.15.47:0/1185322116 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f01500a3ce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 -- 172.21.15.47:0/1185322116 >> 172.21.15.47:0/1185322116 conn(0x7f01500aa710 msgr2=0x7f01500a6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 -- 172.21.15.47:0/1185322116 shutdown_connections 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.499+0000 7f0155632700 1 -- 172.21.15.47:0/1185322116 wait complete. 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 10 monclient: build_initial_monmap 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 10 monclient: monmap: 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:49.519 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 1 librados: starting msgr at 2024-04-20T02:35:49.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 1 librados: starting objecter 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 1 Processor -- start 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 1 -- start start 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 1 librados: setting wanted keys 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 1 librados: calling monclient init 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.500+0000 7f0155632700 10 monclient: init 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient: _add_conns ranks=[8,3,2,1,7,0,6,4,5] 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f01500a6200 0x7f01501cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): picked mon.noname-e con 0x7f01500a6200 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f0150136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): picked mon.noname-g con 0x7f0150137e90 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0150138270 0x7f0150137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): picked mon.noname-b con 0x7f0150138270 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01501cce10 con 0x7f0150138270 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01501cc670 con 0x7f0150137e90 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0150124570 con 0x7f01500a6200 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f0155632700 10 monclient(hunting): authenticate will time out at 2828.902301s 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0150138270 0x7f0150137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f01500a6200 0x7f01501cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f01500a6200 0x7f01501cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48502/0 (socket says 172.21.15.47:48502) 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0150138270 0x7f0150137320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60246/0 (socket says 172.21.15.47:60246) 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 1 -- 172.21.15.47:0/2262202246 learned_addr learned my addr 172.21.15.47:0/2262202246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): get_auth_request con 0x7f01500a6200 auth_method 0 2024-04-20T02:35:49.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): get_auth_request con 0x7f0150138270 auth_method 0 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014e7fc700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f0150136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014e7fc700 10 monclient(hunting): get_auth_request con 0x7f0150137e90 auth_method 0 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014e7fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient(hunting): handle_auth_done global_id 114248 payload 824 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 -- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f01500a6200 msgr2=0x7f01501cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f01500a6200 0x7f01501cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 -- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 msgr2=0x7f0150136de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f0150136de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient: _finish_hunting 0 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 monclient: found mon.noname-b 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014f7fe700 1 -- 172.21.15.47:0/2262202246 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009550 con 0x7f0150138270 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014effd700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f01500a6200 0x7f01501cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.501+0000 7f014e7fc700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0150137e90 0x7f0150136de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f014f7fe700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0150138270 0x7f0150137320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f014800efc0 tx=0x7f014800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 client.?.objecter ms_handle_connect 0x7f0150138270 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0148009e60 con 0x7f0150138270 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: dump: 2024-04-20T02:35:49.525 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.526 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: _finish_auth 0 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: _check_auth_tickets 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 5 monclient: authenticate success, global_id 114248 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014800f040 con 0x7f0150138270 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01480148f0 con 0x7f0150138270 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 10 monclient: _renew_subs 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01500a4390 con 0x7f0150138270 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f014d7fa700 4 set_mon_vals no callback set 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 10 client.114248.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:49.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f01377fe700 10 monclient: dump: 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.529 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 10 monclient: _renew_subs 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0150123cd0 con 0x7f0150138270 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.502+0000 7f0155632700 1 librados: init done 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f0155632700 10 client.114248.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f0155632700 15 client.114248.objecter _session_command_op_assign -1 1 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f0155632700 20 client.114248.objecter _get_session osd=-1 returning homeless 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f0155632700 10 client.114248.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f0155632700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01500a2a50 con 0x7f0150138270 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0148014480 con 0x7f0150138270 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f013c03a2f0 0x7f013c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f014804acd0 con 0x7f0150138270 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 10 client.114248.objecter ms_dispatch 0x7f0150001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 3 client.114248.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 3 client.114248.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 10 client.114248.objecter checking command 1 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f013c041770 0x7f013c043c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 20 client.114248.objecter _get_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 15 client.114248.objecter _session_command_op_remove -1 1 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 20 client.114248.objecter _get_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 20 client.114248.objecter get_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 15 client.114248.objecter _session_command_op_assign 2 1 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 4 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 10 client.114248.objecter _send_command 1 2024-04-20T02:35:49.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 --> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f013c0442e0 con 0x7f013c041770 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 20 client.114248.objecter dump_active .. 0 homeless 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=697) v2 ==== 24+0+0 (secure 0 0 0) 0x7f014804d360 con 0x7f0150138270 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f01377fe700 10 monclient: handle_get_version_reply finishing 1 version 697 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f014e7fc700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f013c041770 0x7f013c043c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f014effd700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f013c03a2f0 0x7f013c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f014e7fc700 10 monclient: get_auth_request con 0x7f013c041770 auth_method 0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.503+0000 7f014effd700 10 monclient: get_auth_request con 0x7f013c03a2f0 auth_method 0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 10 client.114248.objecter ms_handle_connect 0x7f013c041770 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f014e7fc700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f013c041770 0x7f013c043c10 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 10 client.114248.objecter ms_handle_connect 0x7f013c03a2f0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f014effd700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f013c03a2f0 0x7f013c03c7b0 secure :-1 s=READY pgs=3417 cs=0 l=1 rev1=1 crypto rx=0x7f014000a010 tx=0x7f0140008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== osd.2 v2:172.21.15.47:6808/42311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f013c0442e0 con 0x7f013c041770 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 10 client.114248.objecter ms_dispatch 0x7f0150001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:35:49.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 10 client.114248.objecter _finish_command 1 = system:0 2024-04-20T02:35:49.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 4 2024-04-20T02:35:49.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.504+0000 7f01377fe700 15 client.114248.objecter _session_command_op_remove 2 1 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 10 client.114248.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 15 client.114248.objecter _session_command_op_assign -1 2 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 20 client.114248.objecter _get_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 20 client.114248.objecter _get_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 15 client.114248.objecter _session_command_op_remove -1 2 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 20 client.114248.objecter get_session s=0x7f013c0410a0 osd=2 3 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 15 client.114248.objecter _session_command_op_assign 2 2 2024-04-20T02:35:49.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 4 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 10 client.114248.objecter _send_command 2 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 --> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f01501386b0 con 0x7f013c041770 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f01377fe700 1 -- 172.21.15.47:0/2262202246 <== osd.2 v2:172.21.15.47:6808/42311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0144007130 con 0x7f013c041770 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f01377fe700 10 client.114248.objecter ms_dispatch 0x7f0150001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f01377fe700 10 client.114248.objecter _finish_command 2 = system:0 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f01377fe700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 4 2024-04-20T02:35:49.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.520+0000 7f01377fe700 15 client.114248.objecter _session_command_op_remove 2 2 2024-04-20T02:35:49.536 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:49.536 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T02:35:49.536 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:49.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 10 librados: watch_flush enter 2024-04-20T02:35:49.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 10 librados: watch_flush exit 2024-04-20T02:35:49.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 10 client.114248.objecter close_session for osd.2 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f013c041770 msgr2=0x7f013c043c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:6808/42311,v1:172.21.15.47:6812/42311] conn(0x7f013c041770 0x7f013c043c10 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 20 client.114248.objecter put_session s=0x7f013c0410a0 osd=2 1 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 20 client.114248.objecter shutdown clearing up homeless session... 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 10 client.114248.objecter successfully canceled tick 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f013c03a2f0 msgr2=0x7f013c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f013c03a2f0 0x7f013c03c7b0 secure :-1 s=READY pgs=3417 cs=0 l=1 rev1=1 crypto rx=0x7f014000a010 tx=0x7f0140008030 comp rx=0 tx=0).stop 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 10 monclient: shutdown 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0150138270 msgr2=0x7f0150137320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 --2- 172.21.15.47:0/2262202246 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0150138270 0x7f0150137320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f014800efc0 tx=0x7f014800c3a0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f014f7fe700 1 -- 172.21.15.47:0/2262202246 reap_dead start 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 shutdown_connections 2024-04-20T02:35:49.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 >> 172.21.15.47:0/2262202246 conn(0x7f01500aa710 msgr2=0x7f01501bf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:49.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 shutdown_connections 2024-04-20T02:35:49.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 -- 172.21.15.47:0/2262202246 wait complete. 2024-04-20T02:35:49.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.521+0000 7f0155632700 1 librados: shutdown 2024-04-20T02:35:49.540 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.676+0000 7fee010ad700 10 monclient: get_monmap_and_config 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.676+0000 7fee010ad700 10 monclient: build_initial_monmap 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.676+0000 7fee010ad700 10 monclient: monmap: 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:49.678015+0000 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:49.678015+0000 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:49.677 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:49.678 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.677+0000 7fee010ad700 1 Processor -- start 2024-04-20T02:35:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.677+0000 7fee010ad700 1 -- start start 2024-04-20T02:35:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.677+0000 7fee010ad700 10 monclient: init 2024-04-20T02:35:49.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient: _add_conns ranks=[7,8,5,10,9,4,1,3,2,0,6] 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedfc0a1510 0x7fedfc0a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): picked mon.noname-i con 0x7fedfc0a1510 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 0x7fedfc0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): picked mon.noname-e con 0x7fedfc09f9a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09fe00 0x7fedfc0a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): picked mon.noname-h con 0x7fedfc09fe00 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc0720a0 con 0x7fedfc09fe00 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc070640 con 0x7fedfc0a1510 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc071d30 con 0x7fedfc09f9a0 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:49.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fee010ad700 10 monclient(hunting): authenticate will time out at 2829.079189s 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 0x7fedfc0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 0x7fedfc0a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:48514/0 (socket says 172.21.15.47:48514) 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 -- 172.21.15.47:0/1680883216 learned_addr learned my addr 172.21.15.47:0/1680883216 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): get_auth_request con 0x7fedfc09f9a0 auth_method 0 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedfc0a1510 0x7fedfc0a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): get_auth_request con 0x7fedfc0a1510 auth_method 0 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09fe00 0x7fedfc0a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): get_auth_request con 0x7fedfc09fe00 auth_method 0 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfb59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient(hunting): handle_auth_done global_id 36431 payload 274 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 -- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedfc0a1510 msgr2=0x7fedfc0a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedfc0a1510 0x7fedfc0a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 -- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09fe00 msgr2=0x7fedfc0a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09fe00 0x7fedfc0a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient: _finish_hunting 0 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 monclient: found mon.noname-e 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.47:3304/0 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 -- 172.21.15.47:0/1680883216 --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc0751e0 con 0x7fedfc09f9a0 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.678+0000 7fedfa59c700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 0x7fedfc0a38f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fedec009920 tx=0x7fedec009c30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=994737e2c031b963 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 1 -- 172.21.15.47:0/1680883216 <== mon.8 v2:172.21.15.47:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedec011aa0 con 0x7fedfc09f9a0 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: dump: 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.694 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: _finish_auth 0 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: _check_auth_tickets 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 1 -- 172.21.15.47:0/1680883216 <== mon.8 v2:172.21.15.47:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec011c00 con 0x7fedfc09f9a0 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 5 monclient: authenticate success, global_id 36431 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:49.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 1 -- 172.21.15.47:0/1680883216 <== mon.8 v2:172.21.15.47:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedec011d70 con 0x7fedfc09f9a0 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fedf9d9b700 10 monclient: dump: 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.696 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 4 set_mon_vals no callback set 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 10 monclient: shutdown 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 -- 172.21.15.47:0/1680883216 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 msgr2=0x7fedfc0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 0x7fedfc0a38f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fedec009920 tx=0x7fedec009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 -- 172.21.15.47:0/1680883216 shutdown_connections 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09fe00 0x7fedfc0a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7fedfc0a1510 0x7fedfc0a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 --2- 172.21.15.47:0/1680883216 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7fedfc09f9a0 0x7fedfc0a38f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 -- 172.21.15.47:0/1680883216 >> 172.21.15.47:0/1680883216 conn(0x7fedfc0a8510 msgr2=0x7fedfc0a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 -- 172.21.15.47:0/1680883216 shutdown_connections 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.679+0000 7fee010ad700 1 -- 172.21.15.47:0/1680883216 wait complete. 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.680+0000 7fee010ad700 10 monclient: build_initial_monmap 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.680+0000 7fee010ad700 10 monclient: monmap: 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:49.698 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.680+0000 7fee010ad700 1 librados: starting msgr at 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.680+0000 7fee010ad700 1 librados: starting objecter 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 Processor -- start 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 -- start start 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 librados: setting wanted keys 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 librados: calling monclient init 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient: init 2024-04-20T02:35:49.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient: _add_conns ranks=[5,6,1,8,7,3,0,4,2] 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09f9a0 0x7fedfc1ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): picked mon.noname-h con 0x7fedfc09f9a0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 0x7fedfc135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): picked mon.noname-d con 0x7fedfc09fe00 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc0a1510 0x7fedfc135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): picked mon.noname-f con 0x7fedfc0a1510 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc0a2130 con 0x7fedfc0a1510 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc121b80 con 0x7fedfc09f9a0 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc071040 con 0x7fedfc09fe00 2024-04-20T02:35:49.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.681+0000 7fee010ad700 10 monclient(hunting): authenticate will time out at 2829.082957s 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 0x7fedfc135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 0x7fedfc135950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56356/0 (socket says 172.21.15.47:56356) 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 -- 172.21.15.47:0/299502162 learned_addr learned my addr 172.21.15.47:0/299502162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): get_auth_request con 0x7fedfc09fe00 auth_method 0 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc0a1510 0x7fedfc135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09f9a0 0x7fedfc1ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): get_auth_request con 0x7fedfc09f9a0 auth_method 0 2024-04-20T02:35:49.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): get_auth_request con 0x7fedfc0a1510 auth_method 0 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfb59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:49.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient(hunting): handle_auth_done global_id 55662 payload 824 2024-04-20T02:35:49.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 -- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09f9a0 msgr2=0x7fedfc1ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09f9a0 0x7fedfc1ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 -- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc0a1510 msgr2=0x7fedfc135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc0a1510 0x7fedfc135e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient: _finish_hunting 0 2024-04-20T02:35:49.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 monclient: found mon.noname-d 2024-04-20T02:35:49.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:49.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:49.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 -- 172.21.15.47:0/299502162 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009550 con 0x7fedfc09fe00 2024-04-20T02:35:49.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedfa59c700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 0x7fedfc135950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fedec013000 tx=0x7fedec004750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 client.?.objecter ms_handle_connect 0x7fedfc09fe00 2024-04-20T02:35:49.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:49.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedec0045f0 con 0x7fedfc09fe00 2024-04-20T02:35:49.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:49.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: dump: 2024-04-20T02:35:49.709 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.709 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: _finish_auth 0 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: _check_auth_tickets 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec01f070 con 0x7fedfc09fe00 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fedec003b10 con 0x7fedfc09fe00 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fede37fe700 10 monclient: dump: 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:49.711 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 5 monclient: authenticate success, global_id 55662 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fedf959a700 4 set_mon_vals no callback set 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 10 monclient: _renew_subs 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc123030 con 0x7fedfc09fe00 2024-04-20T02:35:49.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 10 client.55662.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 10 monclient: _renew_subs 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc120f40 con 0x7fedfc09fe00 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.682+0000 7fee010ad700 1 librados: init done 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.683+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fedec021030 con 0x7fedfc09fe00 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.683+0000 7fede37fe700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fede803a2f0 0x7fede803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.683+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fedec04b5f0 con 0x7fedfc09fe00 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.683+0000 7fede37fe700 10 client.55662.objecter ms_dispatch 0x7fedfc001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.683+0000 7fede37fe700 3 client.55662.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.683+0000 7fede37fe700 3 client.55662.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fee010ad700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fee010ad700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fee010ad700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc0008d0 con 0x7fedfc09fe00 2024-04-20T02:35:49.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fede37fe700 20 client.55662.objecter dump_active .. 0 homeless 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fedfad9d700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fede803a2f0 0x7fede803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fedfad9d700 10 monclient: get_auth_request con 0x7fede803a2f0 auth_method 0 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fedfad9d700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fede803a2f0 0x7fede803c7b0 secure :-1 s=READY pgs=3418 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009ac0 tx=0x7fedf0009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.684+0000 7fede37fe700 10 client.55662.objecter ms_handle_connect 0x7fede803a2f0 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.689+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fedec0197c0 con 0x7fedfc09fe00 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.689+0000 7fede37fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:49.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.689+0000 7fede37fe700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.908+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fedfc0a1e30 con 0x7fede803a2f0 2024-04-20T02:35:49.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.909+0000 7fede37fe700 1 -- 172.21.15.47:0/299502162 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fedf0004020 con 0x7fede803a2f0 2024-04-20T02:35:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.911+0000 7fee010ad700 10 librados: watch_flush enter 2024-04-20T02:35:49.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.911+0000 7fee010ad700 10 librados: watch_flush exit 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.911+0000 7fee010ad700 20 client.55662.objecter shutdown clearing up homeless session... 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.911+0000 7fee010ad700 10 client.55662.objecter successfully canceled tick 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fede803a2f0 msgr2=0x7fede803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fede803a2f0 0x7fede803c7b0 secure :-1 s=READY pgs=3418 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009ac0 tx=0x7fedf0009210 comp rx=0 tx=0).stop 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 10 monclient: shutdown 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 msgr2=0x7fedfc135950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 0x7fedfc135950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fedec013000 tx=0x7fedec004750 comp rx=0 tx=0).stop 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 shutdown_connections 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fede803a2f0 0x7fede803c7b0 unknown :-1 s=CLOSED pgs=3418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fedfc0a1510 0x7fedfc135e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7fedfc09f9a0 0x7fedfc1ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 --2- 172.21.15.47:0/299502162 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7fedfc09fe00 0x7fedfc135950 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 >> 172.21.15.47:0/299502162 conn(0x7fedfc0a8510 msgr2=0x7fedfc0a6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 shutdown_connections 2024-04-20T02:35:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 -- 172.21.15.47:0/299502162 wait complete. 2024-04-20T02:35:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:49.912+0000 7fee010ad700 1 librados: shutdown 2024-04-20T02:35:49.914 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2024-04-20T02:35:49.926 INFO:tasks.ceph:Scrubbing osd.3 2024-04-20T02:35:49.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-20T02:35:50.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.065+0000 7f735c506700 10 monclient: get_monmap_and_config 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.065+0000 7f735c506700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.065+0000 7f735c506700 10 monclient: monmap: 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:50.067088+0000 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:50.067088+0000 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.066 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:50.067 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.066+0000 7f735c506700 1 Processor -- start 2024-04-20T02:35:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.066+0000 7f735c506700 1 -- start start 2024-04-20T02:35:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.066+0000 7f735c506700 10 monclient: init 2024-04-20T02:35:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient: _add_conns ranks=[0,5,6,3,4,7,1,8,9,10,2] 2024-04-20T02:35:50.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7354137e90 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): picked mon.noname-h con 0x7f73540a3ce0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 0x7f735413f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): picked mon.noname-d con 0x7f735413d100 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73540751e0 con 0x7f7354137e90 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f735409e0e0 con 0x7f73540a3ce0 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f73540a2d30 con 0x7f735413d100 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735c506700 10 monclient(hunting): authenticate will time out at 2829.468393s 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 0x7f735413f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735a2a2700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7354137e90 0x7f7354138270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57904/0 (socket says 172.21.15.47:57904) 2024-04-20T02:35:50.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 0x7f735413f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56366/0 (socket says 172.21.15.47:56366) 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735a2a2700 1 -- 172.21.15.47:0/2580491359 learned_addr learned my addr 172.21.15.47:0/2580491359 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): get_auth_request con 0x7f735413d100 auth_method 0 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f7359aa1700 1 --2- 172.21.15.47:0/2580491359 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f7359aa1700 10 monclient(hunting): get_auth_request con 0x7f73540a3ce0 auth_method 0 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f7359aa1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f7359aa1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f7359aa1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient(hunting): handle_auth_done global_id 55671 payload 274 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 -- 172.21.15.47:0/2580491359 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 msgr2=0x7f735413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f73592a0700 1 -- 172.21.15.47:0/2580491359 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307439296 0 0) 0x7f73540751e0 con 0x7f7354137e90 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 --2- 172.21.15.47:0/2580491359 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 -- 172.21.15.47:0/2580491359 >> v1:172.21.15.47:6789/0 conn(0x7f7354137e90 legacy=0x7f7354138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 monclient: found mon.noname-d 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f735aaa3700 1 -- 172.21.15.47:0/2580491359 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540a7a50 con 0x7f735413d100 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.067+0000 7f73592a0700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735aaa3700 1 --2- 172.21.15.47:0/2580491359 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 0x7f735413f4f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f734c009df0 tx=0x7f734c00b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=88cee4616aafca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 1 -- 172.21.15.47:0/2580491359 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f734c00e070 con 0x7f735413d100 2024-04-20T02:35:50.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: dump: 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.081 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 5 monclient: authenticate success, global_id 55671 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 1 -- 172.21.15.47:0/2580491359 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c0098f0 con 0x7f735413d100 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 4 set_mon_vals no callback set 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 1 -- 172.21.15.47:0/2580491359 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f734c0128a0 con 0x7f735413d100 2024-04-20T02:35:50.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f73592a0700 10 monclient: dump: 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.083 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 10 monclient: shutdown 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 1 -- 172.21.15.47:0/2580491359 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 msgr2=0x7f735413f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 1 --2- 172.21.15.47:0/2580491359 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 0x7f735413f4f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f734c009df0 tx=0x7f734c00b330 comp rx=0 tx=0).stop 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 1 -- 172.21.15.47:0/2580491359 shutdown_connections 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 1 --2- 172.21.15.47:0/2580491359 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f735413d100 0x7f735413f4f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 1 --2- 172.21.15.47:0/2580491359 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.068+0000 7f735c506700 1 -- 172.21.15.47:0/2580491359 >> 172.21.15.47:0/2580491359 conn(0x7f73540aa710 msgr2=0x7f73540a6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.069+0000 7f735c506700 1 -- 172.21.15.47:0/2580491359 shutdown_connections 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.069+0000 7f735c506700 1 -- 172.21.15.47:0/2580491359 wait complete. 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.069+0000 7f735c506700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.069+0000 7f735c506700 10 monclient: monmap: 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.085 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 1 librados: starting msgr at 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 1 librados: starting objecter 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 1 Processor -- start 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 1 -- start start 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 1 librados: setting wanted keys 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 1 librados: calling monclient init 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.070+0000 7f735c506700 10 monclient: init 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient: _add_conns ranks=[5,0,1,2,7,8,3,4,6] 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): picked mon.noname-h con 0x7f73540a3ce0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7354137e90 0x7f73541cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): picked mon.noname-a con 0x7f7354137e90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735413d100 0x7f73541ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): picked mon.noname-f con 0x7f735413d100 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354122830 con 0x7f7354137e90 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735413bf30 con 0x7f735413d100 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73540a1df0 con 0x7f73540a3ce0 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735c506700 10 monclient(hunting): authenticate will time out at 2829.472269s 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7354137e90 0x7f73541cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7354137e90 0x7f73541cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53534/0 (socket says 172.21.15.47:53534) 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 -- 172.21.15.47:0/2539702071 learned_addr learned my addr 172.21.15.47:0/2539702071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): get_auth_request con 0x7f7354137e90 auth_method 0 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735a2a2700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735aaa3700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735413d100 0x7f73541ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735a2a2700 10 monclient(hunting): get_auth_request con 0x7f73540a3ce0 auth_method 0 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735a2a2700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735a2a2700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735a2a2700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735aaa3700 10 monclient(hunting): get_auth_request con 0x7f735413d100 auth_method 0 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735aaa3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735aaa3700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f735aaa3700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient(hunting): handle_auth_done global_id 104499 payload 824 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 -- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 msgr2=0x7f735413ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f73540a3ce0 0x7f735413ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 -- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735413d100 msgr2=0x7f73541ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f735413d100 0x7f73541ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 monclient: found mon.noname-a 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 -- 172.21.15.47:0/2539702071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c009a70 con 0x7f7354137e90 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f7359aa1700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7354137e90 0x7f73541cc7c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f734400d7d0 tx=0x7f734400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f734affd700 10 client.?.objecter ms_handle_connect 0x7f7354137e90 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.071+0000 7f734affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7344012a60 con 0x7f7354137e90 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: dump: 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.090 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344012bc0 con 0x7f7354137e90 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7344012d30 con 0x7f7354137e90 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 5 monclient: authenticate success, global_id 104499 2024-04-20T02:35:50.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 10 monclient: _renew_subs 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73540a4390 con 0x7f7354137e90 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f7358a9f700 4 set_mon_vals no callback set 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f734affd700 10 monclient: dump: 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.092 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 10 client.104499.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 10 monclient: _renew_subs 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7354136d80 con 0x7f7354137e90 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 1 librados: init done 2024-04-20T02:35:50.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.072+0000 7f735c506700 10 client.104499.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735c506700 15 client.104499.objecter _session_command_op_assign -1 1 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735c506700 20 client.104499.objecter _get_session osd=-1 returning homeless 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735c506700 10 client.104499.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735c506700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73540a4120 con 0x7f7354137e90 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f73440125f0 con 0x7f7354137e90 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f733803a320 0x7f733803c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f734404a970 con 0x7f7354137e90 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 10 client.104499.objecter ms_dispatch 0x7f7354001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 3 client.104499.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 3 client.104499.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 10 client.104499.objecter checking command 1 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f73380417a0 0x7f7338043c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 20 client.104499.objecter _get_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 15 client.104499.objecter _session_command_op_remove -1 1 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 20 client.104499.objecter _get_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 20 client.104499.objecter get_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 15 client.104499.objecter _session_command_op_assign 3 1 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 4 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 10 client.104499.objecter _send_command 1 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 --> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7338044310 con 0x7f73380417a0 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 20 client.104499.objecter dump_active .. 0 homeless 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=697) v2 ==== 24+0+0 (secure 0 0 0) 0x7f734404f2c0 con 0x7f7354137e90 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f734affd700 10 monclient: handle_get_version_reply finishing 1 version 697 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735a2a2700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f733803a320 0x7f733803c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735a2a2700 10 monclient: get_auth_request con 0x7f733803a320 auth_method 0 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735aaa3700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f73380417a0 0x7f7338043c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.073+0000 7f735aaa3700 10 monclient: get_auth_request con 0x7f73380417a0 auth_method 0 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.074+0000 7f734affd700 10 client.104499.objecter ms_handle_connect 0x7f733803a320 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.074+0000 7f735a2a2700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f733803a320 0x7f733803c7e0 secure :-1 s=READY pgs=3419 cs=0 l=1 rev1=1 crypto rx=0x7f7350007fd0 tx=0x7f7350007c20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.074+0000 7f734affd700 10 client.104499.objecter ms_handle_connect 0x7f73380417a0 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.074+0000 7f735aaa3700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f73380417a0 0x7f7338043c40 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.075+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== osd.3 v2:172.21.15.179:6800/42294 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7338044310 con 0x7f73380417a0 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.075+0000 7f734affd700 10 client.104499.objecter ms_dispatch 0x7f7354001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.075+0000 7f734affd700 10 client.104499.objecter _finish_command 1 = system:0 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.075+0000 7f734affd700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 4 2024-04-20T02:35:50.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.075+0000 7f734affd700 15 client.104499.objecter _session_command_op_remove 3 1 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 10 client.104499.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 15 client.104499.objecter _session_command_op_assign -1 2 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 20 client.104499.objecter _get_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 20 client.104499.objecter _get_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 15 client.104499.objecter _session_command_op_remove -1 2 2024-04-20T02:35:50.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 20 client.104499.objecter get_session s=0x7f73380410d0 osd=3 3 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 15 client.104499.objecter _session_command_op_assign 3 2 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 4 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 10 client.104499.objecter _send_command 2 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.090+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 --> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f73540a7560 con 0x7f73380417a0 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f734affd700 1 -- 172.21.15.47:0/2539702071 <== osd.3 v2:172.21.15.179:6800/42294 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f734c0098f0 con 0x7f73380417a0 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f734affd700 10 client.104499.objecter ms_dispatch 0x7f7354001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f734affd700 10 client.104499.objecter _finish_command 2 = system:0 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f734affd700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 4 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f734affd700 15 client.104499.objecter _session_command_op_remove 3 2 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:50.098 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T02:35:50.099 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 10 librados: watch_flush enter 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 10 librados: watch_flush exit 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 10 client.104499.objecter close_session for osd.3 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f73380417a0 msgr2=0x7f7338043c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6800/42294,v1:172.21.15.179:6801/42294] conn(0x7f73380417a0 0x7f7338043c40 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 20 client.104499.objecter put_session s=0x7f73380410d0 osd=3 1 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 20 client.104499.objecter shutdown clearing up homeless session... 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 10 client.104499.objecter successfully canceled tick 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f733803a320 msgr2=0x7f733803c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f733803a320 0x7f733803c7e0 secure :-1 s=READY pgs=3419 cs=0 l=1 rev1=1 crypto rx=0x7f7350007fd0 tx=0x7f7350007c20 comp rx=0 tx=0).stop 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 10 monclient: shutdown 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7354137e90 msgr2=0x7f73541cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 --2- 172.21.15.47:0/2539702071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7354137e90 0x7f73541cc7c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f734400d7d0 tx=0x7f734400db90 comp rx=0 tx=0).stop 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735aaa3700 1 -- 172.21.15.47:0/2539702071 reap_dead start 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 shutdown_connections 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 >> 172.21.15.47:0/2539702071 conn(0x7f73540aa710 msgr2=0x7f735409c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 shutdown_connections 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 -- 172.21.15.47:0/2539702071 wait complete. 2024-04-20T02:35:50.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.091+0000 7f735c506700 1 librados: shutdown 2024-04-20T02:35:50.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-20T02:35:50.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.240+0000 7fb44c6d7700 10 monclient: get_monmap_and_config 2024-04-20T02:35:50.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.240+0000 7fb44c6d7700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.240+0000 7fb44c6d7700 10 monclient: monmap: 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:50.241929+0000 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:50.241929+0000 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.242 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.243 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.244 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.244 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.244 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:50.244 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:50.244 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.241+0000 7fb44c6d7700 1 Processor -- start 2024-04-20T02:35:50.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.241+0000 7fb44c6d7700 1 -- start start 2024-04-20T02:35:50.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.241+0000 7fb44c6d7700 10 monclient: init 2024-04-20T02:35:50.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient: _add_conns ranks=[1,9,2,4,8,10,3,6,0,7,5] 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb4440a50f0 addr v1:172.21.15.179:6789/0 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4441378a0 0x7fb444138440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): picked mon.noname-a con 0x7fb4441378a0 addr v2:172.21.15.47:3300/0 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 0x7fb44413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): picked mon.noname-b con 0x7fb44413cbc0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4440751e0 con 0x7fb4440a50f0 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440a54d0 con 0x7fb4441378a0 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44406da20 con 0x7fb44413cbc0 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44c6d7700 10 monclient(hunting): authenticate will time out at 2829.643285s 2024-04-20T02:35:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 0x7fb44413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb4441378a0 0x7fb444138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 0x7fb44413efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60260/0 (socket says 172.21.15.47:60260) 2024-04-20T02:35:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 1 -- 172.21.15.47:0/31126686 learned_addr learned my addr 172.21.15.47:0/31126686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): get_auth_request con 0x7fb44413cbc0 auth_method 0 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): get_auth_request con 0x7fb4441378a0 auth_method 0 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient(hunting): handle_auth_done global_id 104508 payload 274 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 -- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 msgr2=0x7fb44413efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 --2- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 0x7fb44413efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 -- 172.21.15.47:0/31126686 >> v1:172.21.15.179:6789/0 conn(0x7fb4440a50f0 legacy=0x7fb444137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 monclient: found mon.noname-a 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 -- 172.21.15.47:0/31126686 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44406e090 con 0x7fb4441378a0 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb44ac74700 1 --2- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 0x7fb44413efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.242+0000 7fb449c72700 1 --2- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4441378a0 0x7fb444138440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb434009840 tx=0x7fb434009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7df42c445cc1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 1 -- 172.21.15.47:0/31126686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb434011aa0 con 0x7fb4441378a0 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: dump: 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.265 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.267 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 1 -- 172.21.15.47:0/31126686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb434011c00 con 0x7fb4441378a0 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 5 monclient: authenticate success, global_id 104508 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:50.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 1 -- 172.21.15.47:0/31126686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb434011d70 con 0x7fb4441378a0 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 4 set_mon_vals no callback set 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb449471700 10 monclient: dump: 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.269 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 10 monclient: shutdown 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 -- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4441378a0 msgr2=0x7fb444138440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4441378a0 0x7fb444138440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb434009840 tx=0x7fb434009b50 comp rx=0 tx=0).stop 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 -- 172.21.15.47:0/31126686 shutdown_connections 2024-04-20T02:35:50.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb44413cbc0 0x7fb44413efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/31126686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4441378a0 0x7fb444138440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 -- 172.21.15.47:0/31126686 >> 172.21.15.47:0/31126686 conn(0x7fb4440a8510 msgr2=0x7fb4440a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 -- 172.21.15.47:0/31126686 shutdown_connections 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.243+0000 7fb44c6d7700 1 -- 172.21.15.47:0/31126686 wait complete. 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.244+0000 7fb44c6d7700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.244+0000 7fb44c6d7700 10 monclient: monmap: 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:50.271 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.244+0000 7fb44c6d7700 1 librados: starting msgr at 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.244+0000 7fb44c6d7700 1 librados: starting objecter 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.244+0000 7fb44c6d7700 1 Processor -- start 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 -- start start 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 librados: setting wanted keys 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 librados: calling monclient init 2024-04-20T02:35:50.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient: init 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient: _add_conns ranks=[0,4,1,8,5,3,6,7,2] 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 0x7fb444135960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): picked mon.noname-a con 0x7fb4440a50f0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb44413cbc0 0x7fb444135ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): picked mon.noname-c con 0x7fb44413cbc0 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4441cc3b0 0x7fb44411e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): picked mon.noname-f con 0x7fb4441cc3b0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444121e50 con 0x7fb4440a50f0 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441cd060 con 0x7fb4441cc3b0 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb444072260 con 0x7fb44413cbc0 2024-04-20T02:35:50.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44c6d7700 10 monclient(hunting): authenticate will time out at 2829.646618s 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 0x7fb444135960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb44413cbc0 0x7fb444135ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 0x7fb444135960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53562/0 (socket says 172.21.15.47:53562) 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 1 -- 172.21.15.47:0/1725386577 learned_addr learned my addr 172.21.15.47:0/1725386577 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): get_auth_request con 0x7fb4440a50f0 auth_method 0 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): get_auth_request con 0x7fb44413cbc0 auth_method 0 2024-04-20T02:35:50.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44a473700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4441cc3b0 0x7fb44411e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb449c72700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): get_auth_request con 0x7fb4441cc3b0 auth_method 0 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.245+0000 7fb44ac74700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 10 monclient(hunting): handle_auth_done global_id 104517 payload 824 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 -- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb44413cbc0 msgr2=0x7fb444135ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb44413cbc0 0x7fb444135ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 -- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4441cc3b0 msgr2=0x7fb44411e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4441cc3b0 0x7fb44411e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 monclient: found mon.noname-a 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 -- 172.21.15.47:0/1725386577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c009b30 con 0x7fb4440a50f0 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44a473700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 0x7fb444135960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c450 tx=0x7fb44000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 client.?.objecter ms_handle_connect 0x7fb4440a50f0 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4400011a0 con 0x7fb4440a50f0 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: dump: 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.277 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44000ce60 con 0x7fb4440a50f0 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 5 monclient: authenticate success, global_id 104517 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb440005540 con 0x7fb4440a50f0 2024-04-20T02:35:50.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 10 monclient: _renew_subs 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4340094a0 con 0x7fb4440a50f0 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb448c70700 4 set_mon_vals no callback set 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 10 client.104517.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb43affd700 10 monclient: dump: 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.279 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 10 monclient: _renew_subs 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb444122310 con 0x7fb4440a50f0 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.246+0000 7fb44c6d7700 1 librados: init done 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb4400050d0 con 0x7fb4440a50f0 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb44c6d7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb44c6d7700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb44c6d7700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb42803a2f0 0x7fb42803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44411ff60 con 0x7fb4440a50f0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fb4400492d0 con 0x7fb4440a50f0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 10 client.104517.objecter ms_dispatch 0x7fb444001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 3 client.104517.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 3 client.104517.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb449c72700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb42803a2f0 0x7fb42803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb449c72700 10 monclient: get_auth_request con 0x7fb42803a2f0 auth_method 0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.247+0000 7fb43affd700 20 client.104517.objecter dump_active .. 0 homeless 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.248+0000 7fb43affd700 10 client.104517.objecter ms_handle_connect 0x7fb42803a2f0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.248+0000 7fb449c72700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb42803a2f0 0x7fb42803c7b0 secure :-1 s=READY pgs=3420 cs=0 l=1 rev1=1 crypto rx=0x7fb444067e30 tx=0x7fb434008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.252+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fb440018680 con 0x7fb4440a50f0 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.252+0000 7fb43affd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.252+0000 7fb43affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.472+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb44406e650 con 0x7fb42803a2f0 2024-04-20T02:35:50.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.472+0000 7fb43affd700 1 -- 172.21.15.47:0/1725386577 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb434004020 con 0x7fb42803a2f0 2024-04-20T02:35:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.475+0000 7fb44c6d7700 10 librados: watch_flush enter 2024-04-20T02:35:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.475+0000 7fb44c6d7700 10 librados: watch_flush exit 2024-04-20T02:35:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.475+0000 7fb44c6d7700 20 client.104517.objecter shutdown clearing up homeless session... 2024-04-20T02:35:50.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.475+0000 7fb44c6d7700 10 client.104517.objecter successfully canceled tick 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb42803a2f0 msgr2=0x7fb42803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb42803a2f0 0x7fb42803c7b0 secure :-1 s=READY pgs=3420 cs=0 l=1 rev1=1 crypto rx=0x7fb444067e30 tx=0x7fb434008030 comp rx=0 tx=0).stop 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 10 monclient: shutdown 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 msgr2=0x7fb444135960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 0x7fb444135960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c450 tx=0x7fb44000c760 comp rx=0 tx=0).stop 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 shutdown_connections 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7fb42803a2f0 0x7fb42803c7b0 unknown :-1 s=CLOSED pgs=3420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4441cc3b0 0x7fb44411e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4440a50f0 0x7fb444135960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 --2- 172.21.15.47:0/1725386577 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb44413cbc0 0x7fb444135ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 >> 172.21.15.47:0/1725386577 conn(0x7fb4440a8510 msgr2=0x7fb44413c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 shutdown_connections 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 -- 172.21.15.47:0/1725386577 wait complete. 2024-04-20T02:35:50.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.476+0000 7fb44c6d7700 1 librados: shutdown 2024-04-20T02:35:50.479 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2024-04-20T02:35:50.491 INFO:tasks.ceph:Scrubbing osd.4 2024-04-20T02:35:50.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-20T02:35:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.630+0000 7f0f3fcb2700 10 monclient: get_monmap_and_config 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.630+0000 7f0f3fcb2700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.630+0000 7f0f3fcb2700 10 monclient: monmap: 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:50.632068+0000 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:50.632068+0000 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.631 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:50.632 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.631+0000 7f0f3fcb2700 1 Processor -- start 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.631+0000 7f0f3fcb2700 1 -- start start 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.631+0000 7f0f3fcb2700 10 monclient: init 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient: _add_conns ranks=[0,6,7,5,3,9,8,4,10,2,1] 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0f38134a50 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 0x7f0f38139d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): picked mon.noname-d con 0x7f0f381379a0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f3813a2d0 0x7f0f3813c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): picked mon.noname-i con 0x7f0f3813a2d0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f380751e0 con 0x7f0f38134a50 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38134690 con 0x7f0f381379a0 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3809f120 con 0x7f0f3813a2d0 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3fcb2700 10 monclient(hunting): authenticate will time out at 2830.033358s 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 0x7f0f38139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 0x7f0f38139d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56380/0 (socket says 172.21.15.47:56380) 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 -- 172.21.15.47:0/890489801 learned_addr learned my addr 172.21.15.47:0/890489801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): get_auth_request con 0x7f0f381379a0 auth_method 0 2024-04-20T02:35:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3da4e700 1 --1- 172.21.15.47:0/890489801 >> v1:172.21.15.47:6789/0 conn(0x7f0f38134a50 0x7f0f38134e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57916/0 (socket says 172.21.15.47:57916) 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3e24f700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f3813a2d0 0x7f0f3813c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3e24f700 10 monclient(hunting): get_auth_request con 0x7f0f3813a2d0 auth_method 0 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3e24f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3e24f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3e24f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_done global_id 55680 payload 274 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 -- 172.21.15.47:0/890489801 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f3813a2d0 msgr2=0x7f0f3813c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f3813a2d0 0x7f0f3813c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 -- 172.21.15.47:0/890489801 >> v1:172.21.15.47:6789/0 conn(0x7f0f38134a50 legacy=0x7f0f38134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 monclient: found mon.noname-d 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3ca4c700 1 -- 172.21.15.47:0/890489801 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588048051 0 0) 0x7f0f380751e0 con 0x7f0f38134a50 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3d24d700 1 -- 172.21.15.47:0/890489801 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3809ff40 con 0x7f0f381379a0 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3ca4c700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.632+0000 7f0f3e24f700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f3813a2d0 0x7f0f3813c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3d24d700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 0x7f0f38139d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009920 tx=0x7f0f28009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ae07fa7c637edd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 1 -- 172.21.15.47:0/890489801 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f28009e70 con 0x7f0f381379a0 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: dump: 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 1 -- 172.21.15.47:0/890489801 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2800bcd0 con 0x7f0f381379a0 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 5 monclient: authenticate success, global_id 55680 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 4 set_mon_vals no callback set 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 1 -- 172.21.15.47:0/890489801 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f2800be40 con 0x7f0f381379a0 2024-04-20T02:35:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3ca4c700 10 monclient: dump: 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 10 monclient: shutdown 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/890489801 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 msgr2=0x7f0f38139d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 0x7f0f38139d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009920 tx=0x7f0f28009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/890489801 shutdown_connections 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7f0f3813a2d0 0x7f0f3813c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 --2- 172.21.15.47:0/890489801 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f0f381379a0 0x7f0f38139d90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/890489801 >> 172.21.15.47:0/890489801 conn(0x7f0f380a3f90 msgr2=0x7f0f3806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/890489801 shutdown_connections 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.633+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/890489801 wait complete. 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 10 monclient: monmap: 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 1 librados: starting msgr at 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 1 librados: starting objecter 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 1 Processor -- start 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 1 -- start start 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 1 librados: setting wanted keys 2024-04-20T02:35:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 1 librados: calling monclient init 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.634+0000 7f0f3fcb2700 10 monclient: init 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient: _add_conns ranks=[0,8,5,3,6,7,1,2,4] 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f38134a50 0x7f0f3809a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): picked mon.noname-a con 0x7f0f38134a50 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f381379a0 0x7f0f3809ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): picked mon.noname-e con 0x7f0f381379a0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f3813a2d0 0x7f0f3809b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): picked mon.noname-h con 0x7f0f3813a2d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38121df0 con 0x7f0f38134a50 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38072460 con 0x7f0f3813a2d0 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3809f350 con 0x7f0f381379a0 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3fcb2700 10 monclient(hunting): authenticate will time out at 2830.036477s 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f38134a50 0x7f0f3809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f381379a0 0x7f0f3809ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f38134a50 0x7f0f3809a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53566/0 (socket says 172.21.15.47:53566) 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 -- 172.21.15.47:0/1565902246 learned_addr learned my addr 172.21.15.47:0/1565902246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): get_auth_request con 0x7f0f38134a50 auth_method 0 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): get_auth_request con 0x7f0f381379a0 auth_method 0 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f3813a2d0 0x7f0f3809b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3d24d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): get_auth_request con 0x7f0f3813a2d0 auth_method 0 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3e24f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 10 monclient(hunting): handle_auth_done global_id 104535 payload 824 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 -- 172.21.15.47:0/1565902246 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f381379a0 msgr2=0x7f0f3809ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f381379a0 0x7f0f3809ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 -- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f3813a2d0 msgr2=0x7f0f3809b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.635+0000 7f0f3da4e700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f0f3813a2d0 0x7f0f3809b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3da4e700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3da4e700 1 monclient: found mon.noname-a 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3da4e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3da4e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3da4e700 1 -- 172.21.15.47:0/1565902246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28009550 con 0x7f0f38134a50 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3d24d700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f0f381379a0 0x7f0f3809ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3da4e700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f38134a50 0x7f0f3809a930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c520 tx=0x7f0f3400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 client.?.objecter ms_handle_connect 0x7f0f38134a50 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f34004850 con 0x7f0f38134a50 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: dump: 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.659 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 5 monclient: authenticate success, global_id 104535 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f340049f0 con 0x7f0f38134a50 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f34011360 con 0x7f0f38134a50 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 10 monclient: _renew_subs 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3809c420 con 0x7f0f38134a50 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2ffff700 4 set_mon_vals no callback set 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 10 client.104535.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 10 monclient: _renew_subs 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f38122210 con 0x7f0f38134a50 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f3fcb2700 1 librados: init done 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.636+0000 7f0f2e7fc700 10 monclient: dump: 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.662 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3fcb2700 10 client.104535.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3fcb2700 15 client.104535.objecter _session_command_op_assign -1 1 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3fcb2700 20 client.104535.objecter _get_session osd=-1 returning homeless 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3fcb2700 10 client.104535.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3fcb2700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f38135540 con 0x7f0f38134a50 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f0f34004b60 con 0x7f0f38134a50 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f1c03a2f0 0x7f0f1c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f0f34049410 con 0x7f0f38134a50 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 10 client.104535.objecter ms_dispatch 0x7f0f38001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 3 client.104535.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 3 client.104535.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 10 client.104535.objecter checking command 1 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f0f1c041770 0x7f0f1c043c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 20 client.104535.objecter _get_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 15 client.104535.objecter _session_command_op_remove -1 1 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 20 client.104535.objecter _get_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 20 client.104535.objecter get_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 15 client.104535.objecter _session_command_op_assign 4 1 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 4 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 10 client.104535.objecter _send_command 1 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 --> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f1c0442e0 con 0x7f0f1c041770 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 20 client.104535.objecter dump_active .. 0 homeless 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=697) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f3404baa0 con 0x7f0f38134a50 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f2e7fc700 10 monclient: handle_get_version_reply finishing 1 version 697 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3d24d700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f1c03a2f0 0x7f0f1c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.637+0000 7f0f3d24d700 10 monclient: get_auth_request con 0x7f0f1c03a2f0 auth_method 0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.638+0000 7f0f3e24f700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f0f1c041770 0x7f0f1c043c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.638+0000 7f0f3e24f700 10 monclient: get_auth_request con 0x7f0f1c041770 auth_method 0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.638+0000 7f0f3d24d700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f1c03a2f0 0x7f0f1c03c7b0 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009920 tx=0x7f0f28008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.638+0000 7f0f2e7fc700 10 client.104535.objecter ms_handle_connect 0x7f0f1c03a2f0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.638+0000 7f0f2e7fc700 10 client.104535.objecter ms_handle_connect 0x7f0f1c041770 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.638+0000 7f0f3e24f700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f0f1c041770 0x7f0f1c043c10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.639+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== osd.4 v2:172.21.15.179:6810/42295 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0f1c0442e0 con 0x7f0f1c041770 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.639+0000 7f0f2e7fc700 10 client.104535.objecter ms_dispatch 0x7f0f38001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.639+0000 7f0f2e7fc700 10 client.104535.objecter _finish_command 1 = system:0 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.639+0000 7f0f2e7fc700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 4 2024-04-20T02:35:50.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.639+0000 7f0f2e7fc700 15 client.104535.objecter _session_command_op_remove 4 1 2024-04-20T02:35:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 client.104535.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-04-20T02:35:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 15 client.104535.objecter _session_command_op_assign -1 2 2024-04-20T02:35:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter _get_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter _get_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 15 client.104535.objecter _session_command_op_remove -1 2 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter get_session s=0x7f0f1c0410a0 osd=4 3 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 15 client.104535.objecter _session_command_op_assign 4 2 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 4 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 client.104535.objecter _send_command 2 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 --> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0f3809ec20 con 0x7f0f1c041770 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f2e7fc700 1 -- 172.21.15.47:0/1565902246 <== osd.4 v2:172.21.15.179:6810/42295 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0f3000f040 con 0x7f0f1c041770 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f2e7fc700 10 client.104535.objecter ms_dispatch 0x7f0f38001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f2e7fc700 10 client.104535.objecter _finish_command 2 = system:0 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f2e7fc700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 4 2024-04-20T02:35:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f2e7fc700 15 client.104535.objecter _session_command_op_remove 4 2 2024-04-20T02:35:50.668 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:50.668 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T02:35:50.668 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 librados: watch_flush enter 2024-04-20T02:35:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 librados: watch_flush exit 2024-04-20T02:35:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 client.104535.objecter close_session for osd.4 2024-04-20T02:35:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f0f1c041770 msgr2=0x7f0f1c043c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6810/42295,v1:172.21.15.179:6812/42295] conn(0x7f0f1c041770 0x7f0f1c043c10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter put_session s=0x7f0f1c0410a0 osd=4 1 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 client.104535.objecter shutdown clearing up homeless session... 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 client.104535.objecter successfully canceled tick 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f1c03a2f0 msgr2=0x7f0f1c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f0f1c03a2f0 0x7f0f1c03c7b0 secure :-1 s=READY pgs=3421 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009920 tx=0x7f0f28008030 comp rx=0 tx=0).stop 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 10 monclient: shutdown 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f38134a50 msgr2=0x7f0f3809a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 --2- 172.21.15.47:0/1565902246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f38134a50 0x7f0f3809a930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c520 tx=0x7f0f3400c830 comp rx=0 tx=0).stop 2024-04-20T02:35:50.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3e24f700 1 -- 172.21.15.47:0/1565902246 reap_dead start 2024-04-20T02:35:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 shutdown_connections 2024-04-20T02:35:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 >> 172.21.15.47:0/1565902246 conn(0x7f0f380a3f90 msgr2=0x7f0f380a2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 shutdown_connections 2024-04-20T02:35:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.655+0000 7f0f3fcb2700 1 -- 172.21.15.47:0/1565902246 wait complete. 2024-04-20T02:35:50.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.656+0000 7f0f3fcb2700 1 librados: shutdown 2024-04-20T02:35:50.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-20T02:35:50.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.809+0000 7f7bacd3d700 10 monclient: get_monmap_and_config 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.809+0000 7f7bacd3d700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.809+0000 7f7bacd3d700 10 monclient: monmap: 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:50.810860+0000 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:50.810860+0000 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.810 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:50.811 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:50.811 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.810+0000 7f7bacd3d700 1 Processor -- start 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.810+0000 7f7bacd3d700 1 -- start start 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.810+0000 7f7bacd3d700 10 monclient: init 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient: _add_conns ranks=[3,6,4,0,10,8,9,1,2,7,5] 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ba80a07f0 0x7f7ba80a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): picked mon.noname-g con 0x7f7ba80a07f0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 0x7f7ba80a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): picked mon.noname-d con 0x7f7ba80a0bd0 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba80a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): picked mon.noname-c con 0x7f7ba80a1050 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8067800 con 0x7f7ba80a07f0 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80672f0 con 0x7f7ba80a1050 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba809ed90 con 0x7f7ba80a0bd0 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7bacd3d700 10 monclient(hunting): authenticate will time out at 2830.212214s 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba80a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 0x7f7ba80a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 0x7f7ba80a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56394/0 (socket says 172.21.15.47:56394) 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba80a2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54836/0 (socket says 172.21.15.47:54836) 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 -- 172.21.15.47:0/1327770669 learned_addr learned my addr 172.21.15.47:0/1327770669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): get_auth_request con 0x7f7ba80a1050 auth_method 0 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): get_auth_request con 0x7f7ba80a0bd0 auth_method 0 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba659c700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ba80a07f0 0x7f7ba80a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba659c700 10 monclient(hunting): get_auth_request con 0x7f7ba80a07f0 auth_method 0 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_done global_id 55689 payload 274 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 -- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 msgr2=0x7f7ba80a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba80a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 -- 172.21.15.47:0/1327770669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ba80a07f0 msgr2=0x7f7ba80a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ba80a07f0 0x7f7ba80a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 monclient: found mon.noname-d 2024-04-20T02:35:50.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 -- 172.21.15.47:0/1327770669 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80751e0 con 0x7f7ba80a0bd0 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba6d9d700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba80a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.811+0000 7f7ba5d9b700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 0x7f7ba80a2870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009920 tx=0x7f7b90009c30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7939bdd096d4545c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 1 -- 172.21.15.47:0/1327770669 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b90011aa0 con 0x7f7ba80a0bd0 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: dump: 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.826 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 5 monclient: authenticate success, global_id 55689 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 1 -- 172.21.15.47:0/1327770669 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90011c00 con 0x7f7ba80a0bd0 2024-04-20T02:35:50.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 4 set_mon_vals no callback set 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 1 -- 172.21.15.47:0/1327770669 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b90011d70 con 0x7f7ba80a0bd0 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7ba559a700 10 monclient: dump: 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.828 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 10 monclient: shutdown 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 -- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 msgr2=0x7f7ba80a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 0x7f7ba80a2870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009920 tx=0x7f7b90009c30 comp rx=0 tx=0).stop 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 -- 172.21.15.47:0/1327770669 shutdown_connections 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba80a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ba80a07f0 0x7f7ba80a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/1327770669 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f7ba80a0bd0 0x7f7ba80a2870 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 -- 172.21.15.47:0/1327770669 >> 172.21.15.47:0/1327770669 conn(0x7f7ba80a3f90 msgr2=0x7f7ba806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 -- 172.21.15.47:0/1327770669 shutdown_connections 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.812+0000 7f7bacd3d700 1 -- 172.21.15.47:0/1327770669 wait complete. 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.813+0000 7f7bacd3d700 10 monclient: build_initial_monmap 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.813+0000 7f7bacd3d700 10 monclient: monmap: 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:50.830 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.813+0000 7f7bacd3d700 1 librados: starting msgr at 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.813+0000 7f7bacd3d700 1 librados: starting objecter 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 Processor -- start 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 -- start start 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 librados: setting wanted keys 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 librados: calling monclient init 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient: init 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:50.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient: _add_conns ranks=[2,0,4,7,1,6,3,8,5] 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 0x7f7ba81cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): picked mon.noname-b con 0x7f7ba80a07f0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80a0bd0 0x7f7ba81bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): picked mon.noname-a con 0x7f7ba80a0bd0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba81bd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): picked mon.noname-c con 0x7f7ba80a1050 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81aa420 con 0x7f7ba80a0bd0 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81bd680 con 0x7f7ba80a07f0 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba809e360 con 0x7f7ba80a1050 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7bacd3d700 10 monclient(hunting): authenticate will time out at 2830.215644s 2024-04-20T02:35:50.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba81bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 0x7f7ba81cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba81bd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:54852/0 (socket says 172.21.15.47:54852) 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 1 -- 172.21.15.47:0/3424821089 learned_addr learned my addr 172.21.15.47:0/3424821089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): get_auth_request con 0x7f7ba80a1050 auth_method 0 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 0x7f7ba81cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60274/0 (socket says 172.21.15.47:60274) 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80a0bd0 0x7f7ba81bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): get_auth_request con 0x7f7ba80a07f0 auth_method 0 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): get_auth_request con 0x7f7ba80a0bd0 auth_method 0 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:50.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:50.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.814+0000 7f7ba5d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 10 monclient(hunting): handle_auth_done global_id 84874 payload 824 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 -- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 msgr2=0x7f7ba81cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 0x7f7ba81cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 -- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80a0bd0 msgr2=0x7f7ba81bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80a0bd0 0x7f7ba81bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 10 monclient: _finish_hunting 0 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 monclient: found mon.noname-c 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 -- 172.21.15.47:0/3424821089 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0009a70 con 0x7f7ba80a1050 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba659c700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 0x7f7ba81cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba6d9d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba81bd140 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000efc0 tx=0x7f7ba000c3a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 client.?.objecter ms_handle_connect 0x7f7ba80a1050 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:50.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ba0014480 con 0x7f7ba80a1050 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: dump: 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.836 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: _finish_auth 0 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: _check_auth_tickets 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 5 monclient: authenticate success, global_id 84874 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba000f040 con 0x7f7ba80a1050 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7ba0014a90 con 0x7f7ba80a1050 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 10 monclient: _renew_subs 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b90009550 con 0x7f7ba80a1050 2024-04-20T02:35:50.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7ba4d99700 4 set_mon_vals no callback set 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 10 client.84874.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7b9effd700 10 monclient: dump: 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:50.838 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 10 monclient: _renew_subs 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba81a7fc0 con 0x7f7ba80a1050 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.815+0000 7f7bacd3d700 1 librados: init done 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mon.4 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7ba0014bf0 con 0x7f7ba80a1050 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7bacd3d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7bacd3d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7bacd3d700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.47:3302/0 2024-04-20T02:35:50.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba81aa280 con 0x7f7ba80a1050 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7b8803a2f0 0x7f7b8803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mon.4 v2:172.21.15.47:3302/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f7ba004a800 con 0x7f7ba80a1050 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 10 client.84874.objecter ms_dispatch 0x7f7ba8001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 3 client.84874.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 3 client.84874.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7b9effd700 20 client.84874.objecter dump_active .. 0 homeless 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.816+0000 7f7ba659c700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7b8803a2f0 0x7f7b8803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.817+0000 7f7ba659c700 10 monclient: get_auth_request con 0x7f7b8803a2f0 auth_method 0 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.817+0000 7f7b9effd700 10 client.84874.objecter ms_handle_connect 0x7f7b8803a2f0 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.817+0000 7f7ba659c700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7b8803a2f0 0x7f7b8803c7b0 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f7b98006b20 tx=0x7f7b98008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.821+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mon.4 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7ba81aa280 con 0x7f7ba80a1050 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.821+0000 7f7b9effd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:50.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:50.821+0000 7f7b9effd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:51.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.037+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ba809f350 con 0x7f7b8803a2f0 2024-04-20T02:35:51.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.038+0000 7f7b9effd700 1 -- 172.21.15.47:0/3424821089 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7b98011ae0 con 0x7f7b8803a2f0 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 10 librados: watch_flush enter 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 10 librados: watch_flush exit 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 20 client.84874.objecter shutdown clearing up homeless session... 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 10 client.84874.objecter successfully canceled tick 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7b8803a2f0 msgr2=0x7f7b8803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7b8803a2f0 0x7f7b8803c7b0 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 crypto rx=0x7f7b98006b20 tx=0x7f7b98008040 comp rx=0 tx=0).stop 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 10 monclient: shutdown 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 msgr2=0x7f7ba81bd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba81bd140 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000efc0 tx=0x7f7ba000c3a0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 shutdown_connections 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f7b8803a2f0 0x7f7b8803c7b0 unknown :-1 s=CLOSED pgs=3422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ba80a1050 0x7f7ba81bd140 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ba80a07f0 0x7f7ba81cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 --2- 172.21.15.47:0/3424821089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ba80a0bd0 0x7f7ba81bcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 >> 172.21.15.47:0/3424821089 conn(0x7f7ba80a3f90 msgr2=0x7f7ba80a59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 shutdown_connections 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 -- 172.21.15.47:0/3424821089 wait complete. 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.040+0000 7f7bacd3d700 1 librados: shutdown 2024-04-20T02:35:51.042 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 4 to deep-scrub 2024-04-20T02:35:51.054 INFO:tasks.ceph:Scrubbing osd.5 2024-04-20T02:35:51.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.194+0000 7f2c561a5700 10 monclient: get_monmap_and_config 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.194+0000 7f2c561a5700 10 monclient: build_initial_monmap 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.194+0000 7f2c561a5700 10 monclient: monmap: 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:51.195282+0000 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:51.195282+0000 2024-04-20T02:35:51.194 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:51.195 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.194+0000 7f2c561a5700 1 Processor -- start 2024-04-20T02:35:51.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 1 -- start start 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient: init 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient: _add_conns ranks=[0,10,6,7,3,4,2,8,9,5,1] 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2c50137e90 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c500a3ce0 0x7f2c5013cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): picked mon.noname-f con 0x7f2c500a3ce0 addr v2:172.21.15.179:3300/0 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2c5013d100 0x7f2c5013f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): picked mon.noname-d con 0x7f2c5013d100 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500751e0 con 0x7f2c50137e90 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5009e0e0 con 0x7f2c500a3ce0 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500a2d30 con 0x7f2c5013d100 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:51.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c561a5700 10 monclient(hunting): authenticate will time out at 2830.596636s 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4f7fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2c50137e90 0x7f2c50138270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57922/0 (socket says 172.21.15.47:57922) 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4f7fe700 1 -- 172.21.15.47:0/1238760397 learned_addr learned my addr 172.21.15.47:0/1238760397 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/1238760397 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2c5013d100 0x7f2c5013f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): get_auth_request con 0x7f2c5013d100 auth_method 0 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4effd700 1 --2- 172.21.15.47:0/1238760397 >> v2:172.21.15.179:3300/0 conn(0x7f2c500a3ce0 0x7f2c5013cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4effd700 10 monclient(hunting): get_auth_request con 0x7f2c500a3ce0 auth_method 0 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.195+0000 7f2c4effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 1 -- 172.21.15.47:0/1238760397 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852308749 0 0) 0x7f2c500751e0 con 0x7f2c50137e90 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient(hunting): _negotiate 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient(hunting): my global_id is 104553 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 1 -- 172.21.15.47:0/1238760397 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c3c0035b0 con 0x7f2c50137e90 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_done global_id 55698 payload 274 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 1 -- 172.21.15.47:0/1238760397 >> v2:172.21.15.179:3300/0 conn(0x7f2c500a3ce0 msgr2=0x7f2c5013cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/1238760397 >> v2:172.21.15.179:3300/0 conn(0x7f2c500a3ce0 0x7f2c5013cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 1 -- 172.21.15.47:0/1238760397 >> v1:172.21.15.47:6789/0 conn(0x7f2c50137e90 legacy=0x7f2c50138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 10 monclient: _finish_hunting 0 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 1 monclient: found mon.noname-d 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 1 -- 172.21.15.47:0/1238760397 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500a7a50 con 0x7f2c5013d100 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/1238760397 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2c5013d100 0x7f2c5013f4f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009d90 tx=0x7f2c4800b330 comp rx=0 tx=0).ready entity=mon.6 client_cookie=13bc30f001714b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 1 -- 172.21.15.47:0/1238760397 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c4800e070 con 0x7f2c5013d100 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: dump: 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.209 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:51.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c561a5700 5 monclient: authenticate success, global_id 55698 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 1 -- 172.21.15.47:0/1238760397 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c480098f0 con 0x7f2c5013d100 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c561a5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c561a5700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c561a5700 4 set_mon_vals no callback set 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 1 -- 172.21.15.47:0/1238760397 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c480128a0 con 0x7f2c5013d100 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.196+0000 7f2c4e7fc700 10 monclient: dump: 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 10 monclient: shutdown 2024-04-20T02:35:51.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 -- 172.21.15.47:0/1238760397 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2c5013d100 msgr2=0x7f2c5013f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 --2- 172.21.15.47:0/1238760397 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2c5013d100 0x7f2c5013f4f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009d90 tx=0x7f2c4800b330 comp rx=0 tx=0).stop 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 -- 172.21.15.47:0/1238760397 shutdown_connections 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 --2- 172.21.15.47:0/1238760397 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f2c5013d100 0x7f2c5013f4f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 --2- 172.21.15.47:0/1238760397 >> v2:172.21.15.179:3300/0 conn(0x7f2c500a3ce0 0x7f2c5013cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 -- 172.21.15.47:0/1238760397 >> 172.21.15.47:0/1238760397 conn(0x7f2c500aa710 msgr2=0x7f2c500a6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 -- 172.21.15.47:0/1238760397 shutdown_connections 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.197+0000 7f2c561a5700 1 -- 172.21.15.47:0/1238760397 wait complete. 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 10 monclient: build_initial_monmap 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 10 monclient: monmap: 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:51.213 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 1 librados: starting msgr at 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 1 librados: starting objecter 2024-04-20T02:35:51.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 1 Processor -- start 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 1 -- start start 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 1 librados: setting wanted keys 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 1 librados: calling monclient init 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.198+0000 7f2c561a5700 10 monclient: init 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient: _add_conns ranks=[2,0,4,6,8,7,1,3,5] 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c500a3ce0 0x7f2c50120730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): picked mon.noname-b con 0x7f2c500a3ce0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c50137e90 0x7f2c50120c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): picked mon.noname-a con 0x7f2c50137e90 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c5013d100 0x7f2c501211b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): picked mon.noname-c con 0x7f2c5013d100 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500a4390 con 0x7f2c50137e90 2024-04-20T02:35:51.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500a1df0 con 0x7f2c500a3ce0 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c501227c0 con 0x7f2c5013d100 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c561a5700 10 monclient(hunting): authenticate will time out at 2830.600694s 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c500a3ce0 0x7f2c50120730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c500a3ce0 0x7f2c50120730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60276/0 (socket says 172.21.15.47:60276) 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 1 -- 172.21.15.47:0/733535426 learned_addr learned my addr 172.21.15.47:0/733535426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c50137e90 0x7f2c50120c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): get_auth_request con 0x7f2c500a3ce0 auth_method 0 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): get_auth_request con 0x7f2c50137e90 auth_method 0 2024-04-20T02:35:51.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c5013d100 0x7f2c501211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4ffff700 10 monclient(hunting): get_auth_request con 0x7f2c5013d100 auth_method 0 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4ffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4ffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4ffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4f7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.199+0000 7f2c4effd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4ffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 10 monclient(hunting): handle_auth_done global_id 104562 payload 824 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 -- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c5013d100 msgr2=0x7f2c501211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2c5013d100 0x7f2c501211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 -- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c500a3ce0 msgr2=0x7f2c50120730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c500a3ce0 0x7f2c50120730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 10 monclient: _finish_hunting 0 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 monclient: found mon.noname-a 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 -- 172.21.15.47:0/733535426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c48009a70 con 0x7f2c50137e90 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4f7fe700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2c500a3ce0 0x7f2c50120730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4effd700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c50137e90 0x7f2c50120c70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800e900 tx=0x7f2c3800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 client.?.objecter ms_handle_connect 0x7f2c50137e90 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c38013af0 con 0x7f2c50137e90 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: dump: 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.219 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c38013c70 con 0x7f2c50137e90 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 5 monclient: authenticate success, global_id 104562 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 10 monclient: _renew_subs 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50122aa0 con 0x7f2c50137e90 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2c38013de0 con 0x7f2c50137e90 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c47fff700 10 monclient: dump: 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.221 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c4dffb700 4 set_mon_vals no callback set 2024-04-20T02:35:51.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 10 client.104562.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 10 monclient: _renew_subs 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5013ba70 con 0x7f2c50137e90 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.200+0000 7f2c561a5700 1 librados: init done 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c561a5700 10 client.104562.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c561a5700 15 client.104562.objecter _session_command_op_assign -1 1 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c561a5700 20 client.104562.objecter _get_session osd=-1 returning homeless 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c561a5700 10 client.104562.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c561a5700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c500a4120 con 0x7f2c50137e90 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2c38013680 con 0x7f2c50137e90 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c3003a320 0x7f2c3003c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f2c3804a690 con 0x7f2c50137e90 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 10 client.104562.objecter ms_dispatch 0x7f2c50001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:51.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 3 client.104562.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 3 client.104562.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 10 client.104562.objecter checking command 1 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f2c300417a0 0x7f2c30043c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 20 client.104562.objecter _get_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 15 client.104562.objecter _session_command_op_remove -1 1 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 20 client.104562.objecter _get_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 20 client.104562.objecter get_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 15 client.104562.objecter _session_command_op_assign 5 1 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 4 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 10 client.104562.objecter _send_command 1 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 --> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c30044310 con 0x7f2c300417a0 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.201+0000 7f2c47fff700 20 client.104562.objecter dump_active .. 0 homeless 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=697) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3804cda0 con 0x7f2c50137e90 2024-04-20T02:35:51.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c47fff700 10 monclient: handle_get_version_reply finishing 1 version 697 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c4f7fe700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c3003a320 0x7f2c3003c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c4f7fe700 10 monclient: get_auth_request con 0x7f2c3003a320 auth_method 0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f2c300417a0 0x7f2c30043c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c4ffff700 10 monclient: get_auth_request con 0x7f2c300417a0 auth_method 0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c4f7fe700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c3003a320 0x7f2c3003c7e0 secure :-1 s=READY pgs=3423 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005910 tx=0x7f2c40009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c47fff700 10 client.104562.objecter ms_handle_connect 0x7f2c3003a320 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c47fff700 10 client.104562.objecter ms_handle_connect 0x7f2c300417a0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.202+0000 7f2c4ffff700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f2c300417a0 0x7f2c30043c40 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.203+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== osd.5 v2:172.21.15.179:6808/42296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2c30044310 con 0x7f2c300417a0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.203+0000 7f2c47fff700 10 client.104562.objecter ms_dispatch 0x7f2c50001550 command_reply(tid 1: 0 ) v1 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.203+0000 7f2c47fff700 10 client.104562.objecter _finish_command 1 = system:0 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.203+0000 7f2c47fff700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 4 2024-04-20T02:35:51.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.203+0000 7f2c47fff700 15 client.104562.objecter _session_command_op_remove 5 1 2024-04-20T02:35:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 10 client.104562.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2024-04-20T02:35:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 15 client.104562.objecter _session_command_op_assign -1 2 2024-04-20T02:35:51.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 20 client.104562.objecter _get_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 20 client.104562.objecter _get_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 15 client.104562.objecter _session_command_op_remove -1 2 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 20 client.104562.objecter get_session s=0x7f2c300410d0 osd=5 3 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 15 client.104562.objecter _session_command_op_assign 5 2 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 4 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 10 client.104562.objecter _send_command 2 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.218+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 --> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2c500a7560 con 0x7f2c300417a0 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c47fff700 1 -- 172.21.15.47:0/733535426 <== osd.5 v2:172.21.15.179:6808/42296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2c4800bb20 con 0x7f2c300417a0 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c47fff700 10 client.104562.objecter ms_dispatch 0x7f2c50001550 command_reply(tid 2: 0 ) v1 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c47fff700 10 client.104562.objecter _finish_command 2 = system:0 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c47fff700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 4 2024-04-20T02:35:51.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c47fff700 15 client.104562.objecter _session_command_op_remove 5 2 2024-04-20T02:35:51.228 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T02:35:51.228 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T02:35:51.228 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 10 librados: watch_flush enter 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 10 librados: watch_flush exit 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 10 client.104562.objecter close_session for osd.5 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f2c300417a0 msgr2=0x7f2c30043c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6808/42296,v1:172.21.15.179:6809/42296] conn(0x7f2c300417a0 0x7f2c30043c40 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 20 client.104562.objecter put_session s=0x7f2c300410d0 osd=5 1 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 20 client.104562.objecter shutdown clearing up homeless session... 2024-04-20T02:35:51.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 10 client.104562.objecter successfully canceled tick 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c3003a320 msgr2=0x7f2c3003c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f2c3003a320 0x7f2c3003c7e0 secure :-1 s=READY pgs=3423 cs=0 l=1 rev1=1 crypto rx=0x7f2c40005910 tx=0x7f2c40009040 comp rx=0 tx=0).stop 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 10 monclient: shutdown 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c50137e90 msgr2=0x7f2c50120c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 --2- 172.21.15.47:0/733535426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c50137e90 0x7f2c50120c70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800e900 tx=0x7f2c3800ec10 comp rx=0 tx=0).stop 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c4ffff700 1 -- 172.21.15.47:0/733535426 reap_dead start 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 shutdown_connections 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 >> 172.21.15.47:0/733535426 conn(0x7f2c500aa710 msgr2=0x7f2c500a8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 shutdown_connections 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 -- 172.21.15.47:0/733535426 wait complete. 2024-04-20T02:35:51.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.219+0000 7f2c561a5700 1 librados: shutdown 2024-04-20T02:35:51.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-20T02:35:51.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.369+0000 7f9cbba93700 10 monclient: get_monmap_and_config 2024-04-20T02:35:51.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.369+0000 7f9cbba93700 10 monclient: build_initial_monmap 2024-04-20T02:35:51.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.369+0000 7f9cbba93700 10 monclient: monmap: 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:51.370811+0000 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:51.370811+0000 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:51.370 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:51.371 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:51.371 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:51.371 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:51.371 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.370+0000 7f9cbba93700 1 Processor -- start 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.370+0000 7f9cbba93700 1 -- start start 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.370+0000 7f9cbba93700 10 monclient: init 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient: _add_conns ranks=[4,0,3,7,5,2,10,1,6,9,8] 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9cb40a5c90 0x7f9cb40a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): picked mon.noname-c con 0x7f9cb40a5c90 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9cb40a0eb0 addr v1:172.21.15.47:6789/0 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cb40a6780 0x7f9cb4143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): picked mon.noname-g con 0x7f9cb40a6780 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:35:51.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb40751e0 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb409fd20 con 0x7f9cb40a6780 2024-04-20T02:35:51.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40a7bc0 con 0x7f9cb40a5c90 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cbba93700 10 monclient(hunting): authenticate will time out at 2830.772114s 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9cb40a5c90 0x7f9cb40a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb902e700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9cb40a0eb0 0x7f9cb40a6070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57938/0 (socket says 172.21.15.47:57938) 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb902e700 1 -- 172.21.15.47:0/3427799128 learned_addr learned my addr 172.21.15.47:0/3427799128 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): get_auth_request con 0x7f9cb40a5c90 auth_method 0 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cba030700 1 --2- 172.21.15.47:0/3427799128 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cb40a6780 0x7f9cb4143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cba030700 10 monclient(hunting): get_auth_request con 0x7f9cb40a6780 auth_method 0 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cba030700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cba030700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cba030700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient(hunting): handle_auth_done global_id 84892 payload 274 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb882d700 1 -- 172.21.15.47:0/3427799128 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249978285 0 0) 0x7f9cb40751e0 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 -- 172.21.15.47:0/3427799128 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cb40a6780 msgr2=0x7f9cb4143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 --2- 172.21.15.47:0/3427799128 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cb40a6780 0x7f9cb4143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 -- 172.21.15.47:0/3427799128 >> v1:172.21.15.47:6789/0 conn(0x7f9cb40a0eb0 legacy=0x7f9cb40a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient: _finish_hunting 0 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 monclient: found mon.noname-c 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:51.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.47:3302/0 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 -- 172.21.15.47:0/3427799128 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb409f760 con 0x7f9cb40a5c90 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb882d700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb982f700 1 --2- 172.21.15.47:0/3427799128 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9cb40a5c90 0x7f9cb40a0970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009870 tx=0x7f9ca8009b80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d6bf1995dd589420 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb882d700 1 -- 172.21.15.47:0/3427799128 <== mon.4 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ca8011aa0 con 0x7f9cb40a5c90 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb882d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb882d700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.371+0000 7f9cb882d700 10 monclient: dump: 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.384 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 10 monclient: _finish_auth 0 2024-04-20T02:35:51.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 10 monclient: _check_auth_tickets 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 1 -- 172.21.15.47:0/3427799128 <== mon.4 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca8011c00 con 0x7f9cb40a5c90 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 5 monclient: authenticate success, global_id 84892 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 1 -- 172.21.15.47:0/3427799128 <== mon.4 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ca8011d70 con 0x7f9cb40a5c90 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 4 set_mon_vals no callback set 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cb882d700 10 monclient: dump: 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.386 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 10 monclient: shutdown 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 -- 172.21.15.47:0/3427799128 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9cb40a5c90 msgr2=0x7f9cb40a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 --2- 172.21.15.47:0/3427799128 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9cb40a5c90 0x7f9cb40a0970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009870 tx=0x7f9ca8009b80 comp rx=0 tx=0).stop 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 -- 172.21.15.47:0/3427799128 shutdown_connections 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 --2- 172.21.15.47:0/3427799128 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9cb40a5c90 0x7f9cb40a0970 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 --2- 172.21.15.47:0/3427799128 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9cb40a6780 0x7f9cb4143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 -- 172.21.15.47:0/3427799128 >> 172.21.15.47:0/3427799128 conn(0x7f9cb40aa8b0 msgr2=0x7f9cb40a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 -- 172.21.15.47:0/3427799128 shutdown_connections 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.372+0000 7f9cbba93700 1 -- 172.21.15.47:0/3427799128 wait complete. 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 10 monclient: build_initial_monmap 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 10 monclient: monmap: 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:51.388 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 1 librados: starting msgr at 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 1 librados: starting objecter 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 1 Processor -- start 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 1 -- start start 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 1 librados: setting wanted keys 2024-04-20T02:35:51.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 1 librados: calling monclient init 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.373+0000 7f9cbba93700 10 monclient: init 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient: _add_conns ranks=[0,6,8,3,7,1,5,2,4] 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 0x7f9cb413e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): picked mon.noname-a con 0x7f9cb40a0eb0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9cb40a5c90 0x7f9cb413e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): picked mon.noname-d con 0x7f9cb40a5c90 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9cb40a6780 0x7f9cb413eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): picked mon.noname-e con 0x7f9cb40a6780 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb413f650 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb409d960 con 0x7f9cb40a5c90 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb409deb0 con 0x7f9cb40a6780 2024-04-20T02:35:51.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cbba93700 10 monclient(hunting): authenticate will time out at 2830.775430s 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 0x7f9cb413e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 0x7f9cb413e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53586/0 (socket says 172.21.15.47:53586) 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 -- 172.21.15.47:0/793841625 learned_addr learned my addr 172.21.15.47:0/793841625 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9cb40a6780 0x7f9cb413eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): get_auth_request con 0x7f9cb40a0eb0 auth_method 0 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9cb40a5c90 0x7f9cb413e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): get_auth_request con 0x7f9cb40a5c90 auth_method 0 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): get_auth_request con 0x7f9cb40a6780 auth_method 0 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cba030700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient(hunting): handle_auth_done global_id 104580 payload 824 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 -- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9cb40a6780 msgr2=0x7f9cb413eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9cb40a6780 0x7f9cb413eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 -- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9cb40a5c90 msgr2=0x7f9cb413e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9cb40a5c90 0x7f9cb413e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient: _finish_hunting 0 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 monclient: found mon.noname-a 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb982f700 1 -- 172.21.15.47:0/793841625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80094a0 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.374+0000 7f9cb902e700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9cb40a5c90 0x7f9cb413e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cb982f700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 0x7f9cb413e440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800a010 tx=0x7f9ca8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 client.?.objecter ms_handle_connect 0x7f9cb40a0eb0 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ca8004490 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: dump: 2024-04-20T02:35:51.393 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: _finish_auth 0 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: _check_auth_tickets 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 5 monclient: authenticate success, global_id 104580 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca801f070 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 10 monclient: _renew_subs 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb4141a20 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:51.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9ca8026480 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 10 client.104580.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca77fe700 4 set_mon_vals no callback set 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9ca67fc700 10 monclient: dump: 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.396 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 10 monclient: _renew_subs 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb413ff30 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.375+0000 7f9cbba93700 1 librados: init done 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f9ca80036a0 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9cbba93700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9cbba93700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9cbba93700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb41414f0 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9c9803a2f0 0x7f9c9803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f9ca8021070 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 10 client.104580.objecter ms_dispatch 0x7f9cb4001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 3 client.104580.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 3 client.104580.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9ca67fc700 20 client.104580.objecter dump_active .. 0 homeless 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9cb902e700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9c9803a2f0 0x7f9c9803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.376+0000 7f9cb902e700 10 monclient: get_auth_request con 0x7f9c9803a2f0 auth_method 0 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.377+0000 7f9cb902e700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9c9803a2f0 0x7f9c9803c7b0 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0007b10 tx=0x7f9cb0007a60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.377+0000 7f9ca67fc700 10 client.104580.objecter ms_handle_connect 0x7f9c9803a2f0 2024-04-20T02:35:51.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.381+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f9ca801a430 con 0x7f9cb40a0eb0 2024-04-20T02:35:51.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.381+0000 7f9ca67fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.381+0000 7f9ca67fc700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:51.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.601+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cb4141360 con 0x7f9c9803a2f0 2024-04-20T02:35:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.602+0000 7f9ca67fc700 1 -- 172.21.15.47:0/793841625 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9cb4141360 con 0x7f9c9803a2f0 2024-04-20T02:35:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 10 librados: watch_flush enter 2024-04-20T02:35:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 10 librados: watch_flush exit 2024-04-20T02:35:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 20 client.104580.objecter shutdown clearing up homeless session... 2024-04-20T02:35:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 10 client.104580.objecter successfully canceled tick 2024-04-20T02:35:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9c9803a2f0 msgr2=0x7f9c9803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9c9803a2f0 0x7f9c9803c7b0 secure :-1 s=READY pgs=3424 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0007b10 tx=0x7f9cb0007a60 comp rx=0 tx=0).stop 2024-04-20T02:35:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 10 monclient: shutdown 2024-04-20T02:35:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 msgr2=0x7f9cb413e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 0x7f9cb413e440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800a010 tx=0x7f9ca8004750 comp rx=0 tx=0).stop 2024-04-20T02:35:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 shutdown_connections 2024-04-20T02:35:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f9c9803a2f0 0x7f9c9803c7b0 unknown :-1 s=CLOSED pgs=3424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f9cb40a6780 0x7f9cb413eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9cb40a0eb0 0x7f9cb413e440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 --2- 172.21.15.47:0/793841625 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f9cb40a5c90 0x7f9cb413e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 >> 172.21.15.47:0/793841625 conn(0x7f9cb40aa8b0 msgr2=0x7f9cb40a8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 shutdown_connections 2024-04-20T02:35:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 -- 172.21.15.47:0/793841625 wait complete. 2024-04-20T02:35:51.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.604+0000 7f9cbba93700 1 librados: shutdown 2024-04-20T02:35:51.610 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 5 to deep-scrub 2024-04-20T02:35:51.618 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:35:51.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.754+0000 7ff923493700 10 monclient: get_monmap_and_config 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.754+0000 7ff923493700 10 monclient: build_initial_monmap 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.754+0000 7ff923493700 10 monclient: monmap: 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:35:51.755749+0000 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:35:51.755749+0000 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:51.755 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.755+0000 7ff923493700 1 Processor -- start 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.755+0000 7ff923493700 1 -- start start 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.755+0000 7ff923493700 10 monclient: init 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.755+0000 7ff923493700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.755+0000 7ff923493700 10 monclient: _add_conns ranks=[2,6,7,4,5,8,10,0,1,9,3] 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.755+0000 7ff923493700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 0x7ff91c06db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): picked mon.noname-b con 0x7ff91c06cc40 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06d020 0x7ff91c137910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): picked mon.noname-d con 0x7ff91c06d020 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 1 --2- >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff91c06d4a0 0x7ff91c137e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): picked mon.noname-i con 0x7ff91c06d4a0 addr [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0a4260 con 0x7ff91c06cc40 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0a4430 con 0x7ff91c06d020 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 1 -- --> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0704e0 con 0x7ff91c06d4a0 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff923493700 10 monclient(hunting): authenticate will time out at 2831.157108s 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 0x7ff91c06db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06d020 0x7ff91c137910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 0x7ff91c06db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:60282/0 (socket says 172.21.15.47:60282) 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 1 -- 172.21.15.47:0/2920124676 learned_addr learned my addr 172.21.15.47:0/2920124676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): get_auth_request con 0x7ff91c06cc40 auth_method 0 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): get_auth_request con 0x7ff91c06d020 auth_method 0 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff921a30700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff91c06d4a0 0x7ff91c137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff921a30700 10 monclient(hunting): get_auth_request con 0x7ff91c06d4a0 auth_method 0 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff921a30700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff921a30700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff921a30700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient(hunting): handle_auth_done global_id 55716 payload 274 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 -- 172.21.15.47:0/2920124676 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff91c06d4a0 msgr2=0x7ff91c137e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff91c06d4a0 0x7ff91c137e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 -- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 msgr2=0x7ff91c06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 0x7ff91c06db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient: _finish_hunting 0 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 monclient: found mon.noname-d 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:51.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 -- 172.21.15.47:0/2920124676 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0751e0 con 0x7ff91c06d020 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff92122f700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 0x7ff91c06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff920a2e700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06d020 0x7ff91c137910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0098d0 tx=0x7ff90c009be0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d955ab48a96d4d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff913fff700 1 -- 172.21.15.47:0/2920124676 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff90c011aa0 con 0x7ff91c06d020 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.756+0000 7ff913fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: dump: 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.771 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: _finish_auth 0 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: _check_auth_tickets 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:51.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 1 -- 172.21.15.47:0/2920124676 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c011c20 con 0x7ff91c06d020 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 5 monclient: authenticate success, global_id 55716 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 10 monclient: get_monmap_and_config success 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 1 -- 172.21.15.47:0/2920124676 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff90c011d90 con 0x7ff91c06d020 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 4 set_mon_vals no callback set 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff913fff700 10 monclient: dump: 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 10 monclient: shutdown 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 -- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06d020 msgr2=0x7ff91c137910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06d020 0x7ff91c137910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0098d0 tx=0x7ff90c009be0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 -- 172.21.15.47:0/2920124676 shutdown_connections 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] conn(0x7ff91c06d4a0 0x7ff91c137e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff91c06cc40 0x7ff91c06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 --2- 172.21.15.47:0/2920124676 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06d020 0x7ff91c137910 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 -- 172.21.15.47:0/2920124676 >> 172.21.15.47:0/2920124676 conn(0x7ff91c0a5680 msgr2=0x7ff91c066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 -- 172.21.15.47:0/2920124676 shutdown_connections 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.757+0000 7ff923493700 1 -- 172.21.15.47:0/2920124676 wait complete. 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 10 monclient: build_initial_monmap 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 10 monclient: monmap: 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:35:51.775 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 1 librados: starting msgr at 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 1 librados: starting objecter 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 1 Processor -- start 2024-04-20T02:35:51.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 1 -- start start 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 1 librados: setting wanted keys 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 1 librados: calling monclient init 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.758+0000 7ff923493700 10 monclient: init 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient: _reopen_session rank -1 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient: _add_conns ranks=[6,8,0,5,2,4,3,1,7] 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 0x7ff91c137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): picked mon.noname-d con 0x7ff91c06cc40 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 0x7ff91c138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): picked mon.noname-e con 0x7ff91c06d020 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff91c06d4a0 0x7ff91c1c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): picked mon.noname-a con 0x7ff91c06d4a0 addr [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c138650 con 0x7ff91c06d4a0 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c125340 con 0x7ff91c06cc40 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): start opening mon connection 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c072540 con 0x7ff91c06d020 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): _renew_subs 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff923493700 10 monclient(hunting): authenticate will time out at 2831.160299s 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 0x7ff91c137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 0x7ff91c137bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:56438/0 (socket says 172.21.15.47:56438) 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 1 -- 172.21.15.47:0/793301411 learned_addr learned my addr 172.21.15.47:0/793301411 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): get_auth_request con 0x7ff91c06cc40 auth_method 0 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 0x7ff91c138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 10 monclient(hunting): get_auth_request con 0x7ff91c06d020 auth_method 0 2024-04-20T02:35:51.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff91c06d4a0 0x7ff91c1c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): get_auth_request con 0x7ff91c06d4a0 auth_method 0 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 1 -- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 msgr2=0x7ff91c138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 1 -- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 msgr2=0x7ff91c138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 0x7ff91c138110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff920a2e700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 0x7ff91c138110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient(hunting): handle_auth_done global_id 104589 payload 824 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 -- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 msgr2=0x7ff91c138110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 0x7ff91c138110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 -- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 msgr2=0x7ff91c137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 0x7ff91c137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient: _finish_hunting 0 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 monclient: found mon.noname-a 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff921a30700 1 -- 172.21.15.47:0/793301411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009550 con 0x7ff91c06d4a0 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.759+0000 7ff92122f700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 0x7ff91c137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:35:51.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff921a30700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff91c06d4a0 0x7ff91c1c8c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff914007d10 tx=0x7ff91400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 client.?.objecter ms_handle_connect 0x7ff91c06d4a0 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 client.?.objecter resend_mon_ops 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff914018070 con 0x7ff91c06d4a0 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: dump: 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.781 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: _finish_auth 0 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:35:51.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff914007e00 con 0x7ff91c06d4a0 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 5 monclient: authenticate success, global_id 104589 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 10 monclient: _renew_subs 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c0094a0 con 0x7ff91c06d4a0 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff914014d00 con 0x7ff91c06d4a0 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff911ffb700 10 monclient: dump: 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:35:51.783 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:35:51.784 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff9137fe700 4 set_mon_vals no callback set 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 10 client.104589.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 10 monclient: _renew_subs 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c124fd0 con 0x7ff91c06d4a0 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.760+0000 7ff923493700 1 librados: init done 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ff91400ebe0 con 0x7ff91c06d4a0 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff90003a2f0 0x7ff90003c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(697..697 src has 266..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7ff91404a220 con 0x7ff91c06d4a0 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 10 client.104589.objecter ms_dispatch 0x7ff91c001550 osd_map(697..697 src has 266..697) v4 2024-04-20T02:35:51.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 3 client.104589.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 3 client.104589.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff923493700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff923493700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff923493700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.47:3300/0 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff911ffb700 20 client.104589.objecter dump_active .. 0 homeless 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c067f90 con 0x7ff91c06d4a0 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.761+0000 7ff92122f700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff90003a2f0 0x7ff90003c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.762+0000 7ff92122f700 10 monclient: get_auth_request con 0x7ff90003a2f0 auth_method 0 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.762+0000 7ff92122f700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff90003a2f0 0x7ff90003c7b0 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 crypto rx=0x7ff918006d30 tx=0x7ff9180092c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:35:51.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.762+0000 7ff911ffb700 10 client.104589.objecter ms_handle_connect 0x7ff90003a2f0 2024-04-20T02:35:51.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.767+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ff914014e60 con 0x7ff91c06d4a0 2024-04-20T02:35:51.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.767+0000 7ff911ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:35:51.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.767+0000 7ff911ffb700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:35:51.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.980+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff91c06c800 con 0x7ff90003a2f0 2024-04-20T02:35:51.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.984+0000 7ff911ffb700 1 -- 172.21.15.47:0/793301411 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+133404 (secure 0 0 0) 0x7ff918011e10 con 0x7ff90003a2f0 2024-04-20T02:35:51.986 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 10 librados: watch_flush enter 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 10 librados: watch_flush exit 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 20 client.104589.objecter shutdown clearing up homeless session... 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 10 client.104589.objecter successfully canceled tick 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff90003a2f0 msgr2=0x7ff90003c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff90003a2f0 0x7ff90003c7b0 secure :-1 s=READY pgs=3425 cs=0 l=1 rev1=1 crypto rx=0x7ff918006d30 tx=0x7ff9180092c0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 10 monclient: shutdown 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff91c06d4a0 msgr2=0x7ff91c1c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff91c06d4a0 0x7ff91c1c8c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff914007d10 tx=0x7ff91400e3c0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 shutdown_connections 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7ff90003a2f0 0x7ff90003c7b0 unknown :-1 s=CLOSED pgs=3425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff91c06d4a0 0x7ff91c1c8c90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7ff91c06cc40 0x7ff91c137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 --2- 172.21.15.47:0/793301411 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7ff91c06d020 0x7ff91c138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 >> 172.21.15.47:0/793301411 conn(0x7ff91c0a5680 msgr2=0x7ff91c071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 shutdown_connections 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 -- 172.21.15.47:0/793301411 wait complete. 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:35:51.987+0000 7ff923493700 1 librados: shutdown 2024-04-20T02:35:51.989 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:35:52.001 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1295,"stamp":"2024-04-20T02:35:50.615457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":106,"num_write":166,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470332,"kb_used_data":38368,"kb_used_omap":9,"kb_used_meta":431862,"kb_avail":565760708,"statfs":{"total":579820584960,"available":579338964992,"internally_reserved":0,"allocated":39288832,"data_stored":20149014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":442227367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002302"},"pg_stats":[{"pgid":"5.1c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794551+0000","last_change":"2024-04-20T02:16:44.180381+0000","last_active":"2024-04-20T02:34:02.794551+0000","last_peered":"2024-04-20T02:34:02.794551+0000","last_clean":"2024-04-20T02:34:02.794551+0000","last_became_active":"2024-04-20T02:16:44.179944+0000","last_became_peered":"2024-04-20T02:16:44.179944+0000","last_unstale":"2024-04-20T02:34:02.794551+0000","last_undegraded":"2024-04-20T02:34:02.794551+0000","last_fullsized":"2024-04-20T02:34:02.794551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:23:16.877269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":685,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794538+0000","last_change":"2024-04-20T02:16:44.131425+0000","last_active":"2024-04-20T02:34:02.794538+0000","last_peered":"2024-04-20T02:34:02.794538+0000","last_clean":"2024-04-20T02:34:02.794538+0000","last_became_active":"2024-04-20T02:16:44.130319+0000","last_became_peered":"2024-04-20T02:16:44.130319+0000","last_unstale":"2024-04-20T02:34:02.794538+0000","last_undegraded":"2024-04-20T02:34:02.794538+0000","last_fullsized":"2024-04-20T02:34:02.794538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:11:12.232716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467131+0000","last_change":"2024-04-20T02:16:44.124440+0000","last_active":"2024-04-20T02:34:02.467131+0000","last_peered":"2024-04-20T02:34:02.467131+0000","last_clean":"2024-04-20T02:34:02.467131+0000","last_became_active":"2024-04-20T02:16:44.117655+0000","last_became_peered":"2024-04-20T02:16:44.117655+0000","last_unstale":"2024-04-20T02:34:02.467131+0000","last_undegraded":"2024-04-20T02:34:02.467131+0000","last_fullsized":"2024-04-20T02:34:02.467131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:25:01.668733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":680,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:24.384332+0000","last_change":"2024-04-20T02:16:44.113440+0000","last_active":"2024-04-20T02:35:24.384332+0000","last_peered":"2024-04-20T02:35:24.384332+0000","last_clean":"2024-04-20T02:35:24.384332+0000","last_became_active":"2024-04-20T02:16:44.111746+0000","last_became_peered":"2024-04-20T02:16:44.111746+0000","last_unstale":"2024-04-20T02:35:24.384332+0000","last_undegraded":"2024-04-20T02:35:24.384332+0000","last_fullsized":"2024-04-20T02:35:24.384332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794542+0000","last_change":"2024-04-20T02:16:44.117853+0000","last_active":"2024-04-20T02:34:02.794542+0000","last_peered":"2024-04-20T02:34:02.794542+0000","last_clean":"2024-04-20T02:34:02.794542+0000","last_became_active":"2024-04-20T02:16:44.117331+0000","last_became_peered":"2024-04-20T02:16:44.117331+0000","last_unstale":"2024-04-20T02:34:02.794542+0000","last_undegraded":"2024-04-20T02:34:02.794542+0000","last_fullsized":"2024-04-20T02:34:02.794542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:19:07.833372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466150+0000","last_change":"2024-04-20T02:16:44.123832+0000","last_active":"2024-04-20T02:34:02.466150+0000","last_peered":"2024-04-20T02:34:02.466150+0000","last_clean":"2024-04-20T02:34:02.466150+0000","last_became_active":"2024-04-20T02:16:44.123156+0000","last_became_peered":"2024-04-20T02:16:44.123156+0000","last_unstale":"2024-04-20T02:34:02.466150+0000","last_undegraded":"2024-04-20T02:34:02.466150+0000","last_fullsized":"2024-04-20T02:34:02.466150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:33:01.391874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793886+0000","last_change":"2024-04-20T02:16:44.117753+0000","last_active":"2024-04-20T02:34:02.793886+0000","last_peered":"2024-04-20T02:34:02.793886+0000","last_clean":"2024-04-20T02:34:02.793886+0000","last_became_active":"2024-04-20T02:16:44.117507+0000","last_became_peered":"2024-04-20T02:16:44.117507+0000","last_unstale":"2024-04-20T02:34:02.793886+0000","last_undegraded":"2024-04-20T02:34:02.793886+0000","last_fullsized":"2024-04-20T02:34:02.793886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:23:47.554656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":671,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.465204+0000","last_change":"2024-04-20T02:16:44.229184+0000","last_active":"2024-04-20T02:34:02.465204+0000","last_peered":"2024-04-20T02:34:02.465204+0000","last_clean":"2024-04-20T02:34:02.465204+0000","last_became_active":"2024-04-20T02:16:44.228394+0000","last_became_peered":"2024-04-20T02:16:44.228394+0000","last_unstale":"2024-04-20T02:34:02.465204+0000","last_undegraded":"2024-04-20T02:34:02.465204+0000","last_fullsized":"2024-04-20T02:34:02.465204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793947+0000","last_change":"2024-04-20T02:16:44.171109+0000","last_active":"2024-04-20T02:34:02.793947+0000","last_peered":"2024-04-20T02:34:02.793947+0000","last_clean":"2024-04-20T02:34:02.793947+0000","last_became_active":"2024-04-20T02:16:44.170848+0000","last_became_peered":"2024-04-20T02:16:44.170848+0000","last_unstale":"2024-04-20T02:34:02.793947+0000","last_undegraded":"2024-04-20T02:34:02.793947+0000","last_fullsized":"2024-04-20T02:34:02.793947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:07:14.425892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.458239+0000","last_change":"2024-04-20T02:16:44.144117+0000","last_active":"2024-04-20T02:34:01.458239+0000","last_peered":"2024-04-20T02:34:01.458239+0000","last_clean":"2024-04-20T02:34:01.458239+0000","last_became_active":"2024-04-20T02:16:44.139769+0000","last_became_peered":"2024-04-20T02:16:44.139769+0000","last_unstale":"2024-04-20T02:34:01.458239+0000","last_undegraded":"2024-04-20T02:34:01.458239+0000","last_fullsized":"2024-04-20T02:34:01.458239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:36:23.054719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794547+0000","last_change":"2024-04-20T02:16:44.190078+0000","last_active":"2024-04-20T02:34:02.794547+0000","last_peered":"2024-04-20T02:34:02.794547+0000","last_clean":"2024-04-20T02:34:02.794547+0000","last_became_active":"2024-04-20T02:16:44.187150+0000","last_became_peered":"2024-04-20T02:16:44.187150+0000","last_unstale":"2024-04-20T02:34:02.794547+0000","last_undegraded":"2024-04-20T02:34:02.794547+0000","last_fullsized":"2024-04-20T02:34:02.794547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":680,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:04.372723+0000","last_change":"2024-04-20T02:16:44.174024+0000","last_active":"2024-04-20T02:34:04.372723+0000","last_peered":"2024-04-20T02:34:04.372723+0000","last_clean":"2024-04-20T02:34:04.372723+0000","last_became_active":"2024-04-20T02:16:44.167097+0000","last_became_peered":"2024-04-20T02:16:44.167097+0000","last_unstale":"2024-04-20T02:34:04.372723+0000","last_undegraded":"2024-04-20T02:34:04.372723+0000","last_fullsized":"2024-04-20T02:34:04.372723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467259+0000","last_change":"2024-04-20T02:16:44.168983+0000","last_active":"2024-04-20T02:34:02.467259+0000","last_peered":"2024-04-20T02:34:02.467259+0000","last_clean":"2024-04-20T02:34:02.467259+0000","last_became_active":"2024-04-20T02:16:44.168588+0000","last_became_peered":"2024-04-20T02:16:44.168588+0000","last_unstale":"2024-04-20T02:34:02.467259+0000","last_undegraded":"2024-04-20T02:34:02.467259+0000","last_fullsized":"2024-04-20T02:34:02.467259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:54:25.176259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467132+0000","last_change":"2024-04-20T02:16:44.104041+0000","last_active":"2024-04-20T02:34:02.467132+0000","last_peered":"2024-04-20T02:34:02.467132+0000","last_clean":"2024-04-20T02:34:02.467132+0000","last_became_active":"2024-04-20T02:16:44.103711+0000","last_became_peered":"2024-04-20T02:16:44.103711+0000","last_unstale":"2024-04-20T02:34:02.467132+0000","last_undegraded":"2024-04-20T02:34:02.467132+0000","last_fullsized":"2024-04-20T02:34:02.467132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:57:43.734300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.208649+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.207027+0000","last_became_peered":"2024-04-20T02:16:44.207027+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:48:34.254227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793955+0000","last_change":"2024-04-20T02:16:44.209432+0000","last_active":"2024-04-20T02:34:02.793955+0000","last_peered":"2024-04-20T02:34:02.793955+0000","last_clean":"2024-04-20T02:34:02.793955+0000","last_became_active":"2024-04-20T02:16:44.209154+0000","last_became_peered":"2024-04-20T02:16:44.209154+0000","last_unstale":"2024-04-20T02:34:02.793955+0000","last_undegraded":"2024-04-20T02:34:02.793955+0000","last_fullsized":"2024-04-20T02:34:02.793955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:55:12.327168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467157+0000","last_change":"2024-04-20T02:16:44.129276+0000","last_active":"2024-04-20T02:34:02.467157+0000","last_peered":"2024-04-20T02:34:02.467157+0000","last_clean":"2024-04-20T02:34:02.467157+0000","last_became_active":"2024-04-20T02:16:44.116619+0000","last_became_peered":"2024-04-20T02:16:44.116619+0000","last_unstale":"2024-04-20T02:34:02.467157+0000","last_undegraded":"2024-04-20T02:34:02.467157+0000","last_fullsized":"2024-04-20T02:34:02.467157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:26.681520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":680,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:44.377706+0000","last_change":"2024-04-20T02:16:44.169597+0000","last_active":"2024-04-20T02:34:44.377706+0000","last_peered":"2024-04-20T02:34:44.377706+0000","last_clean":"2024-04-20T02:34:44.377706+0000","last_became_active":"2024-04-20T02:16:44.168518+0000","last_became_peered":"2024-04-20T02:16:44.168518+0000","last_unstale":"2024-04-20T02:34:44.377706+0000","last_undegraded":"2024-04-20T02:34:44.377706+0000","last_fullsized":"2024-04-20T02:34:44.377706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794535+0000","last_change":"2024-04-20T02:16:44.157424+0000","last_active":"2024-04-20T02:34:02.794535+0000","last_peered":"2024-04-20T02:34:02.794535+0000","last_clean":"2024-04-20T02:34:02.794535+0000","last_became_active":"2024-04-20T02:16:44.155758+0000","last_became_peered":"2024-04-20T02:16:44.155758+0000","last_unstale":"2024-04-20T02:34:02.794535+0000","last_undegraded":"2024-04-20T02:34:02.794535+0000","last_fullsized":"2024-04-20T02:34:02.794535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:33:30.913418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467066+0000","last_change":"2024-04-20T02:15:46.242526+0000","last_active":"2024-04-20T02:34:02.467066+0000","last_peered":"2024-04-20T02:34:02.467066+0000","last_clean":"2024-04-20T02:34:02.467066+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:34:02.467066+0000","last_undegraded":"2024-04-20T02:34:02.467066+0000","last_fullsized":"2024-04-20T02:34:02.467066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:25.463150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":697,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466066+0000","last_change":"2024-04-20T02:16:44.190776+0000","last_active":"2024-04-20T02:34:02.466066+0000","last_peered":"2024-04-20T02:34:02.466066+0000","last_clean":"2024-04-20T02:34:02.466066+0000","last_became_active":"2024-04-20T02:16:44.190566+0000","last_became_peered":"2024-04-20T02:16:44.190566+0000","last_unstale":"2024-04-20T02:34:02.466066+0000","last_undegraded":"2024-04-20T02:34:02.466066+0000","last_fullsized":"2024-04-20T02:34:02.466066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:35:06.531468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.467146+0000","last_change":"2024-04-20T02:16:44.169957+0000","last_active":"2024-04-20T02:34:02.467146+0000","last_peered":"2024-04-20T02:34:02.467146+0000","last_clean":"2024-04-20T02:34:02.467146+0000","last_became_active":"2024-04-20T02:16:44.169063+0000","last_became_peered":"2024-04-20T02:16:44.169063+0000","last_unstale":"2024-04-20T02:34:02.467146+0000","last_undegraded":"2024-04-20T02:34:02.467146+0000","last_fullsized":"2024-04-20T02:34:02.467146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T09:05:26.309126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":676,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:34.563300+0000","last_change":"2024-04-20T02:15:46.246460+0000","last_active":"2024-04-20T02:34:34.563300+0000","last_peered":"2024-04-20T02:34:34.563300+0000","last_clean":"2024-04-20T02:34:34.563300+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:34:34.563300+0000","last_undegraded":"2024-04-20T02:34:34.563300+0000","last_fullsized":"2024-04-20T02:34:34.563300+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466569+0000","last_change":"2024-04-20T02:16:44.151129+0000","last_active":"2024-04-20T02:34:02.466569+0000","last_peered":"2024-04-20T02:34:02.466569+0000","last_clean":"2024-04-20T02:34:02.466569+0000","last_became_active":"2024-04-20T02:16:44.150893+0000","last_became_peered":"2024-04-20T02:16:44.150893+0000","last_unstale":"2024-04-20T02:34:02.466569+0000","last_undegraded":"2024-04-20T02:34:02.466569+0000","last_fullsized":"2024-04-20T02:34:02.466569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:43.272447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793882+0000","last_change":"2024-04-20T02:15:46.241847+0000","last_active":"2024-04-20T02:34:02.793882+0000","last_peered":"2024-04-20T02:34:02.793882+0000","last_clean":"2024-04-20T02:34:02.793882+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:34:02.793882+0000","last_undegraded":"2024-04-20T02:34:02.793882+0000","last_fullsized":"2024-04-20T02:34:02.793882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:08:41.774807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:34.376494+0000","last_change":"2024-04-20T02:15:46.243481+0000","last_active":"2024-04-20T02:34:34.376494+0000","last_peered":"2024-04-20T02:34:34.376494+0000","last_clean":"2024-04-20T02:34:34.376494+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:34:34.376494+0000","last_undegraded":"2024-04-20T02:34:34.376494+0000","last_fullsized":"2024-04-20T02:34:34.376494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":835,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:49.337296+0000","last_change":"2024-04-20T02:15:46.243565+0000","last_active":"2024-04-20T02:35:49.337296+0000","last_peered":"2024-04-20T02:35:49.337296+0000","last_clean":"2024-04-20T02:35:49.337296+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:35:49.337296+0000","last_undegraded":"2024-04-20T02:35:49.337296+0000","last_fullsized":"2024-04-20T02:35:49.337296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":673,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:29.569421+0000","last_change":"2024-04-20T02:16:44.176094+0000","last_active":"2024-04-20T02:35:29.569421+0000","last_peered":"2024-04-20T02:35:29.569421+0000","last_clean":"2024-04-20T02:35:29.569421+0000","last_became_active":"2024-04-20T02:16:44.175527+0000","last_became_peered":"2024-04-20T02:16:44.175527+0000","last_unstale":"2024-04-20T02:35:29.569421+0000","last_undegraded":"2024-04-20T02:35:29.569421+0000","last_fullsized":"2024-04-20T02:35:29.569421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":672,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:04.559935+0000","last_change":"2024-04-20T02:16:44.210652+0000","last_active":"2024-04-20T02:34:04.559935+0000","last_peered":"2024-04-20T02:34:04.559935+0000","last_clean":"2024-04-20T02:34:04.559935+0000","last_became_active":"2024-04-20T02:16:44.209341+0000","last_became_peered":"2024-04-20T02:16:44.209341+0000","last_unstale":"2024-04-20T02:34:04.559935+0000","last_undegraded":"2024-04-20T02:34:04.559935+0000","last_fullsized":"2024-04-20T02:34:04.559935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":689,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794438+0000","last_change":"2024-04-20T02:15:46.243402+0000","last_active":"2024-04-20T02:34:02.794438+0000","last_peered":"2024-04-20T02:34:02.794438+0000","last_clean":"2024-04-20T02:34:02.794438+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:34:02.794438+0000","last_undegraded":"2024-04-20T02:34:02.794438+0000","last_fullsized":"2024-04-20T02:34:02.794438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T04:15:00.376105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:34.376523+0000","last_change":"2024-04-20T02:15:46.243529+0000","last_active":"2024-04-20T02:34:34.376523+0000","last_peered":"2024-04-20T02:34:34.376523+0000","last_clean":"2024-04-20T02:34:34.376523+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:34:34.376523+0000","last_undegraded":"2024-04-20T02:34:34.376523+0000","last_fullsized":"2024-04-20T02:34:34.376523+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":673,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:29.569395+0000","last_change":"2024-04-20T02:16:44.153062+0000","last_active":"2024-04-20T02:35:29.569395+0000","last_peered":"2024-04-20T02:35:29.569395+0000","last_clean":"2024-04-20T02:35:29.569395+0000","last_became_active":"2024-04-20T02:16:44.150765+0000","last_became_peered":"2024-04-20T02:16:44.150765+0000","last_unstale":"2024-04-20T02:35:29.569395+0000","last_undegraded":"2024-04-20T02:35:29.569395+0000","last_fullsized":"2024-04-20T02:35:29.569395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":700,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466644+0000","last_change":"2024-04-20T02:15:46.245081+0000","last_active":"2024-04-20T02:34:02.466644+0000","last_peered":"2024-04-20T02:34:02.466644+0000","last_clean":"2024-04-20T02:34:02.466644+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:34:02.466644+0000","last_undegraded":"2024-04-20T02:34:02.466644+0000","last_fullsized":"2024-04-20T02:34:02.466644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:59:56.768364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"18'7","reported_seq":694,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:29.384884+0000","last_change":"2024-04-20T02:16:44.105521+0000","last_active":"2024-04-20T02:35:29.384884+0000","last_peered":"2024-04-20T02:35:29.384884+0000","last_clean":"2024-04-20T02:35:29.384884+0000","last_became_active":"2024-04-20T02:16:44.105229+0000","last_became_peered":"2024-04-20T02:16:44.105229+0000","last_unstale":"2024-04-20T02:35:29.384884+0000","last_undegraded":"2024-04-20T02:35:29.384884+0000","last_fullsized":"2024-04-20T02:35:29.384884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:19.383751+0000","last_change":"2024-04-20T02:15:46.245315+0000","last_active":"2024-04-20T02:35:19.383751+0000","last_peered":"2024-04-20T02:35:19.383751+0000","last_clean":"2024-04-20T02:35:19.383751+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:35:19.383751+0000","last_undegraded":"2024-04-20T02:35:19.383751+0000","last_fullsized":"2024-04-20T02:35:19.383751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:15:45.226760+0000","last_clean_scrub_stamp":"2024-04-20T02:15:45.226760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.793880+0000","last_change":"2024-04-20T02:16:44.108891+0000","last_active":"2024-04-20T02:34:02.793880+0000","last_peered":"2024-04-20T02:34:02.793880+0000","last_clean":"2024-04-20T02:34:02.793880+0000","last_became_active":"2024-04-20T02:16:44.108254+0000","last_became_peered":"2024-04-20T02:16:44.108254+0000","last_unstale":"2024-04-20T02:34:02.793880+0000","last_undegraded":"2024-04-20T02:34:02.793880+0000","last_fullsized":"2024-04-20T02:34:02.793880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:29:23.508086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.466402+0000","last_change":"2024-04-20T02:16:44.156064+0000","last_active":"2024-04-20T02:34:02.466402+0000","last_peered":"2024-04-20T02:34:02.466402+0000","last_clean":"2024-04-20T02:34:02.466402+0000","last_became_active":"2024-04-20T02:16:44.142929+0000","last_became_peered":"2024-04-20T02:16:44.142929+0000","last_unstale":"2024-04-20T02:34:02.466402+0000","last_undegraded":"2024-04-20T02:34:02.466402+0000","last_fullsized":"2024-04-20T02:34:02.466402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:33:42.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:14.373944+0000","last_change":"2024-04-20T02:16:44.162486+0000","last_active":"2024-04-20T02:34:14.373944+0000","last_peered":"2024-04-20T02:34:14.373944+0000","last_clean":"2024-04-20T02:34:14.373944+0000","last_became_active":"2024-04-20T02:16:44.159975+0000","last_became_peered":"2024-04-20T02:16:44.159975+0000","last_unstale":"2024-04-20T02:34:14.373944+0000","last_undegraded":"2024-04-20T02:34:14.373944+0000","last_fullsized":"2024-04-20T02:34:14.373944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794549+0000","last_change":"2024-04-20T02:16:44.131415+0000","last_active":"2024-04-20T02:34:02.794549+0000","last_peered":"2024-04-20T02:34:02.794549+0000","last_clean":"2024-04-20T02:34:02.794549+0000","last_became_active":"2024-04-20T02:16:44.131124+0000","last_became_peered":"2024-04-20T02:16:44.131124+0000","last_unstale":"2024-04-20T02:34:02.794549+0000","last_undegraded":"2024-04-20T02:34:02.794549+0000","last_fullsized":"2024-04-20T02:34:02.794549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:41:15.350582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":679,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:34:02.794272+0000","last_change":"2024-04-20T02:16:44.142261+0000","last_active":"2024-04-20T02:34:02.794272+0000","last_peered":"2024-04-20T02:34:02.794272+0000","last_clean":"2024-04-20T02:34:02.794272+0000","last_became_active":"2024-04-20T02:16:44.141808+0000","last_became_peered":"2024-04-20T02:16:44.141808+0000","last_unstale":"2024-04-20T02:34:02.794272+0000","last_undegraded":"2024-04-20T02:34:02.794272+0000","last_fullsized":"2024-04-20T02:34:02.794272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":695,"reported_epoch":696,"state":"active+clean","last_fresh":"2024-04-20T02:34:01.455383+0000","last_change":"2024-04-20T02:16:44.100117+0000","last_active":"2024-04-20T02:34:01.455383+0000","last_peered":"2024-04-20T02:34:01.455383+0000","last_clean":"2024-04-20T02:34:01.455383+0000","last_became_active":"2024-04-20T02:16:44.099559+0000","last_became_peered":"2024-04-20T02:16:44.099559+0000","last_unstale":"2024-04-20T02:34:01.455383+0000","last_undegraded":"2024-04-20T02:34:01.455383+0000","last_fullsized":"2024-04-20T02:34:01.455383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:21:50.484507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607798,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":6373376,"data_stored":3181863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.314,"15min":0.22900000000000001},"max":{"1min":0.68000000000000005,"5min":0.80000000000000004,"15min":2.9460000000000002},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":2.8570000000000002},"last":0.39000000000000001}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.30399999999999999},"max":{"1min":0.64500000000000002,"5min":0.76900000000000002,"15min":2.391},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":1.292},"last":0.46100000000000002}]},{"osd":2,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.747,"15min":1.1899999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68999999999999995,"5min":0.81599999999999995,"15min":1.2450000000000001},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":1.3029999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.25900000000000001},"max":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.65400000000000003},"last":0.374}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.40999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":1.534},"last":0.438}]}]},{"osd":3,"up_from":12,"seq":51539607797,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77532,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294308,"statfs":{"total":96636764160,"available":96557371392,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.22600000000000001},"max":{"1min":0.748,"5min":1.0880000000000001,"15min":1.4550000000000001},"last":0.371},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":1.103,"15min":1.397},"last":0.435}]},{"osd":1,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.439,"5min":0.307,"15min":0.26000000000000001},"max":{"1min":0.70199999999999996,"5min":1.03,"15min":1.5409999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":1.073,"15min":1.5880000000000001},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":1.0600000000000001,"15min":1.5589999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70899999999999996,"5min":1.1259999999999999,"15min":2.2599999999999998},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.24299999999999999},"max":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":1.0469999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.71099999999999997,"15min":1.335},"last":0.35699999999999998}]},{"osd":5,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.23100000000000001},"max":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.90500000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.29899999999999999,"15min":0.27900000000000003},"max":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.95899999999999996},"last":0.48999999999999999}]}]},{"osd":2,"up_from":12,"seq":51539607797,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69344,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94302496,"statfs":{"total":96636764160,"available":96565755904,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"max":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.88800000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.48399999999999999,"5min":0.67600000000000005,"15min":0.96899999999999997},"last":0.34699999999999998}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.28899999999999998,"5min":0.28699999999999998,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.58099999999999996,"15min":0.97799999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.33200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.45700000000000002,"5min":0.61799999999999999,"15min":1.089},"last":0.35999999999999999}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.62,"5min":0.77000000000000002,"15min":3.823},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.34499999999999997},"max":{"1min":1.226,"5min":1.226,"15min":3.1440000000000001},"last":0.39500000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81200000000000006,"15min":1.1519999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.52000000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.66500000000000004,"5min":0.85499999999999998,"15min":1.2270000000000001},"last":0.377}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.1919999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.51500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":0.92000000000000004,"15min":1.2070000000000001},"last":0.49299999999999999}]}]},{"osd":1,"up_from":12,"seq":51539607800,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6744,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6905856,"data_stored":3710791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.48699999999999999,"15min":1.8220000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.247},"max":{"1min":0.51100000000000001,"5min":0.73499999999999999,"15min":0.92700000000000005},"last":0.36499999999999999}]},{"osd":2,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.55400000000000005,"15min":2.649},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.26500000000000001},"max":{"1min":0.57999999999999996,"5min":0.65900000000000003,"15min":2.5499999999999998},"last":0.34899999999999998}]},{"osd":3,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.98699999999999999,"15min":1.034},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.68500000000000005,"5min":0.90100000000000002,"15min":1.7450000000000001},"last":0.38}]},{"osd":4,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":0.438},{"interface":"front","average":{"1min":0.629,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.35999999999999999},"max":{"1min":0.77000000000000002,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":1.282},"last":0.432}]}]},{"osd":5,"up_from":11,"seq":47244640500,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82148,"kb_used_data":6740,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289692,"statfs":{"total":96636764160,"available":96552644608,"internally_reserved":0,"allocated":6901760,"data_stored":3710786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.28599999999999998,"15min":0.247},"max":{"1min":0.68000000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.37,"15min":0.29699999999999999},"max":{"1min":0.66400000000000003,"5min":0.73599999999999999,"15min":0.86299999999999999},"last":0.32900000000000001}]},{"osd":1,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":0.82999999999999996,"15min":0.92300000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.83599999999999997},"last":0.504}]},{"osd":2,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.69299999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.22800000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65700000000000003},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.34799999999999998,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.84399999999999997},"last":0.42299999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607798,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81624,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94290216,"statfs":{"total":96636764160,"available":96553181184,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77199815},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.23300000000000001},"max":{"1min":0.45900000000000002,"5min":0.56699999999999995,"15min":0.71699999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.8440000000000001},"last":0.30599999999999999}]},{"osd":2,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":7.8849999999999998},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.308,"15min":0.27500000000000002},"max":{"1min":0.54400000000000004,"5min":0.71799999999999997,"15min":4.9870000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":1.2410000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.46300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85599999999999998,"15min":1.2969999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":0.98899999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61599999999999999},"min":{"1min":0.54900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.90400000000000003,"15min":0.98999999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.85399999999999998,"15min":1.0069999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":1.0269999999999999},"last":0.499}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:35:52.007 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.007 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.007 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.007 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.007 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.007 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.008 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.008 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.008 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.008 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.008 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.008 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.009 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.010 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-20T02:15:45.226760+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=15, tm_sec=45, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.011 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.012 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:35:52.012 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-20T02:36:12.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:36:12.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.173+0000 7f5e90877700 10 monclient: get_monmap_and_config 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.173+0000 7f5e90877700 10 monclient: build_initial_monmap 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.174+0000 7f5e90877700 10 monclient: monmap: 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:36:12.174133+0000 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:36:12.174133+0000 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:36:12.173 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:36:12.174 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.174+0000 7f5e90877700 1 Processor -- start 2024-04-20T02:36:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.174+0000 7f5e90877700 1 -- start start 2024-04-20T02:36:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.174+0000 7f5e90877700 10 monclient: init 2024-04-20T02:36:12.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient: _reopen_session rank -1 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient: _add_conns ranks=[10,6,1,0,4,3,7,8,5,9,2] 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5e8c0a0ad0 0x7f5e8c0a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): picked mon.noname-f con 0x7f5e8c0a0ad0 addr v2:172.21.15.179:3300/0 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 0x7f5e8c0a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): picked mon.noname-d con 0x7f5e8c0a2800 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5e8c0a31c0 addr v1:172.21.15.179:6789/0 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c0751e0 con 0x7f5e8c0a31c0 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c09ee40 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0a3630 con 0x7f5e8c0a2800 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): _renew_subs 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e90877700 10 monclient(hunting): authenticate will time out at 2851.575501s 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 0x7f5e8c0a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:12.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 0x7f5e8c0a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:34452/0 (socket says 172.21.15.47:34452) 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 1 -- 172.21.15.47:0/4131078305 learned_addr learned my addr 172.21.15.47:0/4131078305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): get_auth_request con 0x7f5e8c0a2800 auth_method 0 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/4131078305 >> v2:172.21.15.179:3300/0 conn(0x7f5e8c0a0ad0 0x7f5e8c0a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): get_auth_request con 0x7f5e8c0a0ad0 auth_method 0 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:12.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_done global_id 55734 payload 274 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 1 -- 172.21.15.47:0/4131078305 >> v2:172.21.15.179:3300/0 conn(0x7f5e8c0a0ad0 msgr2=0x7f5e8c0a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/4131078305 >> v2:172.21.15.179:3300/0 conn(0x7f5e8c0a0ad0 0x7f5e8c0a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.175+0000 7f5e89d9b700 1 -- 172.21.15.47:0/4131078305 >> v1:172.21.15.179:6789/0 conn(0x7f5e8c0a31c0 legacy=0x7f5e8c13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e89d9b700 10 monclient: _finish_hunting 0 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e89d9b700 1 monclient: found mon.noname-d 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e89d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e89d9b700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e89d9b700 1 -- 172.21.15.47:0/4131078305 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c09ddb0 con 0x7f5e8c0a2800 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/4131078305 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 0x7f5e8c0a2c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098f0 tx=0x7f5e80009c00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7457a34279e3824 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 1 -- 172.21.15.47:0/4131078305 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e80011aa0 con 0x7f5e8c0a2800 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: dump: 2024-04-20T02:36:12.186 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: _finish_auth 0 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: _check_auth_tickets 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 1 -- 172.21.15.47:0/4131078305 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80011c00 con 0x7f5e8c0a2800 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 5 monclient: authenticate success, global_id 55734 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 10 monclient: get_monmap_and_config success 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 1 -- 172.21.15.47:0/4131078305 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e80011d70 con 0x7f5e8c0a2800 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 4 set_mon_vals no callback set 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e8959a700 10 monclient: dump: 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.188 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 10 monclient: shutdown 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:36:12.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 1 -- 172.21.15.47:0/4131078305 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 msgr2=0x7f5e8c0a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.176+0000 7f5e90877700 1 --2- 172.21.15.47:0/4131078305 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 0x7f5e8c0a2c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098f0 tx=0x7f5e80009c00 comp rx=0 tx=0).stop 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 1 -- 172.21.15.47:0/4131078305 shutdown_connections 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 1 --2- 172.21.15.47:0/4131078305 >> v2:172.21.15.179:3300/0 conn(0x7f5e8c0a0ad0 0x7f5e8c0a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 1 --2- 172.21.15.47:0/4131078305 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f5e8c0a2800 0x7f5e8c0a2c80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 1 -- 172.21.15.47:0/4131078305 >> 172.21.15.47:0/4131078305 conn(0x7f5e8c0a7420 msgr2=0x7f5e8c0a38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 1 -- 172.21.15.47:0/4131078305 shutdown_connections 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 1 -- 172.21.15.47:0/4131078305 wait complete. 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 10 monclient: build_initial_monmap 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.177+0000 7f5e90877700 10 monclient: monmap: 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:36:12.190 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 librados: starting msgr at 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 librados: starting objecter 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 Processor -- start 2024-04-20T02:36:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 -- start start 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 librados: setting wanted keys 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 librados: calling monclient init 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 10 monclient: init 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 10 monclient: _reopen_session rank -1 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 10 monclient: _add_conns ranks=[1,4,8,2,7,3,6,5,0] 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 0x7f5e8c1a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 10 monclient(hunting): picked mon.noname-f con 0x7f5e8c0a0ad0 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.178+0000 7f5e90877700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 0x7f5e8c1a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): picked mon.noname-c con 0x7f5e8c0a2800 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5e8c0a31c0 0x7f5e8c1a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): picked mon.noname-e con 0x7f5e8c0a31c0 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1a8e90 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0720c0 con 0x7f5e8c0a2800 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1aa840 con 0x7f5e8c0a31c0 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): _renew_subs 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e90877700 10 monclient(hunting): authenticate will time out at 2851.579055s 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 0x7f5e8c1a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 0x7f5e8c1a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5e8c0a31c0 0x7f5e8c1a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 0x7f5e8c1a6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.47:45104/0 (socket says 172.21.15.47:45104) 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 0x7f5e8c1a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:39258/0 (socket says 172.21.15.47:39258) 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 -- 172.21.15.47:0/1278124036 learned_addr learned my addr 172.21.15.47:0/1278124036 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5e8c0a31c0 0x7f5e8c1a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:46940/0 (socket says 172.21.15.47:46940) 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): get_auth_request con 0x7f5e8c0a0ad0 auth_method 0 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:12.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): get_auth_request con 0x7f5e8c0a31c0 auth_method 0 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): get_auth_request con 0x7f5e8c0a2800 auth_method 0 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:12.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient(hunting): handle_auth_done global_id 56206 payload 824 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 -- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5e8c0a31c0 msgr2=0x7f5e8c1a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5e8c0a31c0 0x7f5e8c1a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 -- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 msgr2=0x7f5e8c1a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 0x7f5e8c1a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient: _finish_hunting 0 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 monclient: found mon.noname-f 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.179:3300/0 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 -- 172.21.15.47:0/1278124036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e80009550 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 0x7f5e8c1a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e8a59c700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 0x7f5e8c1a6c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e9b0 tx=0x7f5e7c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e7affd700 10 client.?.objecter ms_handle_connect 0x7f5e8c0a0ad0 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e7affd700 10 client.?.objecter resend_mon_ops 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e7c009710 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.179+0000 7f5e7affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: dump: 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: _finish_auth 0 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: _check_auth_tickets 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 5 monclient: authenticate success, global_id 56206 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c00ee50 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:36:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e7c00ccc0 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e7affd700 10 monclient: dump: 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:12.198 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e88d99700 4 set_mon_vals no callback set 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 monclient: _renew_subs 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c09e510 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 client.56206.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 monclient: _renew_subs 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c1aa280 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 1 librados: init done 2024-04-20T02:36:12.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.180+0000 7f5e90877700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e90877700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.179:3300/0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c1aa5f0 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f5e7c00c850 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e6c03a2f0 0x7f5e6c03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(697..697 src has 600..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f5e7c04a5d0 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 10 client.56206.objecter ms_dispatch 0x7f5e8c001550 osd_map(697..697 src has 600..697) v4 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 3 client.56206.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 3 client.56206.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e7affd700 20 client.56206.objecter dump_active .. 0 homeless 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e6c03a2f0 0x7f5e6c03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.181+0000 7f5e89d9b700 10 monclient: get_auth_request con 0x7f5e6c03a2f0 auth_method 0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.182+0000 7f5e7affd700 10 client.56206.objecter ms_handle_connect 0x7f5e6c03a2f0 2024-04-20T02:36:12.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.182+0000 7f5e89d9b700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e6c03a2f0 0x7f5e6c03c7b0 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098f0 tx=0x7f5e80008030 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.186+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f5e7c01ada0 con 0x7f5e8c0a0ad0 2024-04-20T02:36:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.186+0000 7f5e7affd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:12.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.186+0000 7f5e7affd700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:36:12.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.399+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e8c09f6e0 con 0x7f5e6c03a2f0 2024-04-20T02:36:12.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.404+0000 7f5e7affd700 1 -- 172.21.15.47:0/1278124036 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+134655 (secure 0 0 0) 0x7f5e80011a20 con 0x7f5e6c03a2f0 2024-04-20T02:36:12.405 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.406+0000 7f5e90877700 10 librados: watch_flush enter 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.406+0000 7f5e90877700 10 librados: watch_flush exit 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.406+0000 7f5e90877700 20 client.56206.objecter shutdown clearing up homeless session... 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.406+0000 7f5e90877700 10 client.56206.objecter successfully canceled tick 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e6c03a2f0 msgr2=0x7f5e6c03c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e6c03a2f0 0x7f5e6c03c7b0 secure :-1 s=READY pgs=3426 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098f0 tx=0x7f5e80008030 comp rx=0 tx=0).stop 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 10 monclient: shutdown 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 msgr2=0x7f5e8c1a6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 0x7f5e8c1a6c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e9b0 tx=0x7f5e7c00ecc0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 shutdown_connections 2024-04-20T02:36:12.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f5e8c0a31c0 0x7f5e8c1a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f5e6c03a2f0 0x7f5e6c03c7b0 unknown :-1 s=CLOSED pgs=3426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8c0a0ad0 0x7f5e8c1a6c80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 --2- 172.21.15.47:0/1278124036 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e8c0a2800 0x7f5e8c1a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 >> 172.21.15.47:0/1278124036 conn(0x7f5e8c0a7420 msgr2=0x7f5e8c0a5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 shutdown_connections 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 -- 172.21.15.47:0/1278124036 wait complete. 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:12.407+0000 7f5e90877700 1 librados: shutdown 2024-04-20T02:36:12.408 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:36:12.419 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1305,"stamp":"2024-04-20T02:36:10.620411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":106,"num_write":166,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470288,"kb_used_data":38368,"kb_used_omap":9,"kb_used_meta":431862,"kb_avail":565760752,"statfs":{"total":579820584960,"available":579339010048,"internally_reserved":0,"allocated":39288832,"data_stored":20149014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":442227367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002845"},"pg_stats":[{"pgid":"5.1c","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.960325+0000","last_change":"2024-04-20T02:36:00.960429+0000","last_active":"2024-04-20T02:36:00.960325+0000","last_peered":"2024-04-20T02:36:00.960325+0000","last_clean":"2024-04-20T02:36:00.960325+0000","last_became_active":"2024-04-20T02:16:44.179944+0000","last_became_peered":"2024-04-20T02:16:44.179944+0000","last_unstale":"2024-04-20T02:36:00.960325+0000","last_undegraded":"2024-04-20T02:36:00.960325+0000","last_fullsized":"2024-04-20T02:36:00.960325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.960301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.960301+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.960301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:24.813961+0000","scrub_duration":0.00198985,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":697,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:05.042460+0000","last_change":"2024-04-20T02:36:05.042576+0000","last_active":"2024-04-20T02:36:05.042460+0000","last_peered":"2024-04-20T02:36:05.042460+0000","last_clean":"2024-04-20T02:36:05.042460+0000","last_became_active":"2024-04-20T02:16:44.130319+0000","last_became_peered":"2024-04-20T02:16:44.130319+0000","last_unstale":"2024-04-20T02:36:05.042460+0000","last_undegraded":"2024-04-20T02:36:05.042460+0000","last_fullsized":"2024-04-20T02:36:05.042460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:05.042434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:05.042434+0000","last_clean_scrub_stamp":"2024-04-20T02:36:05.042434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:38.169448+0000","scrub_duration":0.0035587029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:06.484469+0000","last_change":"2024-04-20T02:36:06.484574+0000","last_active":"2024-04-20T02:36:06.484469+0000","last_peered":"2024-04-20T02:36:06.484469+0000","last_clean":"2024-04-20T02:36:06.484469+0000","last_became_active":"2024-04-20T02:16:44.117655+0000","last_became_peered":"2024-04-20T02:16:44.117655+0000","last_unstale":"2024-04-20T02:36:06.484469+0000","last_undegraded":"2024-04-20T02:36:06.484469+0000","last_fullsized":"2024-04-20T02:36:06.484469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:06.484444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:06.484444+0000","last_clean_scrub_stamp":"2024-04-20T02:36:06.484444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:08.623009+0000","scrub_duration":0.0021908510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":680,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:24.384332+0000","last_change":"2024-04-20T02:16:44.113440+0000","last_active":"2024-04-20T02:35:24.384332+0000","last_peered":"2024-04-20T02:35:24.384332+0000","last_clean":"2024-04-20T02:35:24.384332+0000","last_became_active":"2024-04-20T02:16:44.111746+0000","last_became_peered":"2024-04-20T02:16:44.111746+0000","last_unstale":"2024-04-20T02:35:24.384332+0000","last_undegraded":"2024-04-20T02:35:24.384332+0000","last_fullsized":"2024-04-20T02:35:24.384332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:16:43.015396+0000","last_clean_scrub_stamp":"2024-04-20T02:16:43.015396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.969487+0000","last_change":"2024-04-20T02:35:53.969583+0000","last_active":"2024-04-20T02:35:53.969487+0000","last_peered":"2024-04-20T02:35:53.969487+0000","last_clean":"2024-04-20T02:35:53.969487+0000","last_became_active":"2024-04-20T02:16:44.117331+0000","last_became_peered":"2024-04-20T02:16:44.117331+0000","last_unstale":"2024-04-20T02:35:53.969487+0000","last_undegraded":"2024-04-20T02:35:53.969487+0000","last_fullsized":"2024-04-20T02:35:53.969487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.969460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.969460+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.969460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:57:18.692594+0000","scrub_duration":0.0014488369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:57.108910+0000","last_change":"2024-04-20T02:35:57.109019+0000","last_active":"2024-04-20T02:35:57.108910+0000","last_peered":"2024-04-20T02:35:57.108910+0000","last_clean":"2024-04-20T02:35:57.108910+0000","last_became_active":"2024-04-20T02:16:44.123156+0000","last_became_peered":"2024-04-20T02:16:44.123156+0000","last_unstale":"2024-04-20T02:35:57.108910+0000","last_undegraded":"2024-04-20T02:35:57.108910+0000","last_fullsized":"2024-04-20T02:35:57.108910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:57.108884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:57.108884+0000","last_clean_scrub_stamp":"2024-04-20T02:35:57.108884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:50:51.598896+0000","scrub_duration":0.0029345410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.734257+0000","last_change":"2024-04-20T02:36:00.734368+0000","last_active":"2024-04-20T02:36:00.734257+0000","last_peered":"2024-04-20T02:36:00.734257+0000","last_clean":"2024-04-20T02:36:00.734257+0000","last_became_active":"2024-04-20T02:16:44.117507+0000","last_became_peered":"2024-04-20T02:16:44.117507+0000","last_unstale":"2024-04-20T02:36:00.734257+0000","last_undegraded":"2024-04-20T02:36:00.734257+0000","last_fullsized":"2024-04-20T02:36:00.734257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.734217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.734217+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.734217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:31:28.784264+0000","scrub_duration":0.001716686,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.189355+0000","last_change":"2024-04-20T02:36:00.189355+0000","last_active":"2024-04-20T02:36:00.189355+0000","last_peered":"2024-04-20T02:36:00.189355+0000","last_clean":"2024-04-20T02:36:00.189355+0000","last_became_active":"2024-04-20T02:16:44.228394+0000","last_became_peered":"2024-04-20T02:16:44.228394+0000","last_unstale":"2024-04-20T02:36:00.189355+0000","last_undegraded":"2024-04-20T02:36:00.189355+0000","last_fullsized":"2024-04-20T02:36:00.189355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.189208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.189208+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.189208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:05.467264+0000","scrub_duration":0.0020655449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:59.768613+0000","last_change":"2024-04-20T02:35:59.768751+0000","last_active":"2024-04-20T02:35:59.768613+0000","last_peered":"2024-04-20T02:35:59.768613+0000","last_clean":"2024-04-20T02:35:59.768613+0000","last_became_active":"2024-04-20T02:16:44.170848+0000","last_became_peered":"2024-04-20T02:16:44.170848+0000","last_unstale":"2024-04-20T02:35:59.768613+0000","last_undegraded":"2024-04-20T02:35:59.768613+0000","last_fullsized":"2024-04-20T02:35:59.768613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:59.768589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:59.768589+0000","last_clean_scrub_stamp":"2024-04-20T02:35:59.768589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:45:43.603775+0000","scrub_duration":0.0021545359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:04.089640+0000","last_change":"2024-04-20T02:36:04.089640+0000","last_active":"2024-04-20T02:36:04.089640+0000","last_peered":"2024-04-20T02:36:04.089640+0000","last_clean":"2024-04-20T02:36:04.089640+0000","last_became_active":"2024-04-20T02:16:44.139769+0000","last_became_peered":"2024-04-20T02:16:44.139769+0000","last_unstale":"2024-04-20T02:36:04.089640+0000","last_undegraded":"2024-04-20T02:36:04.089640+0000","last_fullsized":"2024-04-20T02:36:04.089640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:04.089517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:04.089517+0000","last_clean_scrub_stamp":"2024-04-20T02:36:04.089517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:44:49.752468+0000","scrub_duration":0.003217433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:03.509696+0000","last_change":"2024-04-20T02:36:03.509696+0000","last_active":"2024-04-20T02:36:03.509696+0000","last_peered":"2024-04-20T02:36:03.509696+0000","last_clean":"2024-04-20T02:36:03.509696+0000","last_became_active":"2024-04-20T02:16:44.187150+0000","last_became_peered":"2024-04-20T02:16:44.187150+0000","last_unstale":"2024-04-20T02:36:03.509696+0000","last_undegraded":"2024-04-20T02:36:03.509696+0000","last_fullsized":"2024-04-20T02:36:03.509696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:03.509520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:03.509520+0000","last_clean_scrub_stamp":"2024-04-20T02:36:03.509520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:00.546275+0000","scrub_duration":0.0026646439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:02.505119+0000","last_change":"2024-04-20T02:36:02.505228+0000","last_active":"2024-04-20T02:36:02.505119+0000","last_peered":"2024-04-20T02:36:02.505119+0000","last_clean":"2024-04-20T02:36:02.505119+0000","last_became_active":"2024-04-20T02:16:44.167097+0000","last_became_peered":"2024-04-20T02:16:44.167097+0000","last_unstale":"2024-04-20T02:36:02.505119+0000","last_undegraded":"2024-04-20T02:36:02.505119+0000","last_fullsized":"2024-04-20T02:36:02.505119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:02.505084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:02.505084+0000","last_clean_scrub_stamp":"2024-04-20T02:36:02.505084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:31:23.979872+0000","scrub_duration":0.0027940209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:03.561643+0000","last_change":"2024-04-20T02:36:03.561731+0000","last_active":"2024-04-20T02:36:03.561643+0000","last_peered":"2024-04-20T02:36:03.561643+0000","last_clean":"2024-04-20T02:36:03.561643+0000","last_became_active":"2024-04-20T02:16:44.168588+0000","last_became_peered":"2024-04-20T02:16:44.168588+0000","last_unstale":"2024-04-20T02:36:03.561643+0000","last_undegraded":"2024-04-20T02:36:03.561643+0000","last_fullsized":"2024-04-20T02:36:03.561643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:03.561623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:03.561623+0000","last_clean_scrub_stamp":"2024-04-20T02:36:03.561623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:43.448156+0000","scrub_duration":0.0053437399999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:04.511525+0000","last_change":"2024-04-20T02:36:04.511623+0000","last_active":"2024-04-20T02:36:04.511525+0000","last_peered":"2024-04-20T02:36:04.511525+0000","last_clean":"2024-04-20T02:36:04.511525+0000","last_became_active":"2024-04-20T02:16:44.103711+0000","last_became_peered":"2024-04-20T02:16:44.103711+0000","last_unstale":"2024-04-20T02:36:04.511525+0000","last_undegraded":"2024-04-20T02:36:04.511525+0000","last_fullsized":"2024-04-20T02:36:04.511525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:04.511501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:04.511501+0000","last_clean_scrub_stamp":"2024-04-20T02:36:04.511501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:51:37.343223+0000","scrub_duration":0.0031208719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.753317+0000","last_change":"2024-04-20T02:35:53.753399+0000","last_active":"2024-04-20T02:35:53.753317+0000","last_peered":"2024-04-20T02:35:53.753317+0000","last_clean":"2024-04-20T02:35:53.753317+0000","last_became_active":"2024-04-20T02:16:44.207027+0000","last_became_peered":"2024-04-20T02:16:44.207027+0000","last_unstale":"2024-04-20T02:35:53.753317+0000","last_undegraded":"2024-04-20T02:35:53.753317+0000","last_fullsized":"2024-04-20T02:35:53.753317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.753295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.753295+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.753295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:45:53.556524+0000","scrub_duration":0.0019783930000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:55.674083+0000","last_change":"2024-04-20T02:35:55.674192+0000","last_active":"2024-04-20T02:35:55.674083+0000","last_peered":"2024-04-20T02:35:55.674083+0000","last_clean":"2024-04-20T02:35:55.674083+0000","last_became_active":"2024-04-20T02:16:44.209154+0000","last_became_peered":"2024-04-20T02:16:44.209154+0000","last_unstale":"2024-04-20T02:35:55.674083+0000","last_undegraded":"2024-04-20T02:35:55.674083+0000","last_fullsized":"2024-04-20T02:35:55.674083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:55.674057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:55.674057+0000","last_clean_scrub_stamp":"2024-04-20T02:35:55.674057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:19.577135+0000","scrub_duration":0.003317946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:01.565187+0000","last_change":"2024-04-20T02:36:01.565299+0000","last_active":"2024-04-20T02:36:01.565187+0000","last_peered":"2024-04-20T02:36:01.565187+0000","last_clean":"2024-04-20T02:36:01.565187+0000","last_became_active":"2024-04-20T02:16:44.116619+0000","last_became_peered":"2024-04-20T02:16:44.116619+0000","last_unstale":"2024-04-20T02:36:01.565187+0000","last_undegraded":"2024-04-20T02:36:01.565187+0000","last_fullsized":"2024-04-20T02:36:01.565187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:01.565162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:01.565162+0000","last_clean_scrub_stamp":"2024-04-20T02:36:01.565162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:37:46.426151+0000","scrub_duration":0.001696677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:55.490010+0000","last_change":"2024-04-20T02:35:55.490130+0000","last_active":"2024-04-20T02:35:55.490010+0000","last_peered":"2024-04-20T02:35:55.490010+0000","last_clean":"2024-04-20T02:35:55.490010+0000","last_became_active":"2024-04-20T02:16:44.168518+0000","last_became_peered":"2024-04-20T02:16:44.168518+0000","last_unstale":"2024-04-20T02:35:55.490010+0000","last_undegraded":"2024-04-20T02:35:55.490010+0000","last_fullsized":"2024-04-20T02:35:55.490010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:55.489984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:55.489984+0000","last_clean_scrub_stamp":"2024-04-20T02:35:55.489984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:09:35.155853+0000","scrub_duration":0.0034148659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:04.017845+0000","last_change":"2024-04-20T02:36:04.017966+0000","last_active":"2024-04-20T02:36:04.017845+0000","last_peered":"2024-04-20T02:36:04.017845+0000","last_clean":"2024-04-20T02:36:04.017845+0000","last_became_active":"2024-04-20T02:16:44.155758+0000","last_became_peered":"2024-04-20T02:16:44.155758+0000","last_unstale":"2024-04-20T02:36:04.017845+0000","last_undegraded":"2024-04-20T02:36:04.017845+0000","last_fullsized":"2024-04-20T02:36:04.017845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:04.017819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:04.017819+0000","last_clean_scrub_stamp":"2024-04-20T02:36:04.017819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:42.299656+0000","scrub_duration":0.001505057,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":699,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:54.622426+0000","last_change":"2024-04-20T02:35:54.622527+0000","last_active":"2024-04-20T02:35:54.622426+0000","last_peered":"2024-04-20T02:35:54.622426+0000","last_clean":"2024-04-20T02:35:54.622426+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:35:54.622426+0000","last_undegraded":"2024-04-20T02:35:54.622426+0000","last_fullsized":"2024-04-20T02:35:54.622426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:54.622402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:54.622402+0000","last_clean_scrub_stamp":"2024-04-20T02:35:54.622402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:14.700754+0000","scrub_duration":0.001432467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":709,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:56.142200+0000","last_change":"2024-04-20T02:35:56.142296+0000","last_active":"2024-04-20T02:35:56.142200+0000","last_peered":"2024-04-20T02:35:56.142200+0000","last_clean":"2024-04-20T02:35:56.142200+0000","last_became_active":"2024-04-20T02:16:44.190566+0000","last_became_peered":"2024-04-20T02:16:44.190566+0000","last_unstale":"2024-04-20T02:35:56.142200+0000","last_undegraded":"2024-04-20T02:35:56.142200+0000","last_fullsized":"2024-04-20T02:35:56.142200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:56.142174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:56.142174+0000","last_clean_scrub_stamp":"2024-04-20T02:35:56.142174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:03:44.371018+0000","scrub_duration":0.002113073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:02.531633+0000","last_change":"2024-04-20T02:36:02.531734+0000","last_active":"2024-04-20T02:36:02.531633+0000","last_peered":"2024-04-20T02:36:02.531633+0000","last_clean":"2024-04-20T02:36:02.531633+0000","last_became_active":"2024-04-20T02:16:44.169063+0000","last_became_peered":"2024-04-20T02:16:44.169063+0000","last_unstale":"2024-04-20T02:36:02.531633+0000","last_undegraded":"2024-04-20T02:36:02.531633+0000","last_fullsized":"2024-04-20T02:36:02.531633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:02.531608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:02.531608+0000","last_clean_scrub_stamp":"2024-04-20T02:36:02.531608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:11:24.299164+0000","scrub_duration":0.001682682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.077353+0000","last_change":"2024-04-20T02:35:53.077457+0000","last_active":"2024-04-20T02:35:53.077353+0000","last_peered":"2024-04-20T02:35:53.077353+0000","last_clean":"2024-04-20T02:35:53.077353+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:35:53.077353+0000","last_undegraded":"2024-04-20T02:35:53.077353+0000","last_fullsized":"2024-04-20T02:35:53.077353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.077326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.077326+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.077326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:21:54.436030+0000","scrub_duration":0.002076647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.178195+0000","last_change":"2024-04-20T02:36:00.178287+0000","last_active":"2024-04-20T02:36:00.178195+0000","last_peered":"2024-04-20T02:36:00.178195+0000","last_clean":"2024-04-20T02:36:00.178195+0000","last_became_active":"2024-04-20T02:16:44.150893+0000","last_became_peered":"2024-04-20T02:16:44.150893+0000","last_unstale":"2024-04-20T02:36:00.178195+0000","last_undegraded":"2024-04-20T02:36:00.178195+0000","last_fullsized":"2024-04-20T02:36:00.178195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.178171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.178171+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.178171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:44:40.321043+0000","scrub_duration":0.0020325809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":699,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:50.635789+0000","last_change":"2024-04-20T02:35:50.635896+0000","last_active":"2024-04-20T02:35:50.635789+0000","last_peered":"2024-04-20T02:35:50.635789+0000","last_clean":"2024-04-20T02:35:50.635789+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:35:50.635789+0000","last_undegraded":"2024-04-20T02:35:50.635789+0000","last_fullsized":"2024-04-20T02:35:50.635789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:50.635763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:50.635763+0000","last_clean_scrub_stamp":"2024-04-20T02:35:50.635763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:38:26.315486+0000","scrub_duration":0.001359065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:52.510684+0000","last_change":"2024-04-20T02:35:52.510785+0000","last_active":"2024-04-20T02:35:52.510684+0000","last_peered":"2024-04-20T02:35:52.510684+0000","last_clean":"2024-04-20T02:35:52.510684+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:35:52.510684+0000","last_undegraded":"2024-04-20T02:35:52.510684+0000","last_fullsized":"2024-04-20T02:35:52.510684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:52.510661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:52.510661+0000","last_clean_scrub_stamp":"2024-04-20T02:35:52.510661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:40:43.938245+0000","scrub_duration":0.0014765329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"697'119","reported_seq":848,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:50.535148+0000","last_change":"2024-04-20T02:35:50.535232+0000","last_active":"2024-04-20T02:35:50.535148+0000","last_peered":"2024-04-20T02:35:50.535148+0000","last_clean":"2024-04-20T02:35:50.535148+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:35:50.535148+0000","last_undegraded":"2024-04-20T02:35:50.535148+0000","last_fullsized":"2024-04-20T02:35:50.535148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"697'119","last_scrub_stamp":"2024-04-20T02:35:50.535120+0000","last_deep_scrub":"697'119","last_deep_scrub_stamp":"2024-04-20T02:35:50.535120+0000","last_clean_scrub_stamp":"2024-04-20T02:35:50.535120+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:40.506051+0000","scrub_duration":0.006522992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:55.145269+0000","last_change":"2024-04-20T02:35:55.145368+0000","last_active":"2024-04-20T02:35:55.145269+0000","last_peered":"2024-04-20T02:35:55.145269+0000","last_clean":"2024-04-20T02:35:55.145269+0000","last_became_active":"2024-04-20T02:16:44.175527+0000","last_became_peered":"2024-04-20T02:16:44.175527+0000","last_unstale":"2024-04-20T02:35:55.145269+0000","last_undegraded":"2024-04-20T02:35:55.145269+0000","last_fullsized":"2024-04-20T02:35:55.145269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:55.145239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:55.145239+0000","last_clean_scrub_stamp":"2024-04-20T02:35:55.145239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:03:53.365058+0000","scrub_duration":0.0020752169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:59.174205+0000","last_change":"2024-04-20T02:35:59.174331+0000","last_active":"2024-04-20T02:35:59.174205+0000","last_peered":"2024-04-20T02:35:59.174205+0000","last_clean":"2024-04-20T02:35:59.174205+0000","last_became_active":"2024-04-20T02:16:44.209341+0000","last_became_peered":"2024-04-20T02:16:44.209341+0000","last_unstale":"2024-04-20T02:35:59.174205+0000","last_undegraded":"2024-04-20T02:35:59.174205+0000","last_fullsized":"2024-04-20T02:35:59.174205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:59.174177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:59.174177+0000","last_clean_scrub_stamp":"2024-04-20T02:35:59.174177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:19:39.964602+0000","scrub_duration":0.003510811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":701,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.011645+0000","last_change":"2024-04-20T02:35:53.011781+0000","last_active":"2024-04-20T02:35:53.011645+0000","last_peered":"2024-04-20T02:35:53.011645+0000","last_clean":"2024-04-20T02:35:53.011645+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:35:53.011645+0000","last_undegraded":"2024-04-20T02:35:53.011645+0000","last_fullsized":"2024-04-20T02:35:53.011645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.011618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.011618+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.011618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:57:15.737315+0000","scrub_duration":0.003340675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:51.506449+0000","last_change":"2024-04-20T02:35:51.506531+0000","last_active":"2024-04-20T02:35:51.506449+0000","last_peered":"2024-04-20T02:35:51.506449+0000","last_clean":"2024-04-20T02:35:51.506449+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:35:51.506449+0000","last_undegraded":"2024-04-20T02:35:51.506449+0000","last_fullsized":"2024-04-20T02:35:51.506449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:51.506427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:51.506427+0000","last_clean_scrub_stamp":"2024-04-20T02:35:51.506427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:47:48.096698+0000","scrub_duration":0.0017070150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:58.162257+0000","last_change":"2024-04-20T02:35:58.162351+0000","last_active":"2024-04-20T02:35:58.162257+0000","last_peered":"2024-04-20T02:35:58.162257+0000","last_clean":"2024-04-20T02:35:58.162257+0000","last_became_active":"2024-04-20T02:16:44.150765+0000","last_became_peered":"2024-04-20T02:16:44.150765+0000","last_unstale":"2024-04-20T02:35:58.162257+0000","last_undegraded":"2024-04-20T02:35:58.162257+0000","last_fullsized":"2024-04-20T02:35:58.162257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:58.162231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:58.162231+0000","last_clean_scrub_stamp":"2024-04-20T02:35:58.162231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:32:51.127508+0000","scrub_duration":0.0029800709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":712,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.121651+0000","last_change":"2024-04-20T02:35:53.121766+0000","last_active":"2024-04-20T02:35:53.121651+0000","last_peered":"2024-04-20T02:35:53.121651+0000","last_clean":"2024-04-20T02:35:53.121651+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:35:53.121651+0000","last_undegraded":"2024-04-20T02:35:53.121651+0000","last_fullsized":"2024-04-20T02:35:53.121651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.121628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.121628+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.121628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:03:34.484457+0000","scrub_duration":0.0018377700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"697'8","reported_seq":706,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:56.512265+0000","last_change":"2024-04-20T02:35:56.512424+0000","last_active":"2024-04-20T02:35:56.512265+0000","last_peered":"2024-04-20T02:35:56.512265+0000","last_clean":"2024-04-20T02:35:56.512265+0000","last_became_active":"2024-04-20T02:16:44.105229+0000","last_became_peered":"2024-04-20T02:16:44.105229+0000","last_unstale":"2024-04-20T02:35:56.512265+0000","last_undegraded":"2024-04-20T02:35:56.512265+0000","last_fullsized":"2024-04-20T02:35:56.512265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"697'8","last_scrub_stamp":"2024-04-20T02:35:56.512213+0000","last_deep_scrub":"697'8","last_deep_scrub_stamp":"2024-04-20T02:35:56.512213+0000","last_clean_scrub_stamp":"2024-04-20T02:35:56.512213+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:13:41.636081+0000","scrub_duration":0.0059889339999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:54.490031+0000","last_change":"2024-04-20T02:35:54.490164+0000","last_active":"2024-04-20T02:35:54.490031+0000","last_peered":"2024-04-20T02:35:54.490031+0000","last_clean":"2024-04-20T02:35:54.490031+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:35:54.490031+0000","last_undegraded":"2024-04-20T02:35:54.490031+0000","last_fullsized":"2024-04-20T02:35:54.490031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:54.490001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:54.490001+0000","last_clean_scrub_stamp":"2024-04-20T02:35:54.490001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:56:45.665864+0000","scrub_duration":0.001558888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:01.702298+0000","last_change":"2024-04-20T02:36:01.702402+0000","last_active":"2024-04-20T02:36:01.702298+0000","last_peered":"2024-04-20T02:36:01.702298+0000","last_clean":"2024-04-20T02:36:01.702298+0000","last_became_active":"2024-04-20T02:16:44.108254+0000","last_became_peered":"2024-04-20T02:16:44.108254+0000","last_unstale":"2024-04-20T02:36:01.702298+0000","last_undegraded":"2024-04-20T02:36:01.702298+0000","last_fullsized":"2024-04-20T02:36:01.702298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:01.702274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:01.702274+0000","last_clean_scrub_stamp":"2024-04-20T02:36:01.702274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:10.438468+0000","scrub_duration":0.001434318,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":707,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:07.169419+0000","last_change":"2024-04-20T02:36:07.169520+0000","last_active":"2024-04-20T02:36:07.169419+0000","last_peered":"2024-04-20T02:36:07.169419+0000","last_clean":"2024-04-20T02:36:07.169419+0000","last_became_active":"2024-04-20T02:16:44.142929+0000","last_became_peered":"2024-04-20T02:16:44.142929+0000","last_unstale":"2024-04-20T02:36:07.169419+0000","last_undegraded":"2024-04-20T02:36:07.169419+0000","last_fullsized":"2024-04-20T02:36:07.169419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:07.169393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:07.169393+0000","last_clean_scrub_stamp":"2024-04-20T02:36:07.169393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:27:39.074724+0000","scrub_duration":0.0017524210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":690,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:08.466762+0000","last_change":"2024-04-20T02:36:08.466881+0000","last_active":"2024-04-20T02:36:08.466762+0000","last_peered":"2024-04-20T02:36:08.466762+0000","last_clean":"2024-04-20T02:36:08.466762+0000","last_became_active":"2024-04-20T02:16:44.159975+0000","last_became_peered":"2024-04-20T02:16:44.159975+0000","last_unstale":"2024-04-20T02:36:08.466762+0000","last_undegraded":"2024-04-20T02:36:08.466762+0000","last_fullsized":"2024-04-20T02:36:08.466762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:08.466735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:08.466735+0000","last_clean_scrub_stamp":"2024-04-20T02:36:08.466735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:03:30.274365+0000","scrub_duration":0.0020505950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:57.934692+0000","last_change":"2024-04-20T02:35:57.934803+0000","last_active":"2024-04-20T02:35:57.934692+0000","last_peered":"2024-04-20T02:35:57.934692+0000","last_clean":"2024-04-20T02:35:57.934692+0000","last_became_active":"2024-04-20T02:16:44.131124+0000","last_became_peered":"2024-04-20T02:16:44.131124+0000","last_unstale":"2024-04-20T02:35:57.934692+0000","last_undegraded":"2024-04-20T02:35:57.934692+0000","last_fullsized":"2024-04-20T02:35:57.934692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:57.934664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:57.934664+0000","last_clean_scrub_stamp":"2024-04-20T02:35:57.934664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:22:14.399028+0000","scrub_duration":0.0037138090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:05.531558+0000","last_change":"2024-04-20T02:36:05.531661+0000","last_active":"2024-04-20T02:36:05.531558+0000","last_peered":"2024-04-20T02:36:05.531558+0000","last_clean":"2024-04-20T02:36:05.531558+0000","last_became_active":"2024-04-20T02:16:44.141808+0000","last_became_peered":"2024-04-20T02:16:44.141808+0000","last_unstale":"2024-04-20T02:36:05.531558+0000","last_undegraded":"2024-04-20T02:36:05.531558+0000","last_fullsized":"2024-04-20T02:36:05.531558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:05.531530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:05.531530+0000","last_clean_scrub_stamp":"2024-04-20T02:36:05.531530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:40.560229+0000","scrub_duration":0.001370486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:02.131200+0000","last_change":"2024-04-20T02:36:02.131200+0000","last_active":"2024-04-20T02:36:02.131200+0000","last_peered":"2024-04-20T02:36:02.131200+0000","last_clean":"2024-04-20T02:36:02.131200+0000","last_became_active":"2024-04-20T02:16:44.099559+0000","last_became_peered":"2024-04-20T02:16:44.099559+0000","last_unstale":"2024-04-20T02:36:02.131200+0000","last_undegraded":"2024-04-20T02:36:02.131200+0000","last_fullsized":"2024-04-20T02:36:02.131200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:02.131080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:02.131080+0000","last_clean_scrub_stamp":"2024-04-20T02:36:02.131080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:05:14.428501+0000","scrub_duration":0.0026259159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607802,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77528,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294312,"statfs":{"total":96636764160,"available":96557375488,"internally_reserved":0,"allocated":6373376,"data_stored":3181863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.314,"15min":0.22900000000000001},"max":{"1min":0.68000000000000005,"5min":0.80000000000000004,"15min":2.9460000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.311,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.73699999999999999,"15min":2.8570000000000002},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.30399999999999999},"max":{"1min":0.64500000000000002,"5min":0.76900000000000002,"15min":2.391},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.79300000000000004,"15min":1.292},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.30099999999999999},"max":{"1min":0.70399999999999996,"5min":0.747,"15min":1.1899999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68999999999999995,"5min":0.81599999999999995,"15min":1.2450000000000001},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":1.3029999999999999},"last":0.307},{"interface":"front","average":{"1min":0.434,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.32400000000000001,"5min":0.26400000000000001,"15min":0.25900000000000001},"max":{"1min":0.60299999999999998,"5min":0.61799999999999999,"15min":0.65400000000000003},"last":0.27700000000000002}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.40999999999999998},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.45100000000000001,"15min":0.44600000000000001},"min":{"1min":0.308,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":0.71299999999999997,"15min":1.534},"last":0.32900000000000001}]}]},{"osd":3,"up_from":12,"seq":51539607801,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77524,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294316,"statfs":{"total":96636764160,"available":96557379584,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.22600000000000001},"max":{"1min":0.748,"5min":1.0880000000000001,"15min":1.4550000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":1.103,"15min":1.397},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.439,"5min":0.307,"15min":0.26000000000000001},"max":{"1min":0.70199999999999996,"5min":1.03,"15min":1.5409999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.58299999999999996},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.318},"max":{"1min":0.72499999999999998,"5min":1.073,"15min":1.5880000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57799999999999996,"15min":0.55800000000000005},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":1.0600000000000001,"15min":1.5589999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70899999999999996,"5min":1.1259999999999999,"15min":2.2599999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.247,"15min":0.24299999999999999},"max":{"1min":0.61799999999999999,"5min":0.64700000000000002,"15min":1.0469999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.629,"5min":0.71099999999999997,"15min":1.335},"last":0.308}]},{"osd":5,"last update":"Sat Apr 20 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.23100000000000001},"max":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.90500000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.29899999999999999,"15min":0.27900000000000003},"max":{"1min":0.67900000000000005,"5min":0.72299999999999998,"15min":0.95899999999999996},"last":0.34399999999999997}]}]},{"osd":2,"up_from":12,"seq":51539607801,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69332,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94302508,"statfs":{"total":96636764160,"available":96565768192,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.24099999999999999,"15min":0.23799999999999999},"max":{"1min":0.48899999999999999,"5min":0.52500000000000002,"15min":0.88800000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38600000000000001,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.48399999999999999,"5min":0.67600000000000005,"15min":0.96899999999999997},"last":0.443}]},{"osd":1,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.28899999999999998,"5min":0.28699999999999998,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.58099999999999996,"15min":0.97799999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.33200000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.45700000000000002,"5min":0.61799999999999999,"15min":1.089},"last":0.39900000000000002}]},{"osd":3,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.57399999999999995},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.62,"5min":0.77000000000000002,"15min":3.823},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.34499999999999997},"max":{"1min":1.226,"5min":1.226,"15min":3.1440000000000001},"last":0.55800000000000005}]},{"osd":4,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56399999999999995,"15min":0.58699999999999997},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.38200000000000001},"max":{"1min":0.67100000000000004,"5min":0.81200000000000006,"15min":1.1519999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.52000000000000002,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.66500000000000004,"5min":0.85499999999999998,"15min":1.2270000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Apr 20 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.48399999999999999,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.1919999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.51500000000000001,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":0.74199999999999999,"5min":0.92000000000000004,"15min":1.2070000000000001},"last":0.628}]}]},{"osd":1,"up_from":12,"seq":51539607804,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82144,"kb_used_data":6744,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289696,"statfs":{"total":96636764160,"available":96552648704,"internally_reserved":0,"allocated":6905856,"data_stored":3710791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.24299999999999999,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.48699999999999999,"15min":1.8220000000000001},"last":0.312},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.247},"max":{"1min":0.51100000000000001,"5min":0.73499999999999999,"15min":0.92700000000000005},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36699999999999999,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.26400000000000001},"max":{"1min":0.48699999999999999,"5min":0.55400000000000005,"15min":2.649},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.26500000000000001},"max":{"1min":0.57999999999999996,"5min":0.65900000000000003,"15min":2.5499999999999998},"last":0.32900000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.98699999999999999,"15min":1.034},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.68500000000000005,"5min":0.90100000000000002,"15min":1.7450000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64600000000000002,"15min":0.627},"min":{"1min":0.436,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.876,"5min":2.6659999999999999,"15min":2.6659999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Sat Apr 20 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.628,"15min":0.61099999999999999},"min":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.35999999999999999},"max":{"1min":0.77000000000000002,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.48499999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":1.282},"last":0.69899999999999995}]}]},{"osd":5,"up_from":11,"seq":47244640504,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82140,"kb_used_data":6740,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289700,"statfs":{"total":96636764160,"available":96552652800,"internally_reserved":0,"allocated":6901760,"data_stored":3710786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.28599999999999998,"15min":0.247},"max":{"1min":0.68000000000000005,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.45400000000000001,"5min":0.37,"15min":0.29699999999999999},"max":{"1min":0.66400000000000003,"5min":0.73599999999999999,"15min":0.86299999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.41899999999999998,"5min":0.32100000000000001,"15min":0.27400000000000002},"max":{"1min":0.65700000000000003,"5min":0.82999999999999996,"15min":0.92300000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.34300000000000003},"max":{"1min":0.69099999999999995,"5min":0.75700000000000001,"15min":0.83599999999999997},"last":0.55700000000000005}]},{"osd":2,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.71699999999999997,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55800000000000005,"15min":0.54900000000000004},"min":{"1min":0.39900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68500000000000005,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.28299999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.69299999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.22800000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65700000000000003},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Apr 20 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.23899999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.629},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.34799999999999998,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.84399999999999997},"last":0.61899999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607802,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81620,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94290220,"statfs":{"total":96636764160,"available":96553185280,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77199815},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.36599999999999999,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.23300000000000001},"max":{"1min":0.45900000000000002,"5min":0.56699999999999995,"15min":0.71699999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":1.8440000000000001},"last":0.375}]},{"osd":2,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.373,"15min":0.39800000000000002},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":7.8849999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42999999999999999,"15min":0.44600000000000001},"min":{"1min":0.317,"5min":0.308,"15min":0.27500000000000002},"max":{"1min":0.54400000000000004,"5min":0.71799999999999997,"15min":4.9870000000000001},"last":0.317}]},{"osd":3,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":1.2410000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.46300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85599999999999998,"15min":1.2969999999999999},"last":0.46700000000000003}]},{"osd":4,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.51900000000000002,"5min":0.38800000000000001,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.88900000000000001,"15min":0.98899999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65100000000000002,"15min":0.61599999999999999},"min":{"1min":0.54900000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":0.90400000000000003,"15min":0.98999999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Sat Apr 20 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.78900000000000003,"5min":0.85399999999999998,"15min":1.0069999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.496,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85599999999999998,"5min":0.873,"15min":1.0269999999999999},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:36:12.422 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-20T02:16:43.015396+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=16, tm_sec=43, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=2, tm_min=35, tm_sec=47, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T02:36:12.423 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-20T02:36:32.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.568+0000 7f3b1d8dc700 10 monclient: get_monmap_and_config 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.568+0000 7f3b1d8dc700 10 monclient: build_initial_monmap 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.568+0000 7f3b1d8dc700 10 monclient: monmap: 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:36:32.569217+0000 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:36:32.569217+0000 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:36:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:36:32.569 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.568+0000 7f3b1d8dc700 1 Processor -- start 2024-04-20T02:36:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.568+0000 7f3b1d8dc700 1 -- start start 2024-04-20T02:36:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.568+0000 7f3b1d8dc700 10 monclient: init 2024-04-20T02:36:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:36:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient: _add_conns ranks=[10,5,0,4,1,7,6,2,8,9,3] 2024-04-20T02:36:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3b180a17e0 0x7f3b180a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): picked mon.noname-f con 0x7f3b180a17e0 addr v2:172.21.15.179:3300/0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 1 --2- >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3b180709d0 0x7f3b18135b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): picked mon.noname-h con 0x7f3b180709d0 addr [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b181360a0 addr v1:172.21.15.47:6789/0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b180751e0 con 0x7f3b181360a0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b18071f10 con 0x7f3b180a17e0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 1 -- --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18072160 con 0x7f3b180709d0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b1d8dc700 10 monclient(hunting): authenticate will time out at 2871.970409s 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b177fe700 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3b181360a0 0x7f3b180a38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47524/0 (socket says 172.21.15.47:47524) 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b177fe700 1 -- 172.21.15.47:0/1380230959 learned_addr learned my addr 172.21.15.47:0/1380230959 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 1 --2- 172.21.15.47:0/1380230959 >> v2:172.21.15.179:3300/0 conn(0x7f3b180a17e0 0x7f3b180a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 1 --2- 172.21.15.47:0/1380230959 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3b180709d0 0x7f3b18135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): get_auth_request con 0x7f3b180a17e0 auth_method 0 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): get_auth_request con 0x7f3b180709d0 auth_method 0 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b15ffb700 1 -- 172.21.15.47:0/1380230959 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081666656 0 0) 0x7f3b180751e0 con 0x7f3b181360a0 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b15ffb700 10 monclient(hunting): _negotiate 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b15ffb700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b15ffb700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b15ffb700 10 monclient(hunting): my global_id is 104598 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b15ffb700 1 -- 172.21.15.47:0/1380230959 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b040035b0 con 0x7f3b181360a0 2024-04-20T02:36:32.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.569+0000 7f3b167fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 10 monclient(hunting): handle_auth_done global_id 84875 payload 274 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 1 -- 172.21.15.47:0/1380230959 >> v2:172.21.15.179:3300/0 conn(0x7f3b180a17e0 msgr2=0x7f3b180a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 1 --2- 172.21.15.47:0/1380230959 >> v2:172.21.15.179:3300/0 conn(0x7f3b180a17e0 0x7f3b180a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 1 -- 172.21.15.47:0/1380230959 >> v1:172.21.15.47:6789/0 conn(0x7f3b181360a0 legacy=0x7f3b180a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 10 monclient: _finish_hunting 0 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 1 -- 172.21.15.47:0/1380230959 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208694554 0 0) 0x7f3b040035b0 con 0x7f3b181360a0 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 1 monclient: found mon.noname-h 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.179:3302/0 2024-04-20T02:36:32.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 1 -- 172.21.15.47:0/1380230959 --> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18067070 con 0x7f3b180709d0 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b16ffd700 1 --2- 172.21.15.47:0/1380230959 >> v2:172.21.15.179:3300/0 conn(0x7f3b180a17e0 0x7f3b180a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b167fc700 1 --2- 172.21.15.47:0/1380230959 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3b180709d0 0x7f3b18135b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009840 tx=0x7f3b00009b50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9863fb21366b240b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 1 -- 172.21.15.47:0/1380230959 <== mon.5 v2:172.21.15.179:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b10001c60 con 0x7f3b180709d0 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: dump: 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.584 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: _finish_auth 0 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: _check_auth_tickets 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b1d8dc700 5 monclient: authenticate success, global_id 84875 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 1 -- 172.21.15.47:0/1380230959 <== mon.5 v2:172.21.15.179:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b000044a0 con 0x7f3b180709d0 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b1d8dc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:36:32.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b1d8dc700 10 monclient: get_monmap_and_config success 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b1d8dc700 4 set_mon_vals no callback set 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 1 -- 172.21.15.47:0/1380230959 <== mon.5 v2:172.21.15.179:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b00003b10 con 0x7f3b180709d0 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.570+0000 7f3b15ffb700 10 monclient: dump: 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.586 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 10 monclient: shutdown 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/1380230959 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3b180709d0 msgr2=0x7f3b18135b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/1380230959 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3b180709d0 0x7f3b18135b60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009840 tx=0x7f3b00009b50 comp rx=0 tx=0).stop 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/1380230959 shutdown_connections 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/1380230959 >> v2:172.21.15.179:3300/0 conn(0x7f3b180a17e0 0x7f3b180a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/1380230959 >> [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] conn(0x7f3b180709d0 0x7f3b18135b60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/1380230959 >> 172.21.15.47:0/1380230959 conn(0x7f3b180a5040 msgr2=0x7f3b18066610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/1380230959 shutdown_connections 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/1380230959 wait complete. 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 10 monclient: build_initial_monmap 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.571+0000 7f3b1d8dc700 10 monclient: monmap: 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:36:32.588 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 1 librados: starting msgr at 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 1 librados: starting objecter 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 1 Processor -- start 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 1 -- start start 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 1 librados: setting wanted keys 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 1 librados: calling monclient init 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.572+0000 7f3b1d8dc700 10 monclient: init 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient: _reopen_session rank -1 2024-04-20T02:36:32.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient: _add_conns ranks=[2,3,4,7,8,0,5,1,6] 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 0x7f3b181ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): picked mon.noname-b con 0x7f3b180a17e0 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b181360a0 0x7f3b1806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): picked mon.noname-g con 0x7f3b181360a0 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b181cd120 0x7f3b1806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): picked mon.noname-c con 0x7f3b181cd120 addr [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18125620 con 0x7f3b180a17e0 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18126b40 con 0x7f3b181360a0 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18126d90 con 0x7f3b181cd120 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): _renew_subs 2024-04-20T02:36:32.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b1d8dc700 10 monclient(hunting): authenticate will time out at 2871.974150s 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 0x7f3b181ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 0x7f3b181ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:41862/0 (socket says 172.21.15.47:41862) 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b181cd120 0x7f3b1806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 -- 172.21.15.47:0/2029635680 learned_addr learned my addr 172.21.15.47:0/2029635680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): get_auth_request con 0x7f3b180a17e0 auth_method 0 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): get_auth_request con 0x7f3b181cd120 auth_method 0 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b177fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b167fc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b181360a0 0x7f3b1806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b167fc700 10 monclient(hunting): get_auth_request con 0x7f3b181360a0 auth_method 0 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b167fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b167fc700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b167fc700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient(hunting): handle_auth_done global_id 114293 payload 824 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 -- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b181cd120 msgr2=0x7f3b1806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b181cd120 0x7f3b1806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 -- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b181360a0 msgr2=0x7f3b1806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b181360a0 0x7f3b1806c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient: _finish_hunting 0 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 monclient: found mon.noname-b 2024-04-20T02:36:32.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.47:3301/0 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 -- 172.21.15.47:0/2029635680 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080094a0 con 0x7f3b180a17e0 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b167fc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b181360a0 0x7f3b1806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b16ffd700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 0x7f3b181ccbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800ec10 tx=0x7f3b0800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b0ffff700 10 client.?.objecter ms_handle_connect 0x7f3b180a17e0 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.573+0000 7f3b0ffff700 10 client.?.objecter resend_mon_ops 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mon.2 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b0800cd80 con 0x7f3b180a17e0 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: dump: 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.593 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: _finish_auth 0 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: _check_auth_tickets 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mon.2 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b08013470 con 0x7f3b180a17e0 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mon.2 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b0800cee0 con 0x7f3b180a17e0 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 10 monclient: dump: 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.595 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b157fa700 4 set_mon_vals no callback set 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 5 monclient: authenticate success, global_id 114293 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 10 monclient: _renew_subs 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b000094a0 con 0x7f3b180a17e0 2024-04-20T02:36:32.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 10 client.114293.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 10 monclient: _renew_subs 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1809ed30 con 0x7f3b180a17e0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b1d8dc700 1 librados: init done 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.574+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mon.2 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3b0800c910 con 0x7f3b180a17e0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b0ffff700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3af803a2f0 0x7f3af803c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b1d8dc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b1d8dc700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b1d8dc700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.47:3301/0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mon.2 v2:172.21.15.47:3301/0 5 ==== osd_map(697..697 src has 621..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f3b0804a6d0 con 0x7f3b180a17e0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b0ffff700 10 client.114293.objecter ms_dispatch 0x7f3b18001550 osd_map(697..697 src has 621..697) v4 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b0ffff700 3 client.114293.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b0ffff700 3 client.114293.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b18126ed0 con 0x7f3b180a17e0 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b0ffff700 20 client.114293.objecter dump_active .. 0 homeless 2024-04-20T02:36:32.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b167fc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3af803a2f0 0x7f3af803c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.575+0000 7f3b167fc700 10 monclient: get_auth_request con 0x7f3af803a2f0 auth_method 0 2024-04-20T02:36:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.576+0000 7f3b167fc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3af803a2f0 0x7f3af803c7b0 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009840 tx=0x7f3b0001c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.576+0000 7f3b0ffff700 10 client.114293.objecter ms_handle_connect 0x7f3af803a2f0 2024-04-20T02:36:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.580+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mon.2 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f3b18126ed0 con 0x7f3b180a17e0 2024-04-20T02:36:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.580+0000 7f3b0ffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:32.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.580+0000 7f3b0ffff700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:36:32.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.797+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 --> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b1806aac0 con 0x7f3af803a2f0 2024-04-20T02:36:32.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.802+0000 7f3b0ffff700 1 -- 172.21.15.47:0/2029635680 <== mgr.4114 v2:172.21.15.179:6824/41706 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+134789 (secure 0 0 0) 0x7f3b000036a0 con 0x7f3af803a2f0 2024-04-20T02:36:32.804 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T02:36:32.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 10 librados: watch_flush enter 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 10 librados: watch_flush exit 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 20 client.114293.objecter shutdown clearing up homeless session... 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 10 client.114293.objecter successfully canceled tick 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3af803a2f0 msgr2=0x7f3af803c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3af803a2f0 0x7f3af803c7b0 secure :-1 s=READY pgs=3427 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009840 tx=0x7f3b0001c040 comp rx=0 tx=0).stop 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 10 monclient: shutdown 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 msgr2=0x7f3b181ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 0x7f3b181ccbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800ec10 tx=0x7f3b0800ef20 comp rx=0 tx=0).stop 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 shutdown_connections 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3af803a2f0 0x7f3af803c7b0 unknown :-1 s=CLOSED pgs=3427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b181cd120 0x7f3b1806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b180a17e0 0x7f3b181ccbe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 --2- 172.21.15.47:0/2029635680 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3b181360a0 0x7f3b1806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 >> 172.21.15.47:0/2029635680 conn(0x7f3b180a5040 msgr2=0x7f3b18134f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:36:32.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 shutdown_connections 2024-04-20T02:36:32.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 -- 172.21.15.47:0/2029635680 wait complete. 2024-04-20T02:36:32.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.805+0000 7f3b1d8dc700 1 librados: shutdown 2024-04-20T02:36:32.807 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T02:36:32.819 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":1316,"stamp":"2024-04-20T02:36:32.625803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":106,"num_write":166,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":470288,"kb_used_data":38368,"kb_used_omap":9,"kb_used_meta":431862,"kb_avail":565760752,"statfs":{"total":579820584960,"available":579339010048,"internally_reserved":0,"allocated":39288832,"data_stored":20149014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":442227367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002982"},"pg_stats":[{"pgid":"5.1c","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.960325+0000","last_change":"2024-04-20T02:36:00.960429+0000","last_active":"2024-04-20T02:36:00.960325+0000","last_peered":"2024-04-20T02:36:00.960325+0000","last_clean":"2024-04-20T02:36:00.960325+0000","last_became_active":"2024-04-20T02:16:44.179944+0000","last_became_peered":"2024-04-20T02:16:44.179944+0000","last_unstale":"2024-04-20T02:36:00.960325+0000","last_undegraded":"2024-04-20T02:36:00.960325+0000","last_fullsized":"2024-04-20T02:36:00.960325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.960301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.960301+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.960301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:24.813961+0000","scrub_duration":0.00198985,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":697,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:05.042460+0000","last_change":"2024-04-20T02:36:05.042576+0000","last_active":"2024-04-20T02:36:05.042460+0000","last_peered":"2024-04-20T02:36:05.042460+0000","last_clean":"2024-04-20T02:36:05.042460+0000","last_became_active":"2024-04-20T02:16:44.130319+0000","last_became_peered":"2024-04-20T02:16:44.130319+0000","last_unstale":"2024-04-20T02:36:05.042460+0000","last_undegraded":"2024-04-20T02:36:05.042460+0000","last_fullsized":"2024-04-20T02:36:05.042460+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:05.042434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:05.042434+0000","last_clean_scrub_stamp":"2024-04-20T02:36:05.042434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:38.169448+0000","scrub_duration":0.0035587029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:06.484469+0000","last_change":"2024-04-20T02:36:06.484574+0000","last_active":"2024-04-20T02:36:06.484469+0000","last_peered":"2024-04-20T02:36:06.484469+0000","last_clean":"2024-04-20T02:36:06.484469+0000","last_became_active":"2024-04-20T02:16:44.117655+0000","last_became_peered":"2024-04-20T02:16:44.117655+0000","last_unstale":"2024-04-20T02:36:06.484469+0000","last_undegraded":"2024-04-20T02:36:06.484469+0000","last_fullsized":"2024-04-20T02:36:06.484469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:06.484444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:06.484444+0000","last_clean_scrub_stamp":"2024-04-20T02:36:06.484444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:08.623009+0000","scrub_duration":0.0021908510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:09.487603+0000","last_change":"2024-04-20T02:36:09.487721+0000","last_active":"2024-04-20T02:36:09.487603+0000","last_peered":"2024-04-20T02:36:09.487603+0000","last_clean":"2024-04-20T02:36:09.487603+0000","last_became_active":"2024-04-20T02:16:44.111746+0000","last_became_peered":"2024-04-20T02:16:44.111746+0000","last_unstale":"2024-04-20T02:36:09.487603+0000","last_undegraded":"2024-04-20T02:36:09.487603+0000","last_fullsized":"2024-04-20T02:36:09.487603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:09.487576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:09.487576+0000","last_clean_scrub_stamp":"2024-04-20T02:36:09.487576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:00:53.347931+0000","scrub_duration":0.0028059840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.969487+0000","last_change":"2024-04-20T02:35:53.969583+0000","last_active":"2024-04-20T02:35:53.969487+0000","last_peered":"2024-04-20T02:35:53.969487+0000","last_clean":"2024-04-20T02:35:53.969487+0000","last_became_active":"2024-04-20T02:16:44.117331+0000","last_became_peered":"2024-04-20T02:16:44.117331+0000","last_unstale":"2024-04-20T02:35:53.969487+0000","last_undegraded":"2024-04-20T02:35:53.969487+0000","last_fullsized":"2024-04-20T02:35:53.969487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.969460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.969460+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.969460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:57:18.692594+0000","scrub_duration":0.0014488369999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:57.108910+0000","last_change":"2024-04-20T02:35:57.109019+0000","last_active":"2024-04-20T02:35:57.108910+0000","last_peered":"2024-04-20T02:35:57.108910+0000","last_clean":"2024-04-20T02:35:57.108910+0000","last_became_active":"2024-04-20T02:16:44.123156+0000","last_became_peered":"2024-04-20T02:16:44.123156+0000","last_unstale":"2024-04-20T02:35:57.108910+0000","last_undegraded":"2024-04-20T02:35:57.108910+0000","last_fullsized":"2024-04-20T02:35:57.108910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:57.108884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:57.108884+0000","last_clean_scrub_stamp":"2024-04-20T02:35:57.108884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:50:51.598896+0000","scrub_duration":0.0029345410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.734257+0000","last_change":"2024-04-20T02:36:00.734368+0000","last_active":"2024-04-20T02:36:00.734257+0000","last_peered":"2024-04-20T02:36:00.734257+0000","last_clean":"2024-04-20T02:36:00.734257+0000","last_became_active":"2024-04-20T02:16:44.117507+0000","last_became_peered":"2024-04-20T02:16:44.117507+0000","last_unstale":"2024-04-20T02:36:00.734257+0000","last_undegraded":"2024-04-20T02:36:00.734257+0000","last_fullsized":"2024-04-20T02:36:00.734257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.734217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.734217+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.734217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:31:28.784264+0000","scrub_duration":0.001716686,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":683,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.189355+0000","last_change":"2024-04-20T02:36:00.189355+0000","last_active":"2024-04-20T02:36:00.189355+0000","last_peered":"2024-04-20T02:36:00.189355+0000","last_clean":"2024-04-20T02:36:00.189355+0000","last_became_active":"2024-04-20T02:16:44.228394+0000","last_became_peered":"2024-04-20T02:16:44.228394+0000","last_unstale":"2024-04-20T02:36:00.189355+0000","last_undegraded":"2024-04-20T02:36:00.189355+0000","last_fullsized":"2024-04-20T02:36:00.189355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.189208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.189208+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.189208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:05.467264+0000","scrub_duration":0.0020655449999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:59.768613+0000","last_change":"2024-04-20T02:35:59.768751+0000","last_active":"2024-04-20T02:35:59.768613+0000","last_peered":"2024-04-20T02:35:59.768613+0000","last_clean":"2024-04-20T02:35:59.768613+0000","last_became_active":"2024-04-20T02:16:44.170848+0000","last_became_peered":"2024-04-20T02:16:44.170848+0000","last_unstale":"2024-04-20T02:35:59.768613+0000","last_undegraded":"2024-04-20T02:35:59.768613+0000","last_fullsized":"2024-04-20T02:35:59.768613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:59.768589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:59.768589+0000","last_clean_scrub_stamp":"2024-04-20T02:35:59.768589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:45:43.603775+0000","scrub_duration":0.0021545359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:04.089640+0000","last_change":"2024-04-20T02:36:04.089640+0000","last_active":"2024-04-20T02:36:04.089640+0000","last_peered":"2024-04-20T02:36:04.089640+0000","last_clean":"2024-04-20T02:36:04.089640+0000","last_became_active":"2024-04-20T02:16:44.139769+0000","last_became_peered":"2024-04-20T02:16:44.139769+0000","last_unstale":"2024-04-20T02:36:04.089640+0000","last_undegraded":"2024-04-20T02:36:04.089640+0000","last_fullsized":"2024-04-20T02:36:04.089640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:04.089517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:04.089517+0000","last_clean_scrub_stamp":"2024-04-20T02:36:04.089517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:44:49.752468+0000","scrub_duration":0.003217433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:03.509696+0000","last_change":"2024-04-20T02:36:03.509696+0000","last_active":"2024-04-20T02:36:03.509696+0000","last_peered":"2024-04-20T02:36:03.509696+0000","last_clean":"2024-04-20T02:36:03.509696+0000","last_became_active":"2024-04-20T02:16:44.187150+0000","last_became_peered":"2024-04-20T02:16:44.187150+0000","last_unstale":"2024-04-20T02:36:03.509696+0000","last_undegraded":"2024-04-20T02:36:03.509696+0000","last_fullsized":"2024-04-20T02:36:03.509696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:03.509520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:03.509520+0000","last_clean_scrub_stamp":"2024-04-20T02:36:03.509520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:00.546275+0000","scrub_duration":0.0026646439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":692,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:02.505228+0000","last_change":"2024-04-20T02:36:02.505228+0000","last_active":"2024-04-20T02:36:02.505228+0000","last_peered":"2024-04-20T02:36:02.505228+0000","last_clean":"2024-04-20T02:36:02.505228+0000","last_became_active":"2024-04-20T02:16:44.167097+0000","last_became_peered":"2024-04-20T02:16:44.167097+0000","last_unstale":"2024-04-20T02:36:02.505228+0000","last_undegraded":"2024-04-20T02:36:02.505228+0000","last_fullsized":"2024-04-20T02:36:02.505228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:02.505084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:02.505084+0000","last_clean_scrub_stamp":"2024-04-20T02:36:02.505084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:31:23.979872+0000","scrub_duration":0.0027940209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:03.561731+0000","last_change":"2024-04-20T02:36:03.561731+0000","last_active":"2024-04-20T02:36:03.561731+0000","last_peered":"2024-04-20T02:36:03.561731+0000","last_clean":"2024-04-20T02:36:03.561731+0000","last_became_active":"2024-04-20T02:16:44.168588+0000","last_became_peered":"2024-04-20T02:16:44.168588+0000","last_unstale":"2024-04-20T02:36:03.561731+0000","last_undegraded":"2024-04-20T02:36:03.561731+0000","last_fullsized":"2024-04-20T02:36:03.561731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:03.561623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:03.561623+0000","last_clean_scrub_stamp":"2024-04-20T02:36:03.561623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:43.448156+0000","scrub_duration":0.0053437399999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:04.511525+0000","last_change":"2024-04-20T02:36:04.511623+0000","last_active":"2024-04-20T02:36:04.511525+0000","last_peered":"2024-04-20T02:36:04.511525+0000","last_clean":"2024-04-20T02:36:04.511525+0000","last_became_active":"2024-04-20T02:16:44.103711+0000","last_became_peered":"2024-04-20T02:16:44.103711+0000","last_unstale":"2024-04-20T02:36:04.511525+0000","last_undegraded":"2024-04-20T02:36:04.511525+0000","last_fullsized":"2024-04-20T02:36:04.511525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:04.511501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:04.511501+0000","last_clean_scrub_stamp":"2024-04-20T02:36:04.511501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:51:37.343223+0000","scrub_duration":0.0031208719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.753399+0000","last_change":"2024-04-20T02:35:53.753399+0000","last_active":"2024-04-20T02:35:53.753399+0000","last_peered":"2024-04-20T02:35:53.753399+0000","last_clean":"2024-04-20T02:35:53.753399+0000","last_became_active":"2024-04-20T02:16:44.207027+0000","last_became_peered":"2024-04-20T02:16:44.207027+0000","last_unstale":"2024-04-20T02:35:53.753399+0000","last_undegraded":"2024-04-20T02:35:53.753399+0000","last_fullsized":"2024-04-20T02:35:53.753399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.753295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.753295+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.753295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:45:53.556524+0000","scrub_duration":0.0019783930000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:55.674083+0000","last_change":"2024-04-20T02:35:55.674192+0000","last_active":"2024-04-20T02:35:55.674083+0000","last_peered":"2024-04-20T02:35:55.674083+0000","last_clean":"2024-04-20T02:35:55.674083+0000","last_became_active":"2024-04-20T02:16:44.209154+0000","last_became_peered":"2024-04-20T02:16:44.209154+0000","last_unstale":"2024-04-20T02:35:55.674083+0000","last_undegraded":"2024-04-20T02:35:55.674083+0000","last_fullsized":"2024-04-20T02:35:55.674083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:55.674057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:55.674057+0000","last_clean_scrub_stamp":"2024-04-20T02:35:55.674057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:19.577135+0000","scrub_duration":0.003317946,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:01.565187+0000","last_change":"2024-04-20T02:36:01.565299+0000","last_active":"2024-04-20T02:36:01.565187+0000","last_peered":"2024-04-20T02:36:01.565187+0000","last_clean":"2024-04-20T02:36:01.565187+0000","last_became_active":"2024-04-20T02:16:44.116619+0000","last_became_peered":"2024-04-20T02:16:44.116619+0000","last_unstale":"2024-04-20T02:36:01.565187+0000","last_undegraded":"2024-04-20T02:36:01.565187+0000","last_fullsized":"2024-04-20T02:36:01.565187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:01.565162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:01.565162+0000","last_clean_scrub_stamp":"2024-04-20T02:36:01.565162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:37:46.426151+0000","scrub_duration":0.001696677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:55.490010+0000","last_change":"2024-04-20T02:35:55.490130+0000","last_active":"2024-04-20T02:35:55.490010+0000","last_peered":"2024-04-20T02:35:55.490010+0000","last_clean":"2024-04-20T02:35:55.490010+0000","last_became_active":"2024-04-20T02:16:44.168518+0000","last_became_peered":"2024-04-20T02:16:44.168518+0000","last_unstale":"2024-04-20T02:35:55.490010+0000","last_undegraded":"2024-04-20T02:35:55.490010+0000","last_fullsized":"2024-04-20T02:35:55.490010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:55.489984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:55.489984+0000","last_clean_scrub_stamp":"2024-04-20T02:35:55.489984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:09:35.155853+0000","scrub_duration":0.0034148659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:04.017845+0000","last_change":"2024-04-20T02:36:04.017966+0000","last_active":"2024-04-20T02:36:04.017845+0000","last_peered":"2024-04-20T02:36:04.017845+0000","last_clean":"2024-04-20T02:36:04.017845+0000","last_became_active":"2024-04-20T02:16:44.155758+0000","last_became_peered":"2024-04-20T02:16:44.155758+0000","last_unstale":"2024-04-20T02:36:04.017845+0000","last_undegraded":"2024-04-20T02:36:04.017845+0000","last_fullsized":"2024-04-20T02:36:04.017845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:04.017819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:04.017819+0000","last_clean_scrub_stamp":"2024-04-20T02:36:04.017819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:42.299656+0000","scrub_duration":0.001505057,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":699,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:54.622426+0000","last_change":"2024-04-20T02:35:54.622527+0000","last_active":"2024-04-20T02:35:54.622426+0000","last_peered":"2024-04-20T02:35:54.622426+0000","last_clean":"2024-04-20T02:35:54.622426+0000","last_became_active":"2024-04-20T02:15:46.242249+0000","last_became_peered":"2024-04-20T02:15:46.242249+0000","last_unstale":"2024-04-20T02:35:54.622426+0000","last_undegraded":"2024-04-20T02:35:54.622426+0000","last_fullsized":"2024-04-20T02:35:54.622426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:54.622402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:54.622402+0000","last_clean_scrub_stamp":"2024-04-20T02:35:54.622402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:54:14.700754+0000","scrub_duration":0.001432467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":709,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:56.142200+0000","last_change":"2024-04-20T02:35:56.142296+0000","last_active":"2024-04-20T02:35:56.142200+0000","last_peered":"2024-04-20T02:35:56.142200+0000","last_clean":"2024-04-20T02:35:56.142200+0000","last_became_active":"2024-04-20T02:16:44.190566+0000","last_became_peered":"2024-04-20T02:16:44.190566+0000","last_unstale":"2024-04-20T02:35:56.142200+0000","last_undegraded":"2024-04-20T02:35:56.142200+0000","last_fullsized":"2024-04-20T02:35:56.142200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:56.142174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:56.142174+0000","last_clean_scrub_stamp":"2024-04-20T02:35:56.142174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:03:44.371018+0000","scrub_duration":0.002113073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:02.531633+0000","last_change":"2024-04-20T02:36:02.531734+0000","last_active":"2024-04-20T02:36:02.531633+0000","last_peered":"2024-04-20T02:36:02.531633+0000","last_clean":"2024-04-20T02:36:02.531633+0000","last_became_active":"2024-04-20T02:16:44.169063+0000","last_became_peered":"2024-04-20T02:16:44.169063+0000","last_unstale":"2024-04-20T02:36:02.531633+0000","last_undegraded":"2024-04-20T02:36:02.531633+0000","last_fullsized":"2024-04-20T02:36:02.531633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:02.531608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:02.531608+0000","last_clean_scrub_stamp":"2024-04-20T02:36:02.531608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:11:24.299164+0000","scrub_duration":0.001682682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":687,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.077353+0000","last_change":"2024-04-20T02:35:53.077457+0000","last_active":"2024-04-20T02:35:53.077353+0000","last_peered":"2024-04-20T02:35:53.077353+0000","last_clean":"2024-04-20T02:35:53.077353+0000","last_became_active":"2024-04-20T02:15:46.245980+0000","last_became_peered":"2024-04-20T02:15:46.245980+0000","last_unstale":"2024-04-20T02:35:53.077353+0000","last_undegraded":"2024-04-20T02:35:53.077353+0000","last_fullsized":"2024-04-20T02:35:53.077353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.077326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.077326+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.077326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:21:54.436030+0000","scrub_duration":0.002076647,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":709,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:00.178287+0000","last_change":"2024-04-20T02:36:00.178287+0000","last_active":"2024-04-20T02:36:00.178287+0000","last_peered":"2024-04-20T02:36:00.178287+0000","last_clean":"2024-04-20T02:36:00.178287+0000","last_became_active":"2024-04-20T02:16:44.150893+0000","last_became_peered":"2024-04-20T02:16:44.150893+0000","last_unstale":"2024-04-20T02:36:00.178287+0000","last_undegraded":"2024-04-20T02:36:00.178287+0000","last_fullsized":"2024-04-20T02:36:00.178287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:00.178171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:00.178171+0000","last_clean_scrub_stamp":"2024-04-20T02:36:00.178171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T03:44:40.321043+0000","scrub_duration":0.0020325809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":699,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:50.635789+0000","last_change":"2024-04-20T02:35:50.635896+0000","last_active":"2024-04-20T02:35:50.635789+0000","last_peered":"2024-04-20T02:35:50.635789+0000","last_clean":"2024-04-20T02:35:50.635789+0000","last_became_active":"2024-04-20T02:15:46.241551+0000","last_became_peered":"2024-04-20T02:15:46.241551+0000","last_unstale":"2024-04-20T02:35:50.635789+0000","last_undegraded":"2024-04-20T02:35:50.635789+0000","last_fullsized":"2024-04-20T02:35:50.635789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:50.635763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:50.635763+0000","last_clean_scrub_stamp":"2024-04-20T02:35:50.635763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T02:38:26.315486+0000","scrub_duration":0.001359065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:52.510684+0000","last_change":"2024-04-20T02:35:52.510785+0000","last_active":"2024-04-20T02:35:52.510684+0000","last_peered":"2024-04-20T02:35:52.510684+0000","last_clean":"2024-04-20T02:35:52.510684+0000","last_became_active":"2024-04-20T02:15:46.242814+0000","last_became_peered":"2024-04-20T02:15:46.242814+0000","last_unstale":"2024-04-20T02:35:52.510684+0000","last_undegraded":"2024-04-20T02:35:52.510684+0000","last_fullsized":"2024-04-20T02:35:52.510684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:52.510661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:52.510661+0000","last_clean_scrub_stamp":"2024-04-20T02:35:52.510661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:40:43.938245+0000","scrub_duration":0.0014765329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"697'119","reported_seq":848,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:50.535148+0000","last_change":"2024-04-20T02:35:50.535232+0000","last_active":"2024-04-20T02:35:50.535148+0000","last_peered":"2024-04-20T02:35:50.535148+0000","last_clean":"2024-04-20T02:35:50.535148+0000","last_became_active":"2024-04-20T02:15:46.243208+0000","last_became_peered":"2024-04-20T02:15:46.243208+0000","last_unstale":"2024-04-20T02:35:50.535148+0000","last_undegraded":"2024-04-20T02:35:50.535148+0000","last_fullsized":"2024-04-20T02:35:50.535148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"697'119","last_scrub_stamp":"2024-04-20T02:35:50.535120+0000","last_deep_scrub":"697'119","last_deep_scrub_stamp":"2024-04-20T02:35:50.535120+0000","last_clean_scrub_stamp":"2024-04-20T02:35:50.535120+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:40.506051+0000","scrub_duration":0.006522992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:55.145269+0000","last_change":"2024-04-20T02:35:55.145368+0000","last_active":"2024-04-20T02:35:55.145269+0000","last_peered":"2024-04-20T02:35:55.145269+0000","last_clean":"2024-04-20T02:35:55.145269+0000","last_became_active":"2024-04-20T02:16:44.175527+0000","last_became_peered":"2024-04-20T02:16:44.175527+0000","last_unstale":"2024-04-20T02:35:55.145269+0000","last_undegraded":"2024-04-20T02:35:55.145269+0000","last_fullsized":"2024-04-20T02:35:55.145269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:55.145239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:55.145239+0000","last_clean_scrub_stamp":"2024-04-20T02:35:55.145239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T07:03:53.365058+0000","scrub_duration":0.0020752169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:59.174331+0000","last_change":"2024-04-20T02:35:59.174331+0000","last_active":"2024-04-20T02:35:59.174331+0000","last_peered":"2024-04-20T02:35:59.174331+0000","last_clean":"2024-04-20T02:35:59.174331+0000","last_became_active":"2024-04-20T02:16:44.209341+0000","last_became_peered":"2024-04-20T02:16:44.209341+0000","last_unstale":"2024-04-20T02:35:59.174331+0000","last_undegraded":"2024-04-20T02:35:59.174331+0000","last_fullsized":"2024-04-20T02:35:59.174331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:59.174177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:59.174177+0000","last_clean_scrub_stamp":"2024-04-20T02:35:59.174177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:19:39.964602+0000","scrub_duration":0.003510811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":701,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.011645+0000","last_change":"2024-04-20T02:35:53.011781+0000","last_active":"2024-04-20T02:35:53.011645+0000","last_peered":"2024-04-20T02:35:53.011645+0000","last_clean":"2024-04-20T02:35:53.011645+0000","last_became_active":"2024-04-20T02:15:46.243097+0000","last_became_peered":"2024-04-20T02:15:46.243097+0000","last_unstale":"2024-04-20T02:35:53.011645+0000","last_undegraded":"2024-04-20T02:35:53.011645+0000","last_fullsized":"2024-04-20T02:35:53.011645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.011618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.011618+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.011618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T05:57:15.737315+0000","scrub_duration":0.003340675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:51.506449+0000","last_change":"2024-04-20T02:35:51.506531+0000","last_active":"2024-04-20T02:35:51.506449+0000","last_peered":"2024-04-20T02:35:51.506449+0000","last_clean":"2024-04-20T02:35:51.506449+0000","last_became_active":"2024-04-20T02:15:46.243184+0000","last_became_peered":"2024-04-20T02:15:46.243184+0000","last_unstale":"2024-04-20T02:35:51.506449+0000","last_undegraded":"2024-04-20T02:35:51.506449+0000","last_fullsized":"2024-04-20T02:35:51.506449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:51.506427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:51.506427+0000","last_clean_scrub_stamp":"2024-04-20T02:35:51.506427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:47:48.096698+0000","scrub_duration":0.0017070150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":684,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:58.162257+0000","last_change":"2024-04-20T02:35:58.162351+0000","last_active":"2024-04-20T02:35:58.162257+0000","last_peered":"2024-04-20T02:35:58.162257+0000","last_clean":"2024-04-20T02:35:58.162257+0000","last_became_active":"2024-04-20T02:16:44.150765+0000","last_became_peered":"2024-04-20T02:16:44.150765+0000","last_unstale":"2024-04-20T02:35:58.162257+0000","last_undegraded":"2024-04-20T02:35:58.162257+0000","last_fullsized":"2024-04-20T02:35:58.162257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:58.162231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:58.162231+0000","last_clean_scrub_stamp":"2024-04-20T02:35:58.162231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:32:51.127508+0000","scrub_duration":0.0029800709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":712,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:53.121651+0000","last_change":"2024-04-20T02:35:53.121766+0000","last_active":"2024-04-20T02:35:53.121651+0000","last_peered":"2024-04-20T02:35:53.121651+0000","last_clean":"2024-04-20T02:35:53.121651+0000","last_became_active":"2024-04-20T02:15:46.244796+0000","last_became_peered":"2024-04-20T02:15:46.244796+0000","last_unstale":"2024-04-20T02:35:53.121651+0000","last_undegraded":"2024-04-20T02:35:53.121651+0000","last_fullsized":"2024-04-20T02:35:53.121651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:53.121628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:53.121628+0000","last_clean_scrub_stamp":"2024-04-20T02:35:53.121628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:03:34.484457+0000","scrub_duration":0.0018377700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"697'8","reported_seq":706,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:56.512265+0000","last_change":"2024-04-20T02:35:56.512424+0000","last_active":"2024-04-20T02:35:56.512265+0000","last_peered":"2024-04-20T02:35:56.512265+0000","last_clean":"2024-04-20T02:35:56.512265+0000","last_became_active":"2024-04-20T02:16:44.105229+0000","last_became_peered":"2024-04-20T02:16:44.105229+0000","last_unstale":"2024-04-20T02:35:56.512265+0000","last_undegraded":"2024-04-20T02:35:56.512265+0000","last_fullsized":"2024-04-20T02:35:56.512265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"697'8","last_scrub_stamp":"2024-04-20T02:35:56.512213+0000","last_deep_scrub":"697'8","last_deep_scrub_stamp":"2024-04-20T02:35:56.512213+0000","last_clean_scrub_stamp":"2024-04-20T02:35:56.512213+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:13:41.636081+0000","scrub_duration":0.0059889339999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:54.490031+0000","last_change":"2024-04-20T02:35:54.490164+0000","last_active":"2024-04-20T02:35:54.490031+0000","last_peered":"2024-04-20T02:35:54.490031+0000","last_clean":"2024-04-20T02:35:54.490031+0000","last_became_active":"2024-04-20T02:15:46.244933+0000","last_became_peered":"2024-04-20T02:15:46.244933+0000","last_unstale":"2024-04-20T02:35:54.490031+0000","last_undegraded":"2024-04-20T02:35:54.490031+0000","last_fullsized":"2024-04-20T02:35:54.490031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:54.490001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:54.490001+0000","last_clean_scrub_stamp":"2024-04-20T02:35:54.490001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:56:45.665864+0000","scrub_duration":0.001558888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":696,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:01.702298+0000","last_change":"2024-04-20T02:36:01.702402+0000","last_active":"2024-04-20T02:36:01.702298+0000","last_peered":"2024-04-20T02:36:01.702298+0000","last_clean":"2024-04-20T02:36:01.702298+0000","last_became_active":"2024-04-20T02:16:44.108254+0000","last_became_peered":"2024-04-20T02:16:44.108254+0000","last_unstale":"2024-04-20T02:36:01.702298+0000","last_undegraded":"2024-04-20T02:36:01.702298+0000","last_fullsized":"2024-04-20T02:36:01.702298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:01.702274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:01.702274+0000","last_clean_scrub_stamp":"2024-04-20T02:36:01.702274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:28:10.438468+0000","scrub_duration":0.001434318,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":707,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:07.169419+0000","last_change":"2024-04-20T02:36:07.169520+0000","last_active":"2024-04-20T02:36:07.169419+0000","last_peered":"2024-04-20T02:36:07.169419+0000","last_clean":"2024-04-20T02:36:07.169419+0000","last_became_active":"2024-04-20T02:16:44.142929+0000","last_became_peered":"2024-04-20T02:16:44.142929+0000","last_unstale":"2024-04-20T02:36:07.169419+0000","last_undegraded":"2024-04-20T02:36:07.169419+0000","last_fullsized":"2024-04-20T02:36:07.169419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:07.169393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:07.169393+0000","last_clean_scrub_stamp":"2024-04-20T02:36:07.169393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:27:39.074724+0000","scrub_duration":0.0017524210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":690,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:08.466762+0000","last_change":"2024-04-20T02:36:08.466881+0000","last_active":"2024-04-20T02:36:08.466762+0000","last_peered":"2024-04-20T02:36:08.466762+0000","last_clean":"2024-04-20T02:36:08.466762+0000","last_became_active":"2024-04-20T02:16:44.159975+0000","last_became_peered":"2024-04-20T02:16:44.159975+0000","last_unstale":"2024-04-20T02:36:08.466762+0000","last_undegraded":"2024-04-20T02:36:08.466762+0000","last_fullsized":"2024-04-20T02:36:08.466762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:08.466735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:08.466735+0000","last_clean_scrub_stamp":"2024-04-20T02:36:08.466735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:03:30.274365+0000","scrub_duration":0.0020505950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":695,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:35:57.934692+0000","last_change":"2024-04-20T02:35:57.934803+0000","last_active":"2024-04-20T02:35:57.934692+0000","last_peered":"2024-04-20T02:35:57.934692+0000","last_clean":"2024-04-20T02:35:57.934692+0000","last_became_active":"2024-04-20T02:16:44.131124+0000","last_became_peered":"2024-04-20T02:16:44.131124+0000","last_unstale":"2024-04-20T02:35:57.934692+0000","last_undegraded":"2024-04-20T02:35:57.934692+0000","last_fullsized":"2024-04-20T02:35:57.934692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:35:57.934664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:35:57.934664+0000","last_clean_scrub_stamp":"2024-04-20T02:35:57.934664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T08:22:14.399028+0000","scrub_duration":0.0037138090000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":691,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:05.531558+0000","last_change":"2024-04-20T02:36:05.531661+0000","last_active":"2024-04-20T02:36:05.531558+0000","last_peered":"2024-04-20T02:36:05.531558+0000","last_clean":"2024-04-20T02:36:05.531558+0000","last_became_active":"2024-04-20T02:16:44.141808+0000","last_became_peered":"2024-04-20T02:16:44.141808+0000","last_unstale":"2024-04-20T02:36:05.531558+0000","last_undegraded":"2024-04-20T02:36:05.531558+0000","last_fullsized":"2024-04-20T02:36:05.531558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:05.531530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:05.531530+0000","last_clean_scrub_stamp":"2024-04-20T02:36:05.531530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:40.560229+0000","scrub_duration":0.001370486,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":708,"reported_epoch":697,"state":"active+clean","last_fresh":"2024-04-20T02:36:02.131200+0000","last_change":"2024-04-20T02:36:02.131200+0000","last_active":"2024-04-20T02:36:02.131200+0000","last_peered":"2024-04-20T02:36:02.131200+0000","last_clean":"2024-04-20T02:36:02.131200+0000","last_became_active":"2024-04-20T02:16:44.099559+0000","last_became_peered":"2024-04-20T02:16:44.099559+0000","last_unstale":"2024-04-20T02:36:02.131200+0000","last_undegraded":"2024-04-20T02:36:02.131200+0000","last_fullsized":"2024-04-20T02:36:02.131200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T02:36:02.131080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T02:36:02.131080+0000","last_clean_scrub_stamp":"2024-04-20T02:36:02.131080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T06:05:14.428501+0000","scrub_duration":0.0026259159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607806,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77528,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294312,"statfs":{"total":96636764160,"available":96557375488,"internally_reserved":0,"allocated":6373376,"data_stored":3181863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.53600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.22900000000000001},"max":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":2.9460000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.311,"15min":0.249},"max":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":2.8570000000000002},"last":0.54700000000000004}]},{"osd":1,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.30399999999999999},"max":{"1min":0.71299999999999997,"5min":0.76900000000000002,"15min":2.391},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.311},"max":{"1min":0.67300000000000004,"5min":0.79300000000000004,"15min":1.292},"last":0.53800000000000003}]},{"osd":2,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.39900000000000002,"5min":0.37,"15min":0.30099999999999999},"max":{"1min":0.70499999999999996,"5min":0.747,"15min":1.1899999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.39500000000000002,"5min":0.35499999999999998,"15min":0.30299999999999999},"max":{"1min":0.66000000000000003,"5min":0.81599999999999995,"15min":1.2450000000000001},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.307,"5min":0.27400000000000002,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.65900000000000003,"15min":1.3029999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.42899999999999999,"15min":0.41099999999999998},"min":{"1min":0.27700000000000002,"5min":0.27100000000000002,"15min":0.26400000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.65400000000000003},"last":0.29599999999999999}]},{"osd":5,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.44600000000000001,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.29399999999999998,"15min":0.246},"max":{"1min":0.65300000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27600000000000002},"max":{"1min":0.64100000000000001,"5min":0.71299999999999997,"15min":1.534},"last":0.26900000000000002}]}]},{"osd":3,"up_from":12,"seq":51539607805,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77524,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94294316,"statfs":{"total":96636764160,"available":96557379584,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.22600000000000001},"max":{"1min":0.66800000000000004,"5min":1.0880000000000001,"15min":1.4550000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.312},"max":{"1min":0.71699999999999997,"5min":1.103,"15min":1.397},"last":0.68100000000000005}]},{"osd":1,"last update":"Sat Apr 20 02:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.35099999999999998,"5min":0.307,"15min":0.26000000000000001},"max":{"1min":0.68600000000000005,"5min":1.03,"15min":1.5409999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60299999999999998,"15min":0.58599999999999997},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.318},"max":{"1min":0.745,"5min":1.073,"15min":1.5880000000000001},"last":0.66200000000000003}]},{"osd":2,"last update":"Sat Apr 20 02:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":1.0600000000000001,"15min":1.5589999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.751,"5min":1.1259999999999999,"15min":2.2599999999999998},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Apr 20 02:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.40400000000000003},"min":{"1min":0.27400000000000002,"5min":0.247,"15min":0.24299999999999999},"max":{"1min":0.61199999999999999,"5min":0.64700000000000002,"15min":1.0469999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58999999999999997,"5min":0.71099999999999997,"15min":1.335},"last":0.26800000000000002}]},{"osd":5,"last update":"Sat Apr 20 02:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.32200000000000001,"5min":0.28100000000000003,"15min":0.23100000000000001},"max":{"1min":0.56100000000000005,"5min":0.67200000000000004,"15min":0.90500000000000003},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.34399999999999997,"5min":0.29899999999999999,"15min":0.27900000000000003},"max":{"1min":0.66900000000000004,"5min":0.72299999999999998,"15min":0.95899999999999996},"last":0.318}]}]},{"osd":2,"up_from":12,"seq":51539607805,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69332,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94302508,"statfs":{"total":96636764160,"available":96565768192,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36399999999999999,"15min":0.35899999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23799999999999999},"max":{"1min":0.47699999999999998,"5min":0.52500000000000002,"15min":0.88800000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.38600000000000001,"15min":0.39700000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.54600000000000004,"5min":0.67600000000000005,"15min":0.96899999999999997},"last":0.46200000000000002}]},{"osd":1,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.40500000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.50800000000000001,"5min":0.58099999999999996,"15min":0.97799999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.39800000000000002},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25600000000000001},"max":{"1min":0.57999999999999996,"5min":0.61799999999999999,"15min":1.089},"last":0.499}]},{"osd":3,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.76500000000000001,"5min":0.77000000000000002,"15min":3.823},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.39100000000000001,"15min":0.34499999999999997},"max":{"1min":0.82499999999999996,"5min":1.226,"15min":3.1440000000000001},"last":0.75800000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":1.135},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.2270000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Apr 20 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.36899999999999999},"max":{"1min":0.78900000000000003,"5min":1.079,"15min":1.1919999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.48499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79700000000000004,"5min":0.92000000000000004,"15min":1.2070000000000001},"last":0.73199999999999998}]}]},{"osd":1,"up_from":12,"seq":51539607808,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82144,"kb_used_data":6744,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289696,"statfs":{"total":96636764160,"available":96552648704,"internally_reserved":0,"allocated":6905856,"data_stored":3710791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23400000000000001},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.83699999999999997},"last":0.309},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.30199999999999999,"5min":0.26000000000000001,"15min":0.247},"max":{"1min":0.60399999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Apr 20 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26400000000000001},"max":{"1min":0.496,"5min":0.55400000000000005,"15min":1.698},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42499999999999999},"min":{"1min":0.316,"5min":0.312,"15min":0.26500000000000001},"max":{"1min":0.64500000000000002,"5min":0.65900000000000003,"15min":0.68999999999999995},"last":0.32400000000000001}]},{"osd":3,"last update":"Sat Apr 20 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.373},"max":{"1min":0.71099999999999997,"5min":0.82899999999999996,"15min":1.034},"last":0.496},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56399999999999995,"15min":0.57699999999999996},"min":{"1min":0.38,"5min":0.374,"15min":0.374},"max":{"1min":0.65600000000000003,"5min":0.89300000000000002,"15min":1.7450000000000001},"last":0.51100000000000001}]},{"osd":4,"last update":"Sat Apr 20 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":2.6659999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.66000000000000003},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.40699999999999997},"max":{"1min":0.79100000000000004,"5min":0.93799999999999994,"15min":2.7989999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Sat Apr 20 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.35999999999999999},"max":{"1min":0.82199999999999995,"5min":0.874,"15min":2.6240000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.432,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.754,"5min":0.82199999999999995,"15min":1.282},"last":0.57599999999999996}]}]},{"osd":5,"up_from":11,"seq":47244640509,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82140,"kb_used_data":6740,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94289700,"statfs":{"total":96636764160,"available":96552652800,"internally_reserved":0,"allocated":6901760,"data_stored":3710786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.35899999999999999,"15min":0.247},"max":{"1min":0.61299999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.72299999999999998,"15min":0.86299999999999999},"last":0.628}]},{"osd":1,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.27400000000000002},"max":{"1min":0.65300000000000002,"5min":0.82999999999999996,"15min":0.92300000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53100000000000003,"15min":0.54200000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69499999999999995,"5min":0.75700000000000001,"15min":0.83599999999999997},"last":0.69599999999999995}]},{"osd":2,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67100000000000004,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70699999999999996,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.69299999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.42499999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.22800000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.23899999999999999},"max":{"1min":0.629,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.432,"15min":0.434},"min":{"1min":0.28899999999999998,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.69699999999999995,"15min":0.84399999999999997},"last":0.504}]}]},{"osd":0,"up_from":12,"seq":51539607806,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81620,"kb_used_data":6220,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94290220,"statfs":{"total":96636764160,"available":96553185280,"internally_reserved":0,"allocated":6369280,"data_stored":3181858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77199815},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36299999999999999,"15min":0.35899999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23300000000000001},"max":{"1min":0.51500000000000001,"5min":0.56699999999999995,"15min":0.71699999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.29399999999999998,"5min":0.27700000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.64900000000000002,"15min":1.8440000000000001},"last":0.40500000000000003}]},{"osd":2,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.376,"15min":0.39900000000000002},"min":{"1min":0.26700000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.57799999999999996,"5min":0.63200000000000001,"15min":7.8849999999999998},"last":0.376},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.27500000000000002},"max":{"1min":0.59599999999999997,"5min":0.71799999999999997,"15min":4.9870000000000001},"last":0.438}]},{"osd":3,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.313},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":1.2410000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.2969999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.44400000000000001,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.98899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.47699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.98999999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Sat Apr 20 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.87,"5min":0.87,"15min":1.0069999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.441,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":1.0269999999999999},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T02:36:32.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-20T02:36:32.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.957+0000 7f3cb70f5700 10 monclient: get_monmap_and_config 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.957+0000 7f3cb70f5700 10 monclient: build_initial_monmap 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.957+0000 7f3cb70f5700 10 monclient: monmap: 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:36:32.958885+0000 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:36:32.958885+0000 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:0: v1:172.21.15.47:6789/0 mon.noname-a-legacy 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:1: v1:172.21.15.179:6789/0 mon.noname-f-legacy 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:36:32.958 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:9: v2:172.21.15.47:3300/0 mon.noname-a 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr:10: v2:172.21.15.179:3300/0 mon.noname-f 2024-04-20T02:36:32.959 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.958+0000 7f3cb70f5700 1 Processor -- start 2024-04-20T02:36:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.958+0000 7f3cb70f5700 1 -- start start 2024-04-20T02:36:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.958+0000 7f3cb70f5700 10 monclient: init 2024-04-20T02:36:32.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient: _reopen_session rank -1 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient: _add_conns ranks=[8,3,9,4,6,10,0,2,7,1,5] 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 0x7f3cb00a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): picked mon.noname-e con 0x7f3cb00a0740 addr [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3cb00a0b20 0x7f3cb00a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): picked mon.noname-g con 0x7f3cb00a0b20 addr [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cb00a2950 0x7f3cb00a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): picked mon.noname-a con 0x7f3cb00a2950 addr v2:172.21.15.47:3300/0 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00a14c0 con 0x7f3cb00a2950 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0067410 con 0x7f3cb00a0b20 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 1 -- --> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb009f460 con 0x7f3cb00a0740 2024-04-20T02:36:32.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): _renew_subs 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb70f5700 10 monclient(hunting): authenticate will time out at 2872.360240s 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cb00a2950 0x7f3cb00a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 0x7f3cb00a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3cb00a2950 0x7f3cb00a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52286/0 (socket says 172.21.15.47:52286) 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 1 --2- >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 0x7f3cb00a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3304/0 says I am v2:172.21.15.47:45702/0 (socket says 172.21.15.47:45702) 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 -- 172.21.15.47:0/4010881790 learned_addr learned my addr 172.21.15.47:0/4010881790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): get_auth_request con 0x7f3cb00a2950 auth_method 0 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): get_auth_request con 0x7f3cb00a0740 auth_method 0 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3cb00a0b20 0x7f3cb00a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): get_auth_request con 0x7f3cb00a0b20 auth_method 0 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3caffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient(hunting): handle_auth_done global_id 104607 payload 274 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 -- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 msgr2=0x7f3cb00a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 0x7f3cb00a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 -- 172.21.15.47:0/4010881790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3cb00a0b20 msgr2=0x7f3cb00a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3cb00a0b20 0x7f3cb00a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient: _finish_hunting 0 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 monclient: found mon.noname-a 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.47:3300/0 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 -- 172.21.15.47:0/4010881790 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00751e0 con 0x7f3cb00a2950 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb4e91700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 0x7f3cb00a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.959+0000 7f3cb5692700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb00a2950 0x7f3cb00a2dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009e80 tx=0x7f3ca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c7247a613b9e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 1 -- 172.21.15.47:0/4010881790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ca800e070 con 0x7f3cb00a2950 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: dump: 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.975 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: _finish_auth 0 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: _check_auth_tickets 2024-04-20T02:36:32.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 1 -- 172.21.15.47:0/4010881790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca800bd40 con 0x7f3cb00a2950 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 5 monclient: authenticate success, global_id 104607 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 10 monclient: get_monmap_and_config success 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 4 set_mon_vals no callback set 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 1 -- 172.21.15.47:0/4010881790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ca8012860 con 0x7f3cb00a2950 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3caf7fe700 10 monclient: dump: 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.977 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 10 monclient: shutdown 2024-04-20T02:36:32.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb00a2950 msgr2=0x7f3cb00a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb00a2950 0x7f3cb00a2dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009e80 tx=0x7f3ca800b330 comp rx=0 tx=0).stop 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4010881790 shutdown_connections 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3cb00a2950 0x7f3cb00a2dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] conn(0x7f3cb00a0740 0x7f3cb00a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4010881790 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3cb00a0b20 0x7f3cb00a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4010881790 >> 172.21.15.47:0/4010881790 conn(0x7f3cb00a7570 msgr2=0x7f3cb00a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4010881790 shutdown_connections 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.960+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4010881790 wait complete. 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 10 monclient: build_initial_monmap 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 10 monclient: monmap: 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 0 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 0.000000 2024-04-20T02:36:32.979 INFO:teuthology.orchestra.run.smithi047.stderr:created 0.000000 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 0 (unknown) 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 1 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.noname-a 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.noname-f 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.noname-b 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.noname-g 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.noname-c 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.noname-h 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.noname-d 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.noname-i 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.noname-e 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 1 librados: starting msgr at 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 1 librados: starting objecter 2024-04-20T02:36:32.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 1 Processor -- start 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 1 -- start start 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 1 librados: setting wanted keys 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 1 librados: calling monclient init 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.961+0000 7f3cb70f5700 10 monclient: init 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient: _reopen_session rank -1 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient: _add_conns ranks=[1,6,2,3,5,4,7,8,0] 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb00a0740 0x7f3cb01cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): picked mon.noname-f con 0x7f3cb00a0740 addr [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 0x7f3cb01bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): picked mon.noname-d con 0x7f3cb00a0b20 addr [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 0x7f3cb01bc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): picked mon.noname-b con 0x7f3cb00a2950 addr [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01cd180 con 0x7f3cb00a0740 2024-04-20T02:36:32.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01aaf90 con 0x7f3cb00a2950 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): start opening mon connection 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 1 -- --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb009e3d0 con 0x7f3cb00a0b20 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): _renew_subs 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb70f5700 10 monclient(hunting): authenticate will time out at 2872.363624s 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 0x7f3cb01bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 0x7f3cb01bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 1 --2- >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 0x7f3cb01bbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3303/0 says I am v2:172.21.15.47:37152/0 (socket says 172.21.15.47:37152) 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 0x7f3cb01bc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:41864/0 (socket says 172.21.15.47:41864) 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 1 -- 172.21.15.47:0/4096288675 learned_addr learned my addr 172.21.15.47:0/4096288675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): get_auth_request con 0x7f3cb00a0b20 auth_method 0 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): get_auth_request con 0x7f3cb00a2950 auth_method 0 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb00a0740 0x7f3cb01cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): get_auth_request con 0x7f3cb00a0740 auth_method 0 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): _init_auth method 2 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): _init_auth creating new auth 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3caffff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb5692700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.962+0000 7f3cb4e91700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 10 monclient(hunting): handle_auth_done global_id 55743 payload 824 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 -- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 msgr2=0x7f3cb01bc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 0x7f3cb01bc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 -- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb00a0740 msgr2=0x7f3cb01cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb00a0740 0x7f3cb01cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 10 monclient: _finish_hunting 0 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 monclient: found mon.noname-d 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.47:3303/0 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 -- 172.21.15.47:0/4096288675 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca40094a0 con 0x7f3cb00a0b20 2024-04-20T02:36:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb5692700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 0x7f3cb01bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb4e91700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb00a0740 0x7f3cb01cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caffff700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 0x7f3cb01bbe50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00e980 tx=0x7f3c9c00ec90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 client.?.objecter ms_handle_connect 0x7f3cb00a0b20 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 client.?.objecter resend_mon_ops 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9c009710 con 0x7f3cb00a0b20 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: dump: 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: _finish_auth 0 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: _check_auth_tickets 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-04-20T02:36:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c00cb00 con 0x7f3cb00a0b20 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 5 monclient: authenticate success, global_id 55743 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 10 monclient: _renew_subs 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca8009b30 con 0x7f3cb00a0b20 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: handle_config config(0 keys) v1 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3c9c00cc70 con 0x7f3cb00a0b20 2024-04-20T02:36:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3caeffd700 4 set_mon_vals no callback set 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 10 client.55743.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cad7fa700 10 monclient: dump: 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:epoch 1 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:fsid 72abca9e-d588-4ab7-bb3a-a08105100daf 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:last_changed 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:created 2024-04-20T02:15:04.177248+0000 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:min_mon_release 18 (reef) 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:election_strategy: 3 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T02:36:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.f 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:3: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.g 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:4: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:5: [v2:172.21.15.179:3302/0,v1:172.21.15.179:6791/0] mon.h 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:6: [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] mon.d 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:7: [v2:172.21.15.179:3303/0,v1:172.21.15.179:6792/0] mon.i 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:8: [v2:172.21.15.47:3304/0,v1:172.21.15.47:6793/0] mon.e 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 10 monclient: _renew_subs 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb01ab830 con 0x7f3cb00a0b20 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.963+0000 7f3cb70f5700 1 librados: init done 2024-04-20T02:36:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f3c9c00c690 con 0x7f3cb00a0b20 2024-04-20T02:36:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cb70f5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cb70f5700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cb70f5700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:36:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T02:36:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb006aac0 con 0x7f3cb00a0b20 2024-04-20T02:36:32.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 5 ==== osd_map(697..697 src has 621..697) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f3c9c04a490 con 0x7f3cb00a0b20 2024-04-20T02:36:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 10 client.55743.objecter ms_dispatch 0x7f3cb0001550 osd_map(697..697 src has 621..697) v4 2024-04-20T02:36:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 3 client.55743.objecter handle_osd_map got epochs [697,697] > 0 2024-04-20T02:36:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 3 client.55743.objecter handle_osd_map decoding full epoch 697 2024-04-20T02:36:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cad7fa700 20 client.55743.objecter dump_active .. 0 homeless 2024-04-20T02:36:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.964+0000 7f3cb4e91700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T02:36:32.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.965+0000 7f3cb4e91700 10 monclient: get_auth_request con 0x7f3c9403a2f0 auth_method 0 2024-04-20T02:36:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.965+0000 7f3cb4e91700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 secure :-1 s=READY pgs=3428 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40114b0 tx=0x7f3ca4009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T02:36:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.965+0000 7f3cad7fa700 10 client.55743.objecter ms_handle_connect 0x7f3c9403a2f0 2024-04-20T02:36:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.969+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f3c9c013470 con 0x7f3cb00a0b20 2024-04-20T02:36:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.969+0000 7f3cad7fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-04-20T02:36:32.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:32.969+0000 7f3cad7fa700 10 monclient: _finish_command 1 = system:0 2024-04-20T02:36:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.187+0000 7f3cb70f5700 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-04-20T02:36:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.187+0000 7f3cb70f5700 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-04-20T02:36:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.187+0000 7f3cb70f5700 10 monclient: _send_mon_message to mon.d at v2:172.21.15.47:3303/0 2024-04-20T02:36:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.187+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 --> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3cb01a9db0 con 0x7f3cb00a0b20 2024-04-20T02:36:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.192+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3c9c013470 con 0x7f3cb00a0b20 2024-04-20T02:36:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.192+0000 7f3cad7fa700 10 monclient: handle_config config(1 keys) v1 2024-04-20T02:36:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.192+0000 7f3caeffd700 4 set_mon_vals no callback set 2024-04-20T02:36:33.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.192+0000 7f3caeffd700 10 set_mon_vals mon_health_to_clog = false 2024-04-20T02:36:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.193+0000 7f3cad7fa700 1 -- 172.21.15.47:0/4096288675 <== mon.6 v2:172.21.15.47:3303/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v4)=0 v4) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3c9c01b3b0 con 0x7f3cb00a0b20 2024-04-20T02:36:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.193+0000 7f3cad7fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-04-20T02:36:33.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.193+0000 7f3cad7fa700 10 monclient: _finish_command 2 = system:0 2024-04-20T02:36:33.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 10 librados: watch_flush enter 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 10 librados: watch_flush exit 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 20 client.55743.objecter shutdown clearing up homeless session... 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 10 client.55743.objecter successfully canceled tick 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 msgr2=0x7f3c9403c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 secure :-1 s=READY pgs=3428 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40114b0 tx=0x7f3ca4009210 comp rx=0 tx=0).stop 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 10 monclient: shutdown 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 msgr2=0x7f3cb01bbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 0x7f3cb01bbe50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00e980 tx=0x7f3c9c00ec90 comp rx=0 tx=0).stop 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 shutdown_connections 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:6824/41706,v1:172.21.15.179:6825/41706] conn(0x7f3c9403a2f0 0x7f3c9403c7b0 unknown :-1 s=CLOSED pgs=3428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cb00a0740 0x7f3cb01cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3cb00a2950 0x7f3cb01bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:33.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 --2- 172.21.15.47:0/4096288675 >> [v2:172.21.15.47:3303/0,v1:172.21.15.47:6792/0] conn(0x7f3cb00a0b20 0x7f3cb01bbe50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T02:36:33.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 >> 172.21.15.47:0/4096288675 conn(0x7f3cb00a7570 msgr2=0x7f3cb0072370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T02:36:33.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 shutdown_connections 2024-04-20T02:36:33.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 -- 172.21.15.47:0/4096288675 wait complete. 2024-04-20T02:36:33.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T02:36:33.196+0000 7f3cb70f5700 1 librados: shutdown 2024-04-20T02:36:33.210 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T02:36:33.210 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T02:36:33.211 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T02:36:33.211 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.251 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T02:36:33.251 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T02:36:33.251 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.327 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T02:36:33.327 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T02:36:33.327 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.397 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T02:36:33.397 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T02:36:33.397 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.441 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T02:36:33.442 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T02:36:33.442 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.483 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T02:36:33.483 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T02:36:33.483 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.518 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T02:36:33.518 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T02:36:33.519 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T02:36:33.519 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.585 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T02:36:33.585 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T02:36:33.586 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T02:36:33.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.601 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T02:36:33.602 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T02:36:33.602 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.657 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T02:36:33.658 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T02:36:33.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.716 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T02:36:33.717 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-04-20T02:36:33.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.777 INFO:tasks.ceph.mon.d:Stopped 2024-04-20T02:36:33.778 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-04-20T02:36:33.778 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.839 INFO:tasks.ceph.mon.e:Stopped 2024-04-20T02:36:33.840 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-04-20T02:36:33.840 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.857 INFO:tasks.ceph.mon.f:Stopped 2024-04-20T02:36:33.857 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-04-20T02:36:33.857 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.911 INFO:tasks.ceph.mon.g:Stopped 2024-04-20T02:36:33.911 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-04-20T02:36:33.911 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:33.966 INFO:tasks.ceph.mon.h:Stopped 2024-04-20T02:36:33.966 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-04-20T02:36:33.967 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T02:36:34.019 INFO:tasks.ceph.mon.i:Stopped 2024-04-20T02:36:34.019 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T02:36:34.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(MON_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'slow request' | head -n 1 2024-04-20T02:36:34.066 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:36:34.067 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T02:36:34.290 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:36:34.290 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T02:36:34.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:36:34.364 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T02:36:34.461 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:36:34.462 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T02:36:34.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:36:34.625 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T02:36:34.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:36:34.697 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T02:36:34.772 INFO:tasks.ceph:Archiving mon data... 2024-04-20T02:36:34.774 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.a.tgz 2024-04-20T02:36:34.774 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T02:36:34.796 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.IPreT912jL 2024-04-20T02:36:34.797 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IPreT912jL 2024-04-20T02:36:35.063 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.IPreT912jL 2024-04-20T02:36:35.158 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.IPreT912jL is 1MB 2024-04-20T02:36:35.267 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.IPreT912jL 2024-04-20T02:36:35.289 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.b.tgz 2024-04-20T02:36:35.289 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T02:36:35.350 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.YtvcYBsVCP 2024-04-20T02:36:35.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YtvcYBsVCP 2024-04-20T02:36:35.607 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.YtvcYBsVCP 2024-04-20T02:36:35.699 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.YtvcYBsVCP is 1MB 2024-04-20T02:36:35.809 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.YtvcYBsVCP 2024-04-20T02:36:35.830 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.c.tgz 2024-04-20T02:36:35.830 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T02:36:35.891 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.MTD8cz20Fp 2024-04-20T02:36:35.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MTD8cz20Fp 2024-04-20T02:36:36.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.MTD8cz20Fp 2024-04-20T02:36:36.240 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.MTD8cz20Fp is 1MB 2024-04-20T02:36:36.324 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.MTD8cz20Fp 2024-04-20T02:36:36.345 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.d.tgz 2024-04-20T02:36:36.345 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T02:36:36.406 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.HRPAU7SkjZ 2024-04-20T02:36:36.406 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.HRPAU7SkjZ 2024-04-20T02:36:36.665 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.HRPAU7SkjZ 2024-04-20T02:36:36.758 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.HRPAU7SkjZ is 1MB 2024-04-20T02:36:36.842 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.HRPAU7SkjZ 2024-04-20T02:36:36.863 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.e.tgz 2024-04-20T02:36:36.863 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T02:36:36.923 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.u94Pnnhl12 2024-04-20T02:36:36.923 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.u94Pnnhl12 2024-04-20T02:36:37.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.u94Pnnhl12 2024-04-20T02:36:37.278 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.u94Pnnhl12 is 1MB 2024-04-20T02:36:37.378 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.u94Pnnhl12 2024-04-20T02:36:37.400 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.f.tgz 2024-04-20T02:36:37.400 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-04-20T02:36:37.421 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.znnpNOBMP1 2024-04-20T02:36:37.421 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.znnpNOBMP1 2024-04-20T02:36:37.742 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.znnpNOBMP1 2024-04-20T02:36:37.828 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.znnpNOBMP1 is 1MB 2024-04-20T02:36:37.934 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.znnpNOBMP1 2024-04-20T02:36:37.951 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.g.tgz 2024-04-20T02:36:37.951 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-04-20T02:36:38.007 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.YPinJNftpE 2024-04-20T02:36:38.007 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.YPinJNftpE 2024-04-20T02:36:38.239 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.YPinJNftpE 2024-04-20T02:36:38.325 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.YPinJNftpE is 1MB 2024-04-20T02:36:38.431 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.YPinJNftpE 2024-04-20T02:36:38.448 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.h.tgz 2024-04-20T02:36:38.449 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-04-20T02:36:38.505 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.vaEKd6Q0An 2024-04-20T02:36:38.505 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.vaEKd6Q0An 2024-04-20T02:36:38.742 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.vaEKd6Q0An 2024-04-20T02:36:38.827 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.vaEKd6Q0An is 1MB 2024-04-20T02:36:38.934 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.vaEKd6Q0An 2024-04-20T02:36:38.951 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/data/mon.i.tgz 2024-04-20T02:36:38.951 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-04-20T02:36:39.007 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.BEf9Zz5ezx 2024-04-20T02:36:39.007 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.BEf9Zz5ezx 2024-04-20T02:36:39.238 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.BEf9Zz5ezx 2024-04-20T02:36:39.324 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.BEf9Zz5ezx is 1MB 2024-04-20T02:36:39.430 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.BEf9Zz5ezx 2024-04-20T02:36:39.447 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T02:36:39.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T02:36:39.450 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T02:36:39.513 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-20T02:36:39.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-20T02:36:39.524 DEBUG:teuthology.orchestra.run.smithi179:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-20T02:36:39.558 INFO:teuthology.orchestra.run.smithi047.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-20T02:36:39.580 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-20T02:36:39.586 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T02:36:39.588 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/remote/smithi047/crash 2024-04-20T02:36:39.589 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T02:36:39.640 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/remote/smithi179/crash 2024-04-20T02:36:39.641 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T02:36:39.669 INFO:tasks.ceph:Compressing logs... 2024-04-20T02:36:39.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T02:36:39.687 DEBUG:teuthology.orchestra.run.smithi179:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T02:37:06.614 INFO:tasks.ceph:Archiving logs... 2024-04-20T02:37:06.615 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/remote/smithi047/log 2024-04-20T02:37:06.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T02:37:10.130 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/remote/smithi179/log 2024-04-20T02:37:10.131 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T02:37:13.535 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T02:37:13.545 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T02:37:13.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T02:37:13.549 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T02:37:13.636 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T02:37:13.646 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T02:37:13.646 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T02:37:13.649 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T02:37:13.666 INFO:teuthology.orchestra.run.smithi047.stderr:bash: ntpq: command not found 2024-04-20T02:37:13.667 INFO:teuthology.orchestra.run.smithi179.stderr:bash: ntpq: command not found 2024-04-20T02:37:13.672 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T02:37:13.672 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-20T02:37:13.672 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 -1182us[-1206us] +/- 75ms 2024-04-20T02:37:13.672 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -652us[ -676us] +/- 68ms 2024-04-20T02:37:13.672 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +1880us[+1856us] +/- 63ms 2024-04-20T02:37:13.672 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:37:13.673 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T02:37:13.673 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-04-20T02:37:13.673 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 48 -1227us[-1142us] +/- 75ms 2024-04-20T02:37:13.673 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -681us[ -596us] +/- 68ms 2024-04-20T02:37:13.673 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +1815us[+1815us] +/- 63ms 2024-04-20T02:37:13.674 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T02:37:13.675 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T02:37:13.685 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T02:37:13.686 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T02:37:13.695 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T02:37:13.799 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T02:37:13.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T02:37:13.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T02:37:13.905 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T02:37:13.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T02:37:13.935 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T02:37:13.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T02:37:13.958 INFO:teuthology.task.internal:Duration was 2511.314384 seconds 2024-04-20T02:37:13.958 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T02:37:13.968 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T02:37:13.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T02:37:13.970 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T02:37:14.023 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T02:37:14.029 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T02:37:14.207 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T02:37:14.207 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T02:37:14.208 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T02:37:14.237 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-04-20T02:37:14.237 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T02:37:14.258 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T02:37:14.259 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T02:37:14.280 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T02:37:14.341 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T02:37:14.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T02:37:14.344 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T02:37:14.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T02:37:14.467 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T02:37:14.467 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T02:37:14.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T02:37:14.526 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T02:37:14.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T02:37:14.542 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T02:37:14.572 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-04-20T02:37:14.594 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-04-20T02:37:14.606 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T02:37:14.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T02:37:14.655 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T02:37:14.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T02:37:14.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T02:37:14.682 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T02:37:14.682 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/remote/smithi047 2024-04-20T02:37:14.683 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T02:37:14.744 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664191/remote/smithi179 2024-04-20T02:37:14.744 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T02:37:14.783 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T02:37:14.783 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T02:37:14.786 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T02:37:14.834 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T02:37:14.847 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T02:37:14.847 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T02:37:14.861 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T02:37:14.862 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T02:37:14.864 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T02:37:14.883 INFO:teuthology.orchestra.run.smithi047.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 02:37 /home/ubuntu/cephtest 2024-04-20T02:37:14.890 INFO:teuthology.orchestra.run.smithi179.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 02:37 /home/ubuntu/cephtest 2024-04-20T02:37:14.892 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T02:37:14.902 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T02:37:14.928 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} duration: 2511.314384460449 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-04-20T02:37:14.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T02:37:15.019 INFO:teuthology.run:pass